The ThinkND Podcast

Evidence Matters, Part 5: Co-Designing for Equity

• Think ND

Share episode

Use Left/Right to seek, Home/End to jump to start or end. Hold shift to jump forward or backward.

0:00 | 1:10:44

Episode Topic: Co-Designing for Equity

Explore how antiracist principles can lead to stronger research, more meaningful partnerships, and smarter service delivery. Drawing from on-the-ground experience, real policy implementation, and the behavioral science insights in Antiracist by Design, uncover a framework and practical tools for advancing equity through evidence.

Featured Speakers:

  • Carrie Cihak, King County, Washington
  • Laura Feeney, J-PAL North America
  • Matthew Freedman, University of California, Irvine
  • Sunny Giron, Community Café Collective
  • Crystal C. Hall, University of Washington
  • Ann McNair, MPH, Best Starts for Kids

Read this episode's recap over on the University of Notre Dame's open online learning community platform, ThinkND: https://go.nd.edu/5011ca.

This podcast is a part of the ThinkND Series titled Evidence Matters. 

Thanks for listening! The ThinkND Podcast is brought to you by ThinkND, the University of Notre Dame's online learning community. We connect you with videos, podcasts, articles, courses, and other resources to inspire minds and spark conversations on topics that matter to you — everything from faith and politics, to science, technology, and your career.

  • Learn more about ThinkND and register for upcoming live events at think.nd.edu.
  • Join our LinkedIn community for updates, episode clips, and more.

Welcome and Introduction

1

On behalf of King County, the Wilson Sheehan Lab for Economic opportunities at Notre Dame and j Powell North America, I wanna give a big welcome to everybody to our Evidence Matters series. It's great to see so many of our friends and colleagues from around the country here. Thanks for joining us and we're recording this webinar and we're going to air it at a later date on King County Television. Great to see you, Maya from Results for America. Thanks for all your support. So welcome to our local television audience who's joining us too. I'm Kerry Hawk. And I'm King County's Evidence and Impact Officer. I'm passionate about advancing racial equity through bringing together the expertise of our staff, our research partners, and our community. And this Evidence Matters webinar series is a partnership between King County Leo, which is the Lab for Economic Opportunities at Notre Dame. Jal and we're all organizations who are working in complimentary ways to fight poverty and increase access to economic opportunity. And so I'm really excited to be here today with my co-host, Laura Feeney. Laura's the co-director at JAL North America. Hi Laura. Tell us a little bit about JAL and introduce yourself.

2

Hi, I'm so excited to be here today. So my name is Laura Feeney, co-Executive Director at JAL North America. JAL is the Abdu Jamil Poverty Action Lab. We are based at MIT. We're a research and policy center aiming to address poverty and challenges like inequality by using research to identify effective solutions, and then making sure that evidence reaches the right audiences, whether that's policy makers, program implementers, decision makers, to be able to make the world that we're in a better place for everyone. We also aim to equip the research and the policy communities with the tools and skills needed to strengthen their approach to doing that research in the first place. To ask better questions, get better answers, and create better policy. So in doing this, we've had partnerships with Care with King County, with Leo, and many others over the years to study the effectiveness of different programs and different approaches in areas from homelessness, prevention to transportation and beyond. So we're excited to be here and have this conversation with all.

1

Yeah. Thanks Laurie. You've been a really great partner to us at King County and to all the other local governments who are on our call today. Really encourage you to check out jal's website. There's a host of great resources to support you and researchers as well, and we've really partnered King County, Leo jal have built this partnership around evidence matters to support the following. First, we wanna share our knowledge. That we're building from these learning partnerships so we can use that to advance better and more equitable outcomes in our communities. We definitely wanna share, celebrate our successes, and we wanna share our challenges too. And mostly we would just wanna build connection with all of you tuning in from around the country so that we can learn from one another. And today I'm super excited. We're gonna dive into learnings from a new book called Anti-Racist by Design, Re-Imagining Applied Behavioral Science. And we have with us one of the co-authors of the book, crystal Hall. And the book is really a call to action and we're delighted to have Crystal here. In addition to authoring the book, Crystal's a professor right here at the University of Washington in Seattle at the Evans School of Public Policy. Crystal, can you give us a quick hello?

3

Yes. Hi everybody. Thank you for joining. Thank you for organizing this amazing session. Carrie and Laura and I have to give a special hello. I see several friends and alums of the Evan School, so I just wanted to give a shout out and say thanks for joining.

4

Looking forward to the conversation. Great. So I'll hold up my copy too.

2

we're really excited to talk to Crystal and hear more from you about what inspired the book and especially go through some of the roadmap that Crystal and her co-author lay out. Speaking just from me and from the JAL side of things. This book was so accessible and clearly written. I think it's really gonna help our team of research staff and our network of researchers think about how to push forward in designing research that is more equitable and inclusive in its methods and able to. Find evidence and find policies that will increase racial equity throughout this country. So we're excited to keep talking.

1

Yeah. I really wanna underscore your point about the book being so accessible, Laura. I'd say you don't need a background in behavioral science to really get a lot out of this book. It's, and I think it also speaks not just to, you know, behavioral science on its own, but more generally to evaluation, testing, design of government programs and innovation more generally. I'd say,

2

yeah. I think one of the things that stuck out to me is that we can improve how services and outcomes work for everyone by focusing not just on how individuals behave within a system, which is often how behavioral science purchasing, but on how the system itself works or how it isn't working or how it's changing what individual behavior could look like. Because often it, it's the system or that interaction that's working in ways that create these really. Harmful and disparate and racist outcomes.

1

Yeah, I think in the book, crystal uses the metaphor of a game and what we're really trying to do is not just help people learn how to play the game that exists, but really how do we redesign and actually we should use the word co-design a game that's more, that's equitable and works better for everybody. So I really love that, that about the book. I think one thing we wanted to acknowledge is that we're having this conversation about evidence and equity and an environment in which those concepts have really been under attack, particularly at the federal level. And I don't want us to get wrapped up around the language if some of that language is unfamiliar to you. I think what you're gonna see in this conversation is that much of what we're really talking about, about applying anti-racist principles to the delivery of governance service. It's just about making government work better for all of us. It's about good government practice, and that's why I think conversations like this continue to be so important and relevant today in this environment.

2

Yeah, and I think if you say, Carrie, this is about good government and just making government work better for all of us. On the research side where j Powell and I usually come from, this is similarly, especially from reading through the book, this is about doing just good and robust research that considers different angles. Investigates what do we know and what don't we know and what don't we know that we don't know. And that's, that's just good research. It's not fundamentally a huge scary shift or it doesn't have to be so scary. It's just about, yeah, doing the things that researchers do really well, but doing it a little bit better and a little bit differently. and those changes can.

1

Thank you for that Laura. I think that's exactly right and I think this is also a great point to have our other guests say hello too. We have with us Anne McNair, who's one of King County Public Health's social science researchers, and Anne's been leading our evaluation work under Best Starts for Kids. And that's a voter improved initiative here in King County that supports every child in our region to grow into adulthood healthy, happy, safe and thriving. Ann, do you wanna say hello

5

to us? Thanks, Carrie. Hi everyone. It's great to have you here today. I'm honored to be part of this discussion.

1

Discussion. It's great to have you here, Anne. And we've also got Sunny Hiran who is with us, and Sunny is one of our community partners at King County. She leads the Community Cafe collaborative, and that's an organization that hosts conversations in communities across the country. So that all that rich knowledge that really exists in our neighborhoods gets translated into action. And King County is really fortunate to have worked with Sonny and the Community Cafe Collaborative to bring that community knowledge directly into government service. Do you wanna say hello, Sonny?

6

Hello everybody. Thank you for having me and inviting in the community perspective for this conversation.

4

Absolutely. It's

1

so important.

2

and then last but not least, we have Matt Friedman, professor of Economics at the University of California at Irvine. Matt's been working with Leo, with JAL, with King County and in this space for about eight years on a variety of projects with a, a lot of themes around improving transportation equity in Seattle region. So Matt, would you say hello?

7

Thank you so much, Laura. Thank you everybody for being here. It's really fantastic to be part of this conversation.

2

It's so great to have you. So let's dive in. And Crystal, let's start hearing a bit more from you. Well, zoom does all kinds of things with the focus. So Crystal, I really loved how in the book you weaved in your and Mindy's personal experience and your research experience as social scientists throughout the book, it really brought the work to life. It was fun and engaging, and I could see how it was happening in action. So can you start sharing with us some of the themes that you felt in your work, the discomfort that you felt in your work that led you to think about whether why this book was necessary and how it came around? How did you start seeing the gap between the promise of behavioral science and what you were trying to achieve and the reality?

3

Yeah. Thank you. I love starting with this question and just this approach to discussing the book because I think it really helps. How this was something that was not something that we suddenly realized. And it was also something for Mindy and I that was really quite personal, which I think is worth acknowledging in a space like this where we are talking about evidence and policy and these things at a high level. And I have to give a shout out to Mindy Hernandez, who is my partner in this work. And I think we just really complimented each other. I wanna make sure that I acknowledge that. So there, there were a few things that were really important to Mindy and I that we reflected on as we started this work. One was just a continued lack of representation that we found not only about relative to our racial identity, and we are two women of color, but also just the different types of life experience related to that. And often we, for me personally, I got into this work thinking about decision making in the context of poverty and always for years and years, not really seeing. Us acknowledge the intersection of racial identity and racism and how that intersects with people that experience the experiences of people that live in poverty. And so that fundamental lack of representation has a clear impact on how we approach any research question, or in my case, often the design of a behavioral intervention. And the other piece that we started thinking about a lot was the fact that we also don't evaluate our work with a lot of nuance. And I would say in the book, all the criticisms that we give of the field and of the work, Mindy and I both recognize that we've been guilty of, right? We've, we design an intervention, we evaluate that intervention, and we're not thinking deeply enough about different impacts that might be felt by different subgroups. That population. So while we might say, yes, this intervention increased the rate of savings, we don't look at, was that different for people of color versus others? Was that different based on gender? Was that different based on where people live? Those types of things. And so we were thinking about that and then also just thinking about the fact that really we weren't authentically engaging with the communities that are most impacted by our work at all stages of the process, which is something that I know we'll come back to. And I think I'm trained as a social psychologist. And I think all of these factors really tie to the fact that as a psychologist, most and most people that identify as behavioral scientists more broadly, we have this very, very myopic focus on the individuals individual behavior, individual psychology, without thinking really deeply about the role that structures and systems play. And this is something that I think we, we acknowledge that the individual matters, but the system matters as well. And we just weren't doing a good job of that. And Mindy and I have been friends and collaborators for a long time, and this really came to a head in the summer of 2020 when the world was grappling with the reality of racism, not only in the US but in other places. And it was showing up in our own work. And that was when we started writing. And we tried to put some of this down and we tried to just try to externalize a lot of the things we'd been feeling for years and years, but not really finding the courage, frankly, to deal with. And we really wanted to make this argument that we don't have to abandon this exploration of the individual to also think about systems and that we can do both. And it's hard for sure, but it's not impossible. And that we really, we have to. Acknowledge both if we wanna be more effective in the work that we do.

Challenges and Solutions in Anti-Racist Research

1

Yeah. Thank you for that, crystal. I really enjoyed reading the book and following along with your and Mindy's journey. It definitely resonated with me too. And part of what I really hear you saying in the book and hearing this conversation is that as social scientists and government, we have a tendency to really put the onus on individuals and the incentives that we design around individual behavior. And we're not really putting enough focus on changing the systems that are holding people back. And it's really those systems that have probably the biggest influence on people's behavior. And if those systems are inequitable, which we know that they are from all of the evidence, then we end up with inequitable outcomes. And the other thing that strikes me is that. If we continue to focus on individual behavior, we're just not serving everybody as well as we can be. And in fact, if we're not taking into account those, the inequity in the systems then nudges that are designed to help people work in the system better, can actually make outcomes more inequitable. And I've seen that in some of our work over time at King County. And so that basically what that means is there's like big groups of people who government isn't serving very well.

3

Yeah, that's exactly right. And one of the examples that we use in the book that I think illustrates this particularly clearly is the way that we've often thought about access to post-secondary education and access to funding for education. And there's research showing, and this is an example that we use in the book. There, there's research that has shown that when students that are in the process of applying to college are provided with various types of support to help them complete applications for financial aid, they tend to get more, they get more funding. And the idea being that this is a benefit to all students to be able to access student loans. And in theory, that makes sense and that makes sense. Only if we continue to look downstream to say what are the outcomes and are there systematic differences that are still present in the experiences of different students. And one thought experiment that we can engage in is the idea that students get more access to funding. That is a good thing as we look at that particular outcome. We also know that students of co color are often less likely to complete their college degree. And so one possibility, and this is an empirical question that I don't have the answer to, but one, one question is, does getting more students to take out loans if we don't decrease the barriers to completion, are we increasing the disparity of debt? Right? Because now we may have more students taking on debt and not finishing school than we would otherwise, right? And that's not necessarily what we might see, but the research that's been done doesn't allow us to examine that. And so that's a place where we're nudging people to engage in the system as it is, but we're not really addressing some of the bigger issues that may be impacting the ultimate goal that I would argue that we're interested in, which is completion of a degree, right? And so we use this example as one that just illustrates the fact that when we really narrow in on that nudge. We're doing it at the expense of thinking about some of the bigger causes of some of the issues that we're ultimately trying to address.

2

So what I hear through this is one of the things we need to do is really make sure we're understanding and thinking through the full chain of how a policy is working or how nudge is working and what we're actually aiming towards, and then how people are engaging in that system. So this is beyond just the nudge at hand or the individual action at hand and kind of what is the full chain of what we're trying to do. And that knowledge helps us design the research or even say, here's what my research can address and here's what you, heres are some, a follow on can add. And then how design the policy or the program. And so again, to me, this is about good research. This isn't necessarily a fundamentally different kind of research. It's just a broader approach. So I've really appreciated that. And the way that you acknowledge. Here and in the book that this is a learning journey that you're on too. It's not that suddenly you become an expert in this overnight.

1

Yeah, for sure. I think it's not only good research, it's good government, right? I think about the work that we've done with Matt around transportation in the Seattle region, access to mobility, and we can design things around individual incentives like reducing fares. But if you don't have a transit stop in your neighborhood, having access to a lower fair doesn't help you as much. And surely the surely it's not about, oh, we're just not gonna make the student loan application easier. It's really about doing that work, but also addressing those underlying system barriers that lead to different impacts for different groups of people. And so what I really like about the book, crystal, is that you also present this roadmap. How we can start thinking about that. So I'd love to have you give us an overview of that roadmap, and we've got a graphic here that I think can help people see that and visualize a path to making research and government service better. So Crystal, you wanna walk us through it?

3

Sure. So this is an image that comes from the book. And the first thing I wanna say is that, this is actually two things I'll say before I walk through it quickly. One is that this is a process that is very iterative. So we have this, it looks like a straight line, but I think as I briefly explain each of these, just want to acknowledge that often there's a lot of, there's a lot of cycling through and going back. And I think that's part of what it means to, to engage in this process authentically. And then the second thing that I really wanna highlight, because I think it, it can't be. Stated enough. It is Laura's point about this is about doing good work. It's about doing good research or designing good policy. And I think especially in the political and cultural moment that we're in right now, I think that can get lost. And when I think about this work, I think you can make an argument about the social justice implications about focusing on racism or other types of bias. For sure. And that's important. And it's also important to acknowledge for us that work in the world of public policy, this is about being more effective with the work that we do. And I think it's so important to acknowledge that when we don't do these things, when we don't work with the communities that. We are intending to serve and support, we run the risk of throwing resources in directions that are gonna be less effective. So I just wanna, I wanna highlight that, and I know that in the conversation that we'll have with Sunny and Matt and Anne, that this point will come up again. But I just think it's really worth highlighting that again now, is that this is about being effective in what we do. And with this map, we took a traditional process that's typically seen in the applied behavioral science world and we wanted to update it to think about if we want to actively engage in anti-racist processes, what does that look like? What does that mean about how we change? And the theme that you'll see as I go through this is just the idea that we work with the communities that are most impacted and we add first at right up front this idea of preparing. And this is the idea that we need to set a goal. We need to set intentional goals. About how we want to move in pursuit of anti-racist practices in these spaces. So sometimes it's about the staff that are on board. Sometimes it's about actively thinking about the blind spots of the research team or others involved. And sometimes it's about things like really focusing on what's the system centered mindset. And this is again, something that I think we'll we can talk more about later. And then we talk about this idea that instead of the researcher or the person with the most power defining the problem, quote unquote, it's about partnering to think about what is the challenge, who is involved, and making sure that the way that the researcher is understanding the problem is consistent with those that are in that community. And it was a similar theme as we moved to this idea of co-discovery and co-design understanding. The factors that have an influence on behavior, both at an individual level and at a systemic level, designing an intervention or an evaluation or a program with the partners that make sense of subpopulations that be, should be considered other constraints. And this is something I would say as a researcher, we often think that we do this or we think we do this adequately enough, right? Where we, maybe we design something or often we already know what the thing is that we wanna test, and then we say, okay, but we'll run it by the population. That's fundamentally different than working with them to design from start to finish. And then in these last two steps, we talk about this idea of how do we actually implement the idea that we may come up with a program that's effective, but if we don't work together to think about implementation, we may make some big mistakes. And similarly, when we think about sharing the results. And scaling up. That's another place where we have to think about the audience and working with them. And one quick example that I'll give before, before I pause is work that Mindy and I did together working in volunteer income tax assistance sites, where we were doing some of these classic approaches to nudge people towards saving a portion of their tax refund. And this is a place where like every mistake that I can point to, we made in the process of doing this work where we came in and we thought we knew what the problem was, we thought we understood the context. I'm sure that as we went through our version of a discovery process, often behavioral scientists like to use the language of diagnosing rather than discovering, which we argue is incredibly problematic. We probably had a very narrow sense of what was really the true challenge, which meant that what we designed didn't reflect the experience of that particular population, and we didn't find strong results. It, they wouldn't have been scalable even if we did, because we weren't authentically engaging with those populations. And so this is a, this is just one example, but just gives a sense of how we really have to be mindful about how we are engaging in these spaces where often the research team and others that come in with this positional power are not authentically engaging with the populations that are going to be most impacted and have a lot of knowledge that often isn't held. And so in the book, we have a chapter on each one of these where we walk through examples of other specific practices of how we can approach a more anti-racist.

4

Engagement with each one of these. That's great, crystal. And it's

1

not only about really making the research and our government programs and interventions more effective, but it's also about the communities themselves, like they have the most at stake in ensuring that what we're doing really works well. And so I love that what you're bringing forward here. To me, it also really responds to what we hear from our own residents in King County about bringing government closer to the people. And it's about having government and researchers and communities work more closely together to both define the problems and develop the solutions. And that just makes good sense. And you're gonna hear a lot about that from Ann and Sunny and Matt as we continue on in the conversation too. I wanted to turn to Laura because Laura, I know the book resonated with you just as much as it did with me, and I'd love to hear you talk a little bit about how you see it as applying to JPEG of research affiliates all across the country and the world, in fact, who are interested in applying their work to public impact and alleviating poverty. And how do you see the approaches in the book applying and where do you see some of the challenges too?

2

Yeah, thanks. So yeah, Jay has both a research network of academics around the world and US staff that supports researchers. And across those groups there's a lot of interest in how to do research more equitably, more inclusively, how to find programs and policies that will increase equity. So we ran an initiative last year trying to get more research proposals addressing racial equity more directly that were continuing to try to do more of. But as we approach this work, there's a lot of fear and uncertainty about how to get started. There's the, what if I get it wrong and I make things worse? I don't have the skills for this. Am I the right person or am I allowed to do this work? I don't have, I'm not a person of color, I don't have these personal experiences, so what can I even do? Is this my place? Place? And then where do I get started? So this can lead to some paralysis. And then just continuing on with how things have always been done. So this book gave really tangible examples and a roadmap. And even hearing from Crystal now about the evolution of the journey versus suddenly becoming an expert overnight was helpful in the roadmap. Just gives some concrete suggestions of how to get started. I think one of the, one of the challenge points is that a lot of these suggestions are to engage with the community. This is something economists are not trained to do, even. When I started, my goal was to be able to work in a corner on spreadsheets and never talk to a person again, it grown personally, but this is something that is really a stretch, learning more how to do that well.

Applying Anti-Racist Principles in King County's Best Starts for Kids Initiative

1

yep. I'm also trained as an economist, so I can definitely attest. Yeah, and that is a challenge and it's something we're gonna talk to and Sunny and Matt about. And Laura, I also know that you've created a bunch of resources at JAL for researchers around those challenges. So we can drop some of those in the chat. And let's start bringing Anne and Sunny and Matt into the conversation too, so that we can really focus on what does this look like in practice? How do we do the work? How do we get started?

2

Yeah. So let's start with Anne. So Carrie mentioned earlier that you lead an evaluation for King County's Best Starts for Kids Initiative. She said it was a voter approved initiative with a focus on early childhood. And so I wanted to hear from you more about what this is, how King County has been intentional about building anti-racist focus and principles into that work. And just tell us what the survey is and why it's important.

5

Thank Flora. Yeah. I have plenty to share and I'll focus on the BSK Health survey as an example, but wanna underscore that anti-racist work is happening across Best Arts for Kids, and we have a lot of great examples and amazing work going on. When Best Arts for Kids was launched, it included support for a countywide population based survey to help fill a gap on children's health, on data for about children's health. And this was important to be able to understand the strength and needs of families with young children at a hyperlocal level. So the data we collect becomes a public resource to demonstrate where investments are needed. And monitor how community conditions are

4

changing. So some of the lessons from Crystal and

2

Minnie's book are about, as researchers being intentional about questioning the status quo, questioning the systems and the practices with an eye towards how the system impact impacts individuals and how that shapes policies. Can you talk about how that influenced how you approached the best stars for kids surveying?

5

Absolutely. Yeah. We've really had to question and challenge some of the traditional practices in survey research and some of the systems we use to collect data so that we could produce a re a truly representative set of data that could be used to advance equity in King County. So this looked like oversampling by race and ethnicity, providing the survey in six languages and three modes. This looked like working with the UDub School of Social Work. Who was the survey administrator? To hire both bilingual and bicultural phone interviewers included expanding our race and ethnicity categories on the survey to be really robust and making sure that we use strength based questions on the survey as much as possible. So the best starts for kids implementation included a targeted universalism approach. So this data supported the development of tailored funding strategies. I remember one Somali serving organization that shared with us, they had never had countywide data before for their community to use in grant applications before having access to the BSK health survey.

1

Yeah, I love that example. and we've done work with the Somali community in the transportation context too. Matt and I were part of a focus group with Somali women where. They really brought to light for us amazing their experience on riding the transit system and the barriers that they faced, which were things that we would've never known about if we hadn't spoken directly with them. So I think that it's that co-discovery that's so important. And one thing that I think we've learned from our work generally at King County and that really comes to light in Crystal's book is communities are really interested in data about their community. They want to help to collect the data they want to add to interpretation of the data, and they have a huge stake in using the data to support better outcomes in their communities. By partnering with community, we're just, we're making our work better. We're making the use of taxpayer dollars much more effective. You talked a lot about the work you did behind the scenes as a survey researcher and a scientist to make the survey more representative, and I know that the team at Public Health was also really intentional about engaging in that kind of co-design and co interpretation with community partners that Crystal highlights in the book. So can you talk a little bit more about how, what ways did you seek that engagement with communities around co-design and co interpretation? How did you build that part in?

5

Mm-hmm. Absolutely. Yeah. Really from the beginning, nine years ago, in 2016 when the initiative was launched, we set to work on the survey and we worked closely with the committee from our children and youth advisory board to design the survey itself, decide what questions were included. And how we would, what languages we would translate the survey into. And then we also completed a community-based pilot working with community partners to test the survey and incorporate their feedback. And then in the survey cycle, since then we have added a lot of elements like participatory data interpretation, which Sunny will share more about partnering with. We partnered with her organization, the Community Cafe collaborative and other community liaisons to do the that interpretation. And then our team has consistently incorporated the feedback we get from those engagements into our survey content and methodology. So for an example, we added a question after being asked about why, whether or not we're measuring discrimination in the county. We added a question about parenting caregivers experiences of racial discrimination. So the resulting report with that data included feedback from Community Cafe sessions and the report itself was also co-designed. And I think we'll share that in the chat as well.

1

Yeah, great. And it's really important, don't just ask for the feedback. You have to also act on the feedback. It's a really important step and I think one of the lessons I've learned over time is like part of the respect in a partnership is as a local government, we can't do everything. We, there are some times where we can't act on feedback that we get, and part of the respect of the partnership is just to be really clear about that. So I'd love to bring Sunny into the conversation too, because a lot of what you talked about, Anne, led to this partnership with the Community Cafe Collaborative and Sunny, I'd love to hear your perspective on what that looks and feels like too.

6

As Ann mentioned, BSK had the desire to have a rep representative set of data, and that's so important. And it also goes back to Crystal's roadmap. It's the very first thing on her roadmap. Wanting to have that set that is representative is so important because as you all know, data tells a story. And much like the accounts of history, the perspective is so crucial. So yeah, it's all, it's so important to have the inclination to tell the whole story. And so this is where our partnership enters into the story. So our organization partners with community members to per, to ensure that community perspective, knowledge, voices, they're all represented in the data. And this process is done through a structured conversation we call a cafe. So these are designed and led by community members, and this enters, the questions are crafted and asked in a. Culturally relevant way. They're, the community hosts are trained by our team, and we provide coaching technical support for the hosts. And these cafes can be held virtually and in person. In fact, before 2020, they were almost exclusively in person, but since then, we've all moved on. And so they take place in a school, a church, somebody's living room, a park. And so all the participants and hosts in support roles receive compensation based on budgets that are created by the community members. And from these gatherings, we're able to harvest is what we call it, authentic detail. Rich

4

qualitative data. Thank you for talking us through that. Sunny. One of

2

the things that had struck me that I talked about just earlier is many of the approaches from Crystal and Minnie's book is highlighted and built upon. Trust and collaboration between researchers, governments, and communities. And we know that there is historic and systemic distrust between communities and government research. A lot of that is well earned distrust and many academic researchers don't have existing relationships with communities and don't have those particular community building skills either. So I appreciate how it sounds like Community Cafe Collaborative is acting as that kind of really skilled liaison of building the trust and having the skills to do that really well. and relieving burden somewhat from the community and having to figure out how to interact with these researchers or government partners or whoever else, and provides an entry point for the government staff or the researchers to bridge some of the gaps in their skills as well and build that trust over time.

6

Yeah, so I appreciate what you said before, Laura, about this, about the, you were talking about the fear and the paralysis of how do you get started, and I feel like there's also a, there's fear and a paralysis coming from a different place, I think on the community side. So yes, many of the communities that we partner with have a historical distrust of government, and especially in the realm of data collection. So the question becomes, if you wanna partner with these communities, how do you bridge that divide in order to start laying a foundation of trust? And I think this is a very important piece. And backing up to Crystal's roadmap, it's maybe the little sliver that's in between the first and second steps, so that laying that trust in building a foundation is so important. So when BSK first approached the collaborative to help interpret the survey data. This was done with a deliberate extension of respect and understanding that it would be a partnership. It was done so gracefully by Anne who continued to nurture the relationship, dis intention on BS, K's part to start a relationship with community members, ask for their feedback based on their lived experiences. That was really a key step into weaving anti-racist practice into the BSKP system and how they collected their data. It was also the beginning of a relationship that grew over time. It's been many years since then. And so I mentioned that because it's very important because authentic community partnership requires trust and that takes time to build. And so really it's an investment of time and resources and the continuing nurturing of mutual respect.

Challenges and Strategies in Community Partnerships

5

Yes, and I'll just add that I've always had really great leadership support to invest that time and those resources. And as government there, there have been times where we have really thought we've done a great job, especially on our team, our first drafts of the survey where we take it out and then we get feedback through these conversations that we've missed the mark. So first off, I'll say that gif, that feedback is truly a gift. And second, missing the mark isn't what defines our work. It's really our work is defined by how we respond to that feedback. So we're always gonna make mistakes, just like Laura was talking about. People can be afraid of making those mistakes, but that trust really builds over time. As we ask for the feedback, we hear it and we respond. And like Crystal was saying, it's an iterative, ongoing process that's not always gonna be linear. Yeah.

6

And I also wanna add one more thing is that when you're partnering with community members, leaders, or organizations, just knowing that while the drive and passion and the know-how are there. We don't always have the capacity to hit the ground running with any particular project. So just being cognizant of that as well is so important. And I think King County BSK did a great, has been doing a great job of acknowledging that some of these scrappy organizations aren't always at running at full capacity.

1

Yeah, and I think part of the respect too, I'll just hop in, is we've made it policy in King County that we pay community members for their expertise, just like we pay for other kinds of expertise. I think part of that is about the respect and the relationship. Part of it's also about building capacity in the community, but that's a really important and key

4

piece of it too. I think the only other small

3

piece that I'll add is I think this is a place where I think. Local government has the ability to be relatively nimble, right? In terms of building these relationships and establishing trust. And in my work and a lot of the examples that I, we give in the book, they come from all over. But there's several from the perspective of working in the federal government. And I just think that there's such an advantage working on a smaller scale often with these types of questions on many levels. But I think especially when we think about building trust and building connections in the community, deepening

4

those relationships. Totally agree. So let's

2

move to Matt's perspective. So perspective from an academic researcher who works outside of government, but is PO partnered really closely with governments.

1

Yeah. I'm really excited to bring Matt into the conversation.'cause I think a lot, any of you who are researchers out there will really relate to his experience and. At this point, we have this really longstanding, very fluid and nimble partnership with Matt, and that's also been built on leading very discreet grant funded research projects that have all the kinds of usual constraints on time, budget, and the like. And I know that it can feel really overwhelming to figure out, I don't have three years to build the relationship with this community. I don't have a budget to go through a whole bunch of processes around translating everything that we do. So Matt, how do you think about where you really start on building anti-racist approaches in?

7

Yeah, thanks Carrie. Completely agree. Projects always face constraints on all sides, whether it is time, their budget, and I think also importantly, the sort of starting point when it comes to what the relationships actually are, who you know and who you've have got connections with both in government and in the community to help support the project. Obviously we can't using terminology before that kind of paralyze us, right? We've got to get started somehow. And I think that there are, there, there are always places that you can start to work with even within whatever constraints that you face at the outset. so in our research that you alluded to carry on and trying to improve transportation opportunities for families with limited resources in the King County region, our starting point was always just trying to recognize that there are numerous barriers, potentially transportation and accessing opportunities. And that the sort of downstream effects of those barriers are in part sort of a product of a variety of policies, procedures, structures that have been in place for decades and decades that have historically disadvantaged certain communities, people of color, also other communities that have just been underrepresented. Among those who have been informing or building, making those policies and structures. And so given a lot of the focus in our research around transportation and access to opportunities and given very long history of residential segregation, king County and throughout the United States, that's always been front and center in, in a lot

4

of our discussions at early stages of projects. So I'd love to dig in just a little more on that of

2

how that cognizance of the role in the history of segregation and underrepresentation, how has that helped you define your research questions or think more about codifying and co discovering with community?

7

Yeah, that's a great question. And as Carrie mentioned, many of our research questions have been informed directly by interfacing with individuals within the community. Or with community organizations who of course work with individuals in the community. Um, and as Crystal was talking about, as Carrie mentioned too, trying to move away from a world in which it's just the researchers who have the question, who come up with the question, but trying to make sure that the questions we have are really informed by the community and the people who are directly having to facing these policies and interventions. So just as an example, in previous projects, we coordinated with community organizations, as Carrie alluded to, to have focus groups with different populations in the King County region and learned in the process an enormous amount. For example, that there's a variety of barriers people face to accessing public transportation. Not just costs, but concerns about safety, concerns about reliability, many other factors. And not only that, but the kind of the extent to which. These different barriers really factor into people's decision making also varies quite a bit across a different population. So that's led us to think a lot harder, and in more nuanced ways about potentially heterogeneous effects of different interventions we might do. And also, of course, as informed King County and our research work about different policies at the county house.

2

I'll follow up once Warren asked once more. Is there anything even more concrete about how that's influenced your work? A nice tangible example.

7

Yeah. I think one, one example is that there are certainly populations where it was clear that there were living in parts of the county that you might consider to be almost transit deserts, where there was very little access to transportation. And, uh, a lot of our initial kind of thoughts around intervening to try to reduce prices or produce fares for accessing transportation, naturally we would be concerned that those types of policies aren't gonna be particularly meaningful for folks who don't have access even to a bus stop. And that was one thing that came out of a lot of these discussions and really helped informed our research. Also, I think more generally, conversations with people in the community have helped us plan more actively and more intentionally around how we outreach to different communities. Communities, for example, that speak different languages. taking into consideration how different policies, interventions might actually be also perceived differently by different communities.

2

That's really helpful. I wanna call back a little bit to something you said just a second ago about heterogeneous effects by race. So many times researchers will approach. These different effects by race as an afterthought and wash out a lot of things as the effect of race on blah, when really it might be a little more of what's going on underneath all of that. Can you talk a little bit about how we can be more intentional about that at the outset and at the end to be aware of what's going on

7

on? Yeah. I think this is a really important point in one where I think we've all tried to, we've all evolved over time in how we think about these things. It was certainly, it's certainly the case that now, and this is in part because of these great collaborations we've had in King County, you think a lot more carefully than in the past to just being very thoughtful and explicit about the role of race in conducting our analysis. So being upfront, even prior to launching a study about how and why we're collecting that race information. Going back to some, some of Anne and Sonny's work. Also how that information that is then subsequently gonna be used and interpreted an analysis. And we found that thinking and being very deliberate about this, even at the pre-analysis stage, both helps us avoid conducting analysis that kind of implicitly perpetuate stereotypes, but also forces us to think about race and racism and the intersectionality of race and other identities at every subsequent stage of the research process as well.

1

Yeah, and Matt, my experience has been too that having that really intentional focus on why we're looking at differential impacts by it makes the research more actionable. A lot of times we read studies, maybe not even a lot of times because we know from the evidence that's been collected often studies don't address race at all. But when they do and it just mentions differential impacts and there's no real explanation of why that might be, then we're just guessing at how we might be able to serve people better. But if we really go in from the beginning and we think about why such differentials might exist, then we can often tie that evidence to an action we can take. And that goes back to the point you made earlier about really looking at the different kinds of barriers people face to transit access. If we aren't addressing the most important barriers, then we're not gonna find impact in our studies. So really thinking these questions through can lead you to, it just makes the research better and it makes our ability to act

4

on it more available too. I wanted to also call back again

2

to something that Matt said about. I avoid implicitly perpetuating stereotypes. So Crystal, I think there is some resources that you had on thinking about what agency researchers have in framing results or even framing research questions and why that's important for that reason, and maybe others too.

3

Yeah, there's a great paper that came out a few years back, and we mentioned this in the book, and it's really geared toward researchers that do work in the field of psychological science. But I think a lot of it is really relevant to this conversation as well. And it's about addressing racism in the field. And one of the points from this paper that stuck with me the most, and I share this with my students, is this idea that we have agency over how we frame results. And that can be really powerful. And one of the ways that we can be mindful is to think about when can we center systems when we talk about results. And so instead of saying. People that are living in poverty are at risk for poor health outcome, poor health outcomes, or people of color, or at risk for poor health outcomes. We can say that those communities are exposed to additional harms that then drive those health outcomes, right? When we show disparities and results, and if we see negative results for populations that have historically been oppressed, we can take that extra step to explain that those outcomes are likely due to racism, systemic historical discrimination, rather than just describing the outcome. And it may seem like a small thing, but I think it's really powerful when we think about what that means in terms of next steps for the decisions we make because of that research or even other research questions that might be prioritized.

4

Yeah. Great point. One

1

of the things that, that Laura mentioned and has been a little bit of a theme through the conversation is that, you know, researchers coming in to work with a community may not have well-developed connections in that community. And Matt, I think one of the things that's been really great about our relationship is that we've intentionally relied on each other's strengths and what government and researchers can both bring to the table. As a government, we partner with lots of community organizations, other governments. We do a lot of community outreach, and so we've relied on those existing relationships in the research too. We weren't expecting you to come from University of California at Irvine with all these great deep connections in the King County community. But rather, what I've really appreciated about you is your willingness to really partner and listen and work with the communities we serve in those co-design. Aspects. So you wanna talk a little bit about that?

Overcoming Positionality and Bias in Research

7

Yeah, I couldn't agree more. None of us are good at all things, and so it's just been amazing to be able to partner with so many organizations and learn from each other. I can't personally overstate how valuable it's been to have these sort of partnerships, not just between researchers and say King County Metro, but also as you mentioned, Carrie being able to take advantage of these existing relationships between different agencies. So for example, some of our working King County involved not only working with King County Metro, but also Department of Social and Health Services and the state. And that was just absolutely instrumental in terms of getting the kind of institutional knowledge that we needed to design interventions and the thoughtful, deliberate way that where we could really, everybody could learn something from them, but then also ultimately also implementing those interventions and interpreting. The results from the evaluation as well.

1

yeah. One other thing we've touched on lightly, but I wanted to address a little bit more deeply, and Ann and Matt, I'll address this to you and Laura you can jump in here as well, is Laura mentioned the fear of not getting it right. And I think that can really paralyze us, particularly as white people coming into this space of looking at differential impacts by race. So how do you overcome that if in fact you have, how do you think about your positionality? I know for me some of those feelings of worrying about whether I'm gonna get it right, I have to remind myself that we have to start somewhere. We have to make progress. We all need to be a part of it. Like how,

4

tell us a little bit about how you work on that. I can start.

5

I think that the short answer is to practice just like many things in life. But as a white woman in re in research and evaluation, I work to really be aware of my positionality and privilege and lack of lived experience. So the ways that I do that are at King County, we utilize affinity groups. So I'm actively involved in a white caucus as well as I just had the pleasure of completing an anti-racist leadership training cohort led by Dr. Roger Rochelle Sard. And it helps me to be able to name the whiteness within myself when it shows up. And then just having those places where you can practice and build your muscles to regularly engage in conversations around, excuse me, around an anti-racism. And remember, like I said, it's an iterative and ongoing process. Like you said, I haven't quite overcome that completely myself.

1

Yeah, and I think we've got some resources we can put in the chat for people too around positionality and how racism operates on all of us white people included.

4

Matt, Laura, do you have anything you wanna share? I'll echo

7

Anne and Carrie, your thoughts on this as well. I think that it's a kind of an evolving process and we're constantly learning. I have to say that as an economist who, like Laura, like Harry, often sit in a room with spreadsheets by myself. I do think that we do learn a lot, both from interacting with people in the community, but then also by the work of Crystal by others who are moving this conversation forward, making it more front and center in the research community work by j Powell that's promoting research that really takes these issues seriously. And I think that seeps into research that it doesn't even necessarily engage with j Powell. And so I think it's strengthening the whole research community and I think we're all learning from each other how to do this better, how to make the research better, and how that research can in turn and inform government policies in a a better way.

1

Great. And we do have quite a few questions popping up in the chat, so should we turn to those now? Laura, what do you think? Alright. Jona has a question and Jona, I'm wondering if you wanna come off mute and just voice it for the group. Hi, can you hear me? We can.

8

Okay. Yeah, I think my question really was referring to like, when funding comes available, say we're gonna release a new RFA or RFP and. I think the lens at which sometimes folks are looking at how they write the language to release the RFP and how they engage with the community that they're targeting is sometimes, like you were talking about like unintentional bias or implicit implicit bias and things like that come into play where you don't see how what you may be putting out may be ex more exclusionary than inclusive. Yeah, and I'm wondering like as or as organizations like the county or something like that, is there, I mean besides of course this book, which is amazing, is there ways that folks who are new to doing those types of things, especially new programming and new funding, is there a way for folks to be trained or get this like a crash on how to look at things more equi equitably?

1

Yeah, that's a great question. Joana in King County, we've done a ton of work. Particularly in Best Starts for Kids, which was a brand new program we intentionally grounded Best Starts for Kids in equity. So that was a built in focus, but we've done a lot of work in our RFP processes to try to make them more inclusive, more accessible. One example is that we have brought on, we've hired technical support organizations that are embedded in communities to help community-based organizations apply for RFPs. We've made some RFPs available in different languages and accepted responses in different languages or different ways of gathering responses. There's an, there's a variety of things that we've done and I don't know if you might have some other thoughts on this question. I don't know about. Trainings that

4

are available, but maybe and does.

5

Yeah. I'm also not sure if we have a training for this yet, but you're right. Technical assistance providers have been really key for helping us increase accessibility of RFPs. We also have a practice of using a work group model to develop an r fp, and so you can actually invite community members to be part of the work group that writes and publishes the r fp. So that's been another

4

really good way to build an equity. Crystal, or Sunny, do you have anything you wanna add? I would say,

3

I wonder, just listening, reflecting on that question, it makes me wonder, and I have no idea how radical or not this sounds, but is there a way to engage in more practice of, at that stage. Actually sharing that with the communities that are impacted to get their sense of what's the right way to even pose that type of any particular type of opportunity. And that's something that I've never been engaged in a process like that. And I think that might be really interesting. It's definitely an interesting thought experiment in my head, which is what, and this has fared for me, is like how might that shape the trajectory of research and evidence building if we build it in even at that super, super early stage.

1

Yeah. And it can be so nuanced, too Janetta. I think partly it depends like how eager are these new departments and programs to do this type of work and how much influence can you have in pushing them forward? Just asking some of the questions about who are we trying to serve, what kind of outcomes are we really trying to get to, a scan of the environment, those kinds of things. And we'd be happy to chat with you more if you wanna learn more. So thank you for that question.

8

Thank you.

1

Maya, our friend Maya. Results from America has the next question. Maya, you wanna jump off of mute and voice your question?

9

Sure. Thanks so much everyone for today's panel. It's great. And Crystal, congratulations on the book. I am thinking a lot about this conversation in the context of the executive actions that we see taking place under the Trump administration. And I'm trying to think through also many of the comments that you all made at the start of the conversation about how this work lives in the work of good government and good research and doing our work well. And so I'm trying to pull that together in my brain and I would love to hear from you all the panelists about how can governments adopt these principles and the framework that you laid out for us, crystal, in this environment without exposing themselves to new risk. And how much of that is it possible to do that with this.

4

Good government framing or not. Anyone wanna take that on? Crystal? You wanna take it on?

3

I'm curious to hear from others, but I think my immediate reaction, and thank you Maya, for the question'cause it's such an important one and I know it's on our minds, is I think I would imagine, especially like at the local government level, so much of this depends on where you sit, right? And so being in Washington state, in King County is very different than being in other parts of the country. And so I think having, being realistic about what the climate is in the space that you're in. And I would also say, I just feel like this is why we can't lean too hard into this argument and framing of this is how we are more effective. This is how we avoid wasting money. And wasting time is engaging in these questions in a really thoughtful way that's trying to be realistic about the root of the differences that we see. And I know that's not always gonna get us maybe to where we need to go, but I do think it's an important point that we can't, that we can't let go of. And then again, just trying to be realistic about what opportunities we have, depending on the particular environment that we're in, at the local level as that relates to the state sometimes and the federal level.

4

Yeah, I'd

2

love to, to add, I feel I'm the furthest from state and local government of probably those on the call, but I think something that's been in the same realm is the tension between what can you try to do and slip under the radar with words like this is effective for everyone versus where do you need that intentionality? And if you don't have, if you don't know what you're trying to achieve, you can't achieve it. And there's some element of trying to slip under the radar and do it, but you need to know, I am trying to achieve results or research or outcomes that are more equitable along these factors to be able to know where to engage and what questions to ask and how to look. So there's some element that you could maybe hide it, but there's some that you eventually hit a barrier. And the other thing that's in my mind came from one of my colleagues who's on this webinar right now reminded me you have to meet people where they are right now, and that's gonna vary based on where you are, where they are in their journey of how they're gonna react to anti-racist versus community engagement versus better government for all. And staying really aware of where you are and how to achieve your goals based on where people are at and helping nudge them along where you want them to be.

1

Yeah, it's a huge question, Maya, and I don't think any of us have the answers and it's continuing to evolve. One thing I do know is that we can't just try harder. We have to shift our strategies, and I think we have to be cognizant and aware of what's happening, but we also can't stray from our focus because we know the evidence shows us how we need to serve communities best, and that's what our fundamental responsibility is as local government. We're gonna continue to do that. Sunny, you wanna hop in?

4

Yeah.

6

I just wanted to add to the conversation that in, I would say probably every community in the nation, there are community members in organizations who are quietly working on the issues that are important to them. And so I would encourage you to look around and see what work is already being done and see how you might be able to fit in to that work, to support it, take the lead, or take the lead from those organizations or those members. I'd different way to look at it. That's what I would recommend. That's a great point. Sunny, it's

1

flipping our conversation on the head a little bit. Really moving from thinking about researchers and governments partnering with community to community, really being in the lead on a lot of this work and us following the community's lead, and I think that's what it looks like in very best practice. I saw that Anna had a question, but she had to jump off. Will, and we're at time here, so I don't know. Lord, do you have a final reflection or something you wanna share?

2

Do I have a final reflection? I'm, I've just appreciated this conversation so much of what are the small things that we can do that could add up to bigger things and get us further along into learning how to, how to use the roadmap or to get started, what are some practical tools? and really starting from what are our needs in any given research or policy or RFP or whatever it is that we're creating and how can we do a little bit more to reflect on what's underneath that might make it work? So that's my biggest takeaways. Just start somewhere and then keep going.

1

Love that. Start somewhere and keep going. That's a perfect note to end on. I wanna thank everybody for joining us today, crystal. Thank you. Matt. Thank you Sunny. Thank you. And thank you. Really encourage everybody to seek out Crystal's book at your local library, at your local bookstore. There's a link in the chat if you wanna find it there. It is very accessible and it's gonna help you get started. So thanks everybody. Really appreciate you being here. Really appreciate the work that you're doing all across the country.

4

Keep the faith.