By: AIF Staff
Last week, as part of its quarterly policy panel series, the American Idea Foundation hosted a conversation with leaders of evidence-based workplace training and education programs to discuss how policymakers can work with community-organizations and philanthropies to help Americans reach their economic potential.
Detailing customized adult-education models, innovations in venture philanthropy, and data-driving job-training models, experts from Per Scholas, Goodwill Industries, New Profit, and the American Enterprise Institute discussed their successes and their challenges as they assist adults who want to further their education and develop vocational skills.
The panel conversation showcased how data-drive efforts are being deployed around the country to assist more Americans realize their version of the American Dream. Joining Speaker Ryan for a conversation about promoting entrepreneurship and a thriving 21st century workforce were:
– Plinio Ayala, President & CEO of Per Scholas
– Brent Orrell, Senior Fellow at the American Enterprise Institute
– Jeff Nelson, Entrepreneur in Residence at New Profit
The panel discussion is available below and some notable excerpts follow.
Speaker Ryan on the importance of evidence-based workforce training & education models:
“The professional class in America has weathered COVID pretty well. Higher-income people didn’t see a huge income decline and we didn’t see lots of lost jobs, but that is not the case with working-class Americans, who are getting really hard with COVID. Policymakers at the federal, state and local level have been grappling with how best to help these individuals and grappling with what’s the right policy mix. So today, we want to talk about one area that is vital to supporting these individuals who are trying to get through COVID, who are looking for employment, or looking to improve their employment position, and that is the workforce development and the job training space…
“Specifically, the panel is going to discuss three areas that matter as we try to ensure that these programs are working and that they are working well.
1) Very little evidence surrounding the efficacy of most job training or vocational programs actually exists, so what works, what doesn’t work, and what can we do to better identify solutions?
2) How do you scale things up? How do you see huge differences in trying to scale effective social programs or deal with a situation where one program might work very well, but is unable to be replicated in other places? How do you build something that has proven evidence to work and then how do you scale it?
3) What are the structural impediments or issues from a government involvement standpoint and what can be done at the policy level to assure that we can scale up effective evidence-based programs?
“As an example of why this issue matters, the American Idea Foundation recently visited the Nurse-Family Partnership Program in South Carolina. I’ve been enamored with MIECHV and the Nurse-Family Partnership program for many years. This program solved a lot of issues. They developed an evidence base. They scaled their programs and they have federal legislation that helps to fund and continue identifying evidence-based programs, and they’ve got metrics that show the program works. They have success they can point to and several of our panelists today are at the very beginning stages of a very similar journey.”
Betsy Delgado on the Excel Center’s evidence-based “secret sauce” & scalability:
“We decided to work with the Laboratory for Economic Opportunities (LEO) because we knew it was really important to have evidence on the results of the Excel Center. We know that 2 out of 3 people who don’t complete a high school diploma are low-income and they make 70% less than those with a high school diploma. So, we know, in our walls, with our 10,000 plus students that we serve across the country that we are working with the most vulnerable, marginalized families and we wanted to make sure that by working with LEO, they would help us understand how a high school diploma from the Excel Center translates into post-secondary career success for our graduates and our families.
“The other thing that was interesting to us is our General Assembly [in Indiana] really wants to make sure that we’re coming back and demonstrating to them these results so they can understand the return on investment of state dollars in our students….
“We use these results from LEO to make improvements to the model so we can better serve our students and their families. And also, as we help the Excel Center network open schools, we want to make sure to maintain fidelity to our work as we scale and replicate. We just launched our RCT that is going to look at our justice-involved students to make sure that our coaching model is wrapping around to these students and their families to demonstrate the same outcomes that we expect to see from the entire model. It has been an incredible partnership. It’s hard but it’s incredibly important but we want to serve our families well…
“Some of our team would say that the secret sauce is our coaching model and that wraps around services for our students, their children and their families. Over 50% of our students have young, school-aged children, so we have to consider the whole family in this model and in the services that we provide. Our services include life-coaching, housing, food, technology, health and employment. Also, we have free childcare on site and we offer free transportation. When we enroll, every student is provided with a life coach that stays with them for their entire educational experience, and their job is to make sure that the students can actually experience economic, educational outcomes that we are striving for them and their families.
“I think the second [secret sauce] would be that it’s the rigorous curriculum that we offer at the Excel Center: in-person learning, all of it meets the state standards, and we only employ highly-qualified teachers. This results in adults earning a state-certified, high school diploma that is recognized…
“And then my final thought on the ‘secret sauce’ is we take the two-generation approach. We think of the whole family. We want full families to thrive. And so, we know that the children of parents who have not completed their high school diploma are 50% more likely to drop out themselves. We know, because of our research, the children of our graduates now have a parent who just graduated from high school, which makes it more likely that their child will graduate themselves. And then finally, we know we’re helping families change their economic trajectory for this generation and then generations to follow.”
Per Scholas’ Plinio Ayala on why more non-profits don’t utilize Randomized Control Trials (RCTs):
“I would answer that by saying that the risk is significant and the reward is negligible. And so, an organization that goes through an RCT and doesn’t show impact, there is a fear of reputational hits and a possible loss of funding. I think as a space, we need to get better to allow organizations that go through evaluations to use them to improve the quality of their work as opposed to punishing them.
“I think Betsy alluded to this as well, but the cost of an RCT is significant and not just in the study itself, but the cost that an organization has to bear to build the infrastructure to conduct an RCT is usually not accounted for in any of these budgets. And lastly, to the reward piece, we have gone through two RCTs and I have not found a channel for funding, at least on the government side, that allows me to scale and scale pretty significantly. The question here is how do we incentivize organizations to participate in this with the understanding that if you show positive results, there will be some investments ready to be made in you at the conclusion.
New Profit’s Jeff Nelson on how philanthropic investments can incentivize successful-models:
“New Profit is a venture philanthropy that started 23 years ago with social entrepreneurs, like Betsy and Plinio, to try to tackle major poverty issues in the United States with unrestricted funding, a lot of capacity building, and then we try to parlay it into federal policy change. I have a couple pieces of advice around trying to create better incentives for quality evidence in the state.
“Needless to say, all sourcing and selection efforts should integrate and prioritize evidence as the central tenant to the investment selection, but way more than that, a couple things that we’ve seen in the last five years:
“One is that evidence planning really matters. Both Betsy and Plinio referenced this. This is not something you wake up on a Wednesday and say we’re going to start doing this next week and we’re going to hire a great team at Notre Dame and we’re going to integrate it. Philanthropies can create incentives in their process to bring in technical assistance for evidence planning and to actually track and support entrepreneurs to make progress on their plans throughout a multi-year gift.
“The second thing would be to integrate return on investment calculations into the process so that you’re taking costs into account in parallel with evidence.
“Third would be to really consider a holistic approach to building evidence. Evidence is both an ongoing commitment to the external evaluations with RCTs or well-designed, quasi experimentals, but it’s also about building good data systems and processes to continuously improve and to create a data-rich environment. And when funders are making investments and elections, they should consider what we call a “preponderance of evidence….”
“We would advocate strongly that you bring on social entrepreneurs like Betsy or Plinio, and you pay them well as Executives in Residence — like an hour a month, two hours a month to be advisors in the process, because they’ve actually gone through what is honestly a really hard thing to tackle, which is to integrate evidence-building into a high-scaled organization.
And then the last thing would just be that, like we are suggesting that there are structural issues at the federal and state government level, we would also advocate that philanthropy needs to have a proportional approach to its investment portfolio.
“If you have an organization like Per Scholas that has a body of evidence and a pathway to scale, there should be proportional investments for something like that. In other words, if it’s taking a while for government to catch up or to create the incentives, at least philanthropy should be able to move quicker to be rewarding those entrepreneurs like Plinio or Betsy, who are undertaking what they are undertaking now.”
AEI’s Brent Orrell on the proper role of government to promote upward mobility:
“You were talking about Nurse-Family Partnership and NFP is not just the gold standard for what it does in terms of short circuiting inter-generational poverty. It’s also the first item on everybody’s list of evidence-based practices because they have been building it for 30 years. They have collected 30 years of longitudinal data and they are so insistent on following this model of pairing registered nurses with first-time, single moms. It is just the example of how to do this right and it has gotten amazing results and we’ve all been in favor of all of its expansion over time.
“The problem is that it’s a unicorn in federally-funded services programs in terms of those kinds of results. So why don’t we get better results from other programs?
“It’s a multifaceted problem. We’ve got a lot of people claiming great results for their programs, because, in fact, intentionally or not, they’re creaming the population. They’re getting the most able people into their programs, whether they mean to or not, and so, when you put that back in to the general population, it doesn’t work. You don’t get the same kinds of results….
“Counter-intuitively, I think that no results or negative results are actually pointing us towards something that’s really important and that is that sometimes people do better with less intervention. This cuts against our desire to be activists in the lives of others and we all want that. We all want to make a difference but if our evaluations are showing us that either we’re not making a difference or, in some cases, the control groups and people who don’t get services do as well or better than the people who are getting services, I think that should speak to us.
“What it says to me is we really need to focus on approaches that build agency in the lives of people, rather than build our agencies. We’re not investing in programs. We should be investing in building the agency of people to take advantage of opportunity.
“There are good examples of this. In the workforce realm, we have these things called “Individual Training Accounts,” which are essentially vouchers for people who go out and get training so they go get the job that they think they need. These work very well and most of the time it is a fairly light touch from the government, so it is providing resources but we’re not intervening in the lives of individuals. There’s a fantastic project in the Bay Area called the “Family Independence Initiative,” which has a no-helping rule. What they mean by “no-helping” is that they’re coaching families to identify the resources that are already available to them through other members of their families or the community to solve their problems rather than always turning to a government agency and it has had some interesting and profound effects in communities where it’s operating. What connects all this stuff is they focus on freedom and the development of personal agency rather than on surrendering that agency to a government or a non-profit organization….
“What I’d like to see is an expansion of incentives for programs that focus on the development of personal agency and autonomy, along with rigorous evaluation so we can tell: “Is this a real thing? If by providing a lighter touch, are we helping people more?” I agree with Paul that good case-management is always an element of success but it has to be good case-management but that means having a coaching model that promotes a pathway towards independence.”
Note: Former Speaker Paul Ryan serves as a policy fellow at the American Enterprise Institute and is a visiting lecturer at the University of Notre Dame.