Ed;
I love a post that makes me think - and yours does that. Most post just dribble on ..."I'm right and you're wrong". In answer to this question from you I just can't understand what switches that toggle in your head I suggest you read the following that I was just getting ready to post. What toggles my brain is Pragmatism. Banning 20 oz soft drinks does NOT get my support because I think it is silly to think it will make one iota of difference. So why waste money and make those who oppose regulation mad - just for the sake of making a statement. All that does is polarize people pushing them into their ideological corners. .....A better way is to find something that works - that both sides can agree on. To understand that I really suggest reading this article. Washington State is very progressive on such pragmatism but it seems so obvious that one wonders what everyone else is waiting for.
May 30, 2012, 7:00 AMThe Dawn of the Evidence-Based BudgetBy DAVID BORNSTEIN
Fixes looks at solutions to social problems and why they work.
TAGS: DATA, FEDERAL BUDGET, SOCIAL PROGRAMS
On May 18, the Office of Management and Budget issued a memorandum that couldn’t be more critical to restoring public trust in government.
For the 2014 budget process, the office advised agencies to include information about how they plan to evaluate the effectiveness of their programs and link their disbursements to evidence. The office said a commitment to using evidence would make approval of budget requests more likely.
Could this be the coming of age of “evidence-based policy making”?
Programs that can prove their effectiveness with data will be more likely to get the funds they seek.
When we consider the vast sums that governments spend — and the responsibility public officials bear — it seems crazy that policy makers don’t routinely make good use of evidence. Many promising programs have suffered as a result. Since 1990, 10 major federal initiatives including Head Start, Job Corps, Upward Bound, and 21st Century Community Learning Centers have been evaluated with large randomized studies — and all but one revealed modest or no impacts. That doesn’t mean they never work. These funding streams support a wide range of approaches, some successful, some not. If more information had been available along the way, the government could have made adjustments to build on success, and reform or eliminate failure. Without timely and reliable feedback, how can we hope to improve government?
There are reasons why the evidence movement may take root in government today — despite the fact that data is no match for emotion in an election year, and many lobbyists, campaign donors, party leaders, constituents and ideologues will ignore evidence that contradicts their interests or beliefs. To begin with, money is tight and the stakes are high, whether we’re talking about poverty, unemployment, or crises in education and health care. “Because of overall budget constraints we are in a moment where everyone feels the imperative to do more with less,” explained Robert Gordon, the budget office’s executive associate director. “It has created a sense of urgency.”
Second, it’s come to light that rigorous research doesn’t have to be expensive, particularly if it makes use of administrative data that governments already collect ( pdf) like test scores, graduation rates, emergency room visits, arrests, unemployment records, and so forth. The savings can, in turn, be significant.
Consider the New York City Schoolwide Performance Bonus Program, which Mayor Bloomberg launched to considerable fanfare in 2007. The $75 million program aimed to provide bonuses (of about $3,000 each) to teachers if they increased student achievement and other outcomes.
At the outset, a researcher named Roland Fryer suggested that the city select schools via random assignment, using a control group for comparison. Since the city would already be collecting data about school outcomes, it would add little cost. It turned out to be a good idea. When results were tracked three years later, they showed no difference in education achievement between schools that got the incentives and those that didn’t. The research even suggested that the incentives may have backfired. Total cost of the study: $50,000.
“With low-cost random controlled trials, you could begin testing hundreds of program models and spur a whole lot of innovation,” explains Jon Baron, president of the Coalition for Evidence-Based Policy, a national leader in this movement.
Without evidence, we rely on anecdote, ideology or faith. That’s the way things were done before the Enlightenment, when bloodletting was believed to cure everything from acne to epilepsy. But science moves slowly. What we consider scientific evaluation today (i.e., treatment and control groups and random-assignments) is actually relatively new. The first randomized study of a medication, streptomycin, was published in 1948, and it was only in 1962, after birth defects caused by thalidomide, that the government began requiring pharmaceutical companies to demonstrate “ substantial evidence of effectiveness” of drugs. Only in recent decades has evidence-based medicine emerged as an important movement in health care, challenging the idea that the doctor always knows best. (She may, but only after consulting the meta-analyses.) Baseball diehards will recognize the parallels to sabermetrics and Billy Beane’s “moneyball.”
Social policy has been mainly on the sidelines of this movement. In the mid 1970s, a group called MDRC began conducting large scale evaluations of social policies. Under presidents Reagan and Clinton, there were several notable randomized studies that examined welfare reform policies. However, by and large, rigorous evidence has been used sporadically by governments. “With the vast majority of federal funds, the government plays the role of a faucet, allocating large streams of funding to state and local organizations, often through a formula or competitive process, where evidence of effectiveness plays little role,” says Baron.
The seeds for the budget office’s policy were planted during the Bush administration, when Robert Shea, a political appointee at the office, led the way developing a questionnaire to assess the management of government programs. “One of the questions was to what extent has an evaluation been done to find whether the program had been effective,” recalled Shea. Baron of the Coalition for Evidence-Based Policy and David Olds, the founder of the Nurse-Family Partnership (which I reported on in my last Fixes column), helped Shea provide guidance to agencies about the key differences between rigorous and non-rigorous evaluations (which are less reliable and easier to spin).
Under the Obama administration the focus on evidence has already been ramped up considerably. The administration has brought this focus to home visitation, teen pregnancy prevention, educational innovation, social innovation, career training and workforce innovation — programs that account for close to $4 billion of spending.
What’s different is not just that these programs have to be evaluated using scientific methods, but that funding is being tied to evidence-based models in the legislation. In the case of the administration’s home visitation initiative, 75 percent of the funding must go to support programs that have been shown to produce results.
Last year, the president promised to reform Head Start along similar lines. Among Head Start grantees, there are high performers and low performers. The problem is that, historically, the government hasn’t known which is which — or insisted that states find out and allocate funds accordingly. It’s as if an investor failed to consider profitability when buying stocks. “Under the new rule, programs are going to be regularly evaluated against a set of clear, high standards,” President Obama declared, adding that funding will go to programs that work and will be taken away from those that don’t.
RELATED More From FixesRead previous contributions to this series.
Evidence isn’t important just for accountability; it’s essential for innovation. Consider a studyconducted in 2008 to test whether college enrollment could be boosted by simplifying the financial aid application (FAFSA) process. During tax season, researchers arranged for a subset of low- to moderate-income families (with young adults at home) to receive assistance at 156 H&R Block tax preparation offices in Ohio and North Carolina. Using tax information, the H&R Block representative automatically pre-populated the FAFSA (which has more than 100 questions and can take hours to complete). The representative then conducted a short interview to complete the form, told the families how much aid they would qualify for, provided tuition information for four local public colleges, and offered to submit the form immediately to the Department of Education. The intervention cost less than $90. Youths whose families received help were 29 percent more likely to attend college for at least two consecutive years. That’s a huge gain for a tiny outlay of effort. The big question is: How do we make sure that evidence like this makes it into policy systematically?
At the state level, the evidence movement is also advancing, led by the Washington State Institute for Public Policy, a neutral research body that assembles evidence on a wide array of social programs — and then translates the findings into a user-friendly analysis so even the most number-challenged lawmaker can figure out the cost and benefits. For example, according to WSIPP’s most recent “Return on Investment” report, among programs that focus on juvenile justice, the highest yielding investment is Functional Family Therapy (with a net present value of $67,108) and the worst bet is a program called Scared Straight (which yields a net loss for society of $5,014).
There are disputes, of course, about the way the institute evaluates evidence and does its calculations. But lawmakers have been using its evidence-based guidelines for a decade and a half, and in that time Washington has made significant gains, particularly in the area of criminal justice, noted Steve Aos, the institute’s director (who is careful not to claim credit). Since 1990, he said, Washington’s arrest rate has dropped by 60 percent, against a 35 percent drop at the national level. And while the nation’s incarceration rate had increased by 260 percent since 1980, Washington’s has gone up 150 percent.
The Pew Charitable Trusts is now working to spread the Washington model to other states through an initiative it calls Results First. “There’s a real potential in this approach to deal with the polarization in the country,” says Gary VanLandingham, the director of Results First. “At the end of the day whether you want big or little government, you want government to do things that work.”
As Robert Shea recalled from his days in the Bush administration: “We were accused of cloaking our efforts to gut government in program evaluation. Everybody ought to be advancing the theory that we want programs to work. But a Democratic administration more naturally disposed to defend social programs is going to have a greater degree of trust when they’re advancing these concepts.”
Join Fixes on Facebook and follow updates on twitter.com/nytimesfixes. |
