Tag: Research

The Evidence Is In: School Choice Works

There are a great many reasons to support educational choice: maximizing freedom, respecting pluralism, reducing social conflict, empowering the poor, and so on. One reason is simply this: it works.



That’s So Meta

One of the main advantages of a meta-analysis is that it can overcome the limitations of individual studies (e.g., small samples sizes) by pooling the results of numerous studies. This meta-analysis is especially important because it includes all random-assignment studies on school choice programs (the gold standard for social science research), while excluding studies that employed less rigorous methods. The analysis included 19 studies on 11 school choice programs (including government-funded voucher programs as well as privately funded scholarship programs) in Colombia, India, and the United States. Each study compared the performance of students who had applied for and randomly won a voucher to a “control group” of students who had applied for a voucher but randomly did not receive one. As Wolf explained, previous meta-analyses and research reviews omitted some gold-standard studies and/or included less rigorous research:


Survey Says: School Choice Improves Student Performance


Is School Choice Worth Celebrating? A Look at the Evidence







Education Policy, The Use of Evidence, and the Fordham Institute

In recent weeks, the Fordham Institute has repeatedly called for government testing and reporting mandates to be imposed on private schools participating in school choice programs (here and here), on the grounds that such “public accountability” improves private school academic outcomes. In defense of this claim, the Fordham Institute cites a study of Milwaukee’s voucher program in which test scores rose following the introduction of such mandates.

Patrick Wolf, director of the research team that conducted the study, has now responded, explaining that his team’s results do not necessarily support Fordham’s claim:


What about the encouraging trend that lower-performing schools in the MPCP are being closed down?  [Fordham] mentions that as well and attributes it to the stricter accountability regulations on the program.  That phenomenon of Schumpeterian “creative destruction” pre-dated the accountability changes in the choice program, however, and appears to have been caused mainly by low enrollments in low-performing choice schools, as parents “voted with their feet” against such institutional failure. Sure, the new high-stakes testing and public reporting requirements might accelerate the creative destruction of low-performing choice schools in Milwaukee, but that remains to be seen. [emphasis added]

But there is a deeper problem with the Fordham claim, to which Wolf alludes: a single study, no matter how carefully executed, is not a scientific basis for policy. Because a single study is not science. Science is a process of making and testing falsifiable predictions. It is about patterns of evidence. Bodies of evidence. Fordham offers only a toe.

And Fordham’s preferred policy not only lacks a body of supporting evidence, it is undermined by a large body of evidence. When I reviewed the within-country studies comparing outcomes among different types of school systems worldwide in 2009, I sorted the results into two categories: 1) all studies that compared “public” schools to “private” schools, where those terms were loosely defined; and 2) studies that compared “market” schools to “monopoly” schools. “Market” schools were those paid for at least in part directly by parents and only minimally regulated. “Monopoly” schools were public school systems such as those common in the U.S.

The purpose of these separate categorizations was to see if limited regulation and direct parent funding make a real difference, or if private schools that are paid for entirely by the state and subjected to Fordham’s “public accountability” have the same advantages as their more market-like counterparts.

The result of this breakdown of the literature was stark. Studies looking at truly market-like education systems are twice as consistent in finding a private sector advantage as those looking at “private” schools more broadly construed (and thus including state-funded and regulated private schools).

The pattern of evidence thus seems to contradict Fordham’s belief in the merits of “public accountability” in market education systems. What it favors are policies that promote the rise of minimally regulated education markets in which parents pay at least some of the cost of their own children’s education directly themselves, whenever possible.  That’s just the sort of system likely to arise under education tax credit programs.

School Choice Lowers Crime



But Don’t We Really Need Government Research?


That’s a response to the argument I made last week against government funding of scientific research. Moving away from public funding of scientific research would solve the problem of private companies capturing publication spoils from research that taxpayers funded.

The Defense Advanced Research Projects Agency did indeed come up with and popularize the protocol called TCP/IP, which the Internet uses. (Everyone’s use of the protocol really makes the Internet what it is, of course, but nevermind that.)

To take the Internet as proof that the government is a necessary producer of research and innovation, you have to reject the scientific method. Unfortunately, there are rarely controls in public policy. We can’t find out what would have happened if government policy had taken a different course, so we don’t know anything more about who should fund research from the fact that government-funded research has produced good things in the past.

But what would have happened if U.S. public policy had taken a different course? I’ve thought about the impossible-to-answer question of where we would have been without DARPA and other government influences on telecom. What most people don’t consider, I believe, is the restraining influence the government-granted AT&T monopoly had on telecommunications for most of the 20th century. AT&T developed a “Teletypewriter Exchange” system in 1931, for example, but had no need to develop it, there being little or no competitive pressure to do so. (Its patent on attaching devices to phone wires undoubtedly helped as well, preventing anyone using AT&T’s wires for modem service.)

Had there been competition, I suspect that someone would have come up with the idea of packet-switched networks—that’s what the Internet is—before Leonard Kleinrock did in 1962. Kleinrock was a student at MIT—he wasn’t at DARPA, which didn’t get into packet-switching until about 1966. (Then again, MIT was almost certainly awash in government money—specifically military money—so there you go. Maybe we owe all the good things we’ve got to war, but I doubt it.)

My guess—and it’s only that—is that we would have had the Internet some decades earlier if not for government interventions in telecommunications. We probably would have had multiple, competing “Internets,” actually, adopted more slowly than the Internet we got. (In a chapter of Privacy in America: Interdisciplinary Perspectives, I explored how government has accelerated the development of computing and communications, overpowering society’s capacity to adjust, with negative consequences for privacy.)

Support for government-funded research requires one to elide opportunity costs, the things foregone when one thing is chosen. As I said before, tradeoffs are ineluctable: Money spent on government research takes away from private research, or from other priorities such as reducing debt. In the absence of taxation to support research, the money would go to the public’s priorities as determined directly by the public in manifold spending and investing decision. Taxation and spending on government research is merely the substitution of centralized, political decision-making for a distributed, direct decision-making system. Its supporters are generally going to be beneficiaries of that system—elites, in short.

Even these beneficiaries of the status quo tend to agree that political decisions about funding for scientific research are warped. The solution to that problem, they’ll say, is fixing the political system—that is, creating a political system that is not so political.

Such a breakthrough is as unlikely as the invention of water that is not wet. Perhaps we can put DARPA on both projects.

Open Government Research—-or Maybe Private Ordering

I came across an interesting information policy scuffle yesterday. It’s worth knowing about in general, and I’ll share my liberconoclastic view of things below.

Congressman Darrell Issa (R-CA) has introduced a bill called the Research Works Act. The consensus is that it’s meant to keep government-funded research from being published for free. This would keep the publication of that research going through scholarly and scientific journals, neatly maintaining profits for an industry that society might not need while restricting public access to research the U.S. taxpayer paid for. (I have my doubts that the language of the bill actually successfully does that, but that’s inconsequential.)

Here’s a good opponent-side article on the bill. The Association of American Publishers likes the bill.

On a discussion list, Jonathan Band articulated how the business of government-funded research works. It’s helpful to know if you haven’t focused on this area before:

  1. Federal and state governments, directly or indirectly, pay salaries of researchers.
  2. Federal government awards grants for specific research projects. Average NIH grant is around $500,000.
  3. Researcher performs the research and writes a draft article about it.
  4. Researcher submits the draft article to publisher.
  5. Publisher requires the researcher to transfer the copyright in the draft article (for free) before it will touch the draft.
  6. Publisher emails the draft article to other researchers in the field.
  7. These “peers” review the article for free as part of their contribution to the field. (As noted in step 1, their salaries are paid by government.)
  8. The researcher revises the draft in response to the peers’ comments.
  9. Publisher does copy editing and publishes article. Publishers acknowledge that their costs per article are under $5,000.
  10. Publisher sells subscriptions to research libraries, which ultimately are largely government funded.

“In other words,” Band concludes, “the public invests $500,000 in the creation of the article, and the publisher invests under $5,000. Yet, the publisher recoups all the profits from the sale of the article. Profit margins for STM publishers exceed 40%.”

I’m inclined to share these concerns. It appears to be a classic example of regulatory controls—in this case, on information—creating supra-normal rents for a particular business sector.

My conclusion is a little different, though. You see, to me, what Band describes is a situation where researchers—who nobody is paying their own money to hire—are doing research that nobody is paying their own money to produce, which results in journal articles that nobody is paying their own money to read. Privatized profit from government-funded research is as anathema to me as the next open government advocate, but I would solve the problem by letting private ordering decide where research dollars go.

Is this a retrograde argument against research? Who could possibly be against research? Publicly funded research is like nutritious vegetables for a healthy modern society!

Well, I’m against researchers, research, and research results that nobody pays their own money for because it’s demanded by political actors responding to political cues. I would rather have research dollars meted out through private ordering, because then research dollars would go to where they’re most likely to produce the scientific and intellectual gains society actually wants.

Tradeoffs are ineluctable: Money spent on government research takes away from private research, or from other priorities such as reducing debt, or reducing taxes so I can spend my money on things like donating to charity or to the impoverished individual of my choice.

Does Scholar Self-Interest Corrupt Policy Research?


Hess told the Times, among other things, that

