Natural Randomized Experiments

A theme of this blog is the challenge and pleasures of finding convincing ways to measure what causes what. While qualitative research, lab experiments, field data, and all other methods have their place, the most convincing statistical evidence of causality comes from experiments with randomization. When randomization is in the lab, it is less convincing that it applies outside the lab. Thus, the sweet spot is naturally occurring randomization.

One of the most famous example among social scientists is the Vietnam draft lottery that affected the probability of military service in a nearly random fashion. Barry Staw used that natural randomized experiment to find convincing evidence of cognitive dissonance effects. Josh Angrist (1990) has used it to study how Vietnam-era service affects education, earnings, and other outcomes.

Others have studied:

• Winners of public lotteries (e.g., Lindahl 2002)
• Private school voucher programs that use lotteries to choose from a waiting list (e.g., Angrist and others, 2001)
• Several nations use lotteries for some immigrants (e.g., Stillman, Gibson & McKenzie, 2007)
• Each year TV offers a host of new game shows, many of which have some randomization within them (e.g., Blavatskyy and Pogrebna 2007)
• Sometimes the order on a ballot is chosen randomly (Ho & Imai, 2004)
• Some colleges select roommates somewhat randomly(e.g., Sacerdote 2001)

Many other random or near-random allocation rules exist, most of which have been studied by at least some social scientists (so please send your favorite citations):

• Any other times people line up and there is a lottery for who goes first.

• This procedure is used sometimes when non-profits or government programs are
  giving out free or discounted goods or services. (A recent, as of 2007 example, of apartments in San Francisco)
• Do any transplant waiting lists have lotteries?
• Administrative rules often have randomization, e.g.:

• Select which three judges hear a case from a panel of Appeals Court
  judges or referees in a sporting event

  Kick-off in football and other sports events

  Who is next to whom in some geographic allocations

  Who is in your class or who is your teacher

  Who competes with whom in one round of a race with qualifying heats, sports
  tournament, bridge, or golf tournament

What are other truly randomized allocations that social scientists should use to create convincing field studies?