Manipulation of information and/or behavior near salient points

A key assumption for the regression discontinuity model is that those just above and just below the threshold are otherwise almost identical (on average). That assumption won't hold if people know their potential score and then can work to change it. For the regression discontinuity analysis, sometimes all you need is an initial score, prior to any manipulation. More interestingly, the distribution of scores near the cutoff tells us about the presence of manipulation. As such, non-random heaping can provide fertile ground for studying when and how people manipulate information and behavior in response to the incentives provided at the threshold.

Examples include:

• Earnings just barely beating benchmarks. Three salient benchmarks are, positive profits (not a loss), last year’s earnings, and analyst's earnings expectations (Zeckhauser and others)
• Do incumbents win close votes? (Jason Snyder)
• Do lots of cars just barely pass their state's air pollution standard (Jason Snyder and others)
• Do lots of poor people barely qualify for welfare? (Emily Conover & Adriana Camacho “Manipulation of Social Program Eligibility" 2008)
• Do lots of students just barely pass a high-stakes exam?
• Are lots of employers just below the size threshold or emissions threshold that triggers stricter regulation?
• Do husbands report earnings just greater than their wives more often than wives report?
• What else?

A general approach is to study all potential regression discontinuity designs and see if the key assumption is met or if there is heaping on one side of the cutoff. If such heaping exists, an interesting question is whether it is due to information distortion or behavior change and, if the latter, if that behavior change is desired or not.

Regression discontinuity

Lots of the world is based on being high on a threshold (e.g., good students get scholarships, top ten get photos in the paper) or low (poor people get welfare, coming in close only counts in horse shoes). On average those who are good enough students to get a scholarship or poor enough to be on welfare are very different from those who do not get scholarships or welfare. But, right at the cutoff point, they tend to be almost identical. Thus, we can study those right above and below an arbitrary cutoff and see the causal effect of an intervention (at least for those right near the cutoff; Campbell and Stanley, 1963).

Here are some examples - please submit your own (plus citations to scholars who have utlized these).

Ability

Any absolute bar you can analyze above vs. below, such as those that use test scores or other ratings, perhaps in a numerical combination, to create a bar with "above" and "below". Examples include:

• High school exit exams
• class rank in states where top X% in each school get admission to elite university
• College entrance decisions, particularly in poor nations with a single national university
• Scholarships (e.g., “Incentives to Learn,” Ted Miguel, Michael Kremer, Rebecca Thornton)
• Job qualifications

• civil service tests

  Height or weight eligibility for military, police, etc

• Ratings that divide continuous quality scores into categories: Very good, good,… as in Consumer Reports
• FICA scores for loan eligibility

Disadvantage

• Poverty score => welfare eligibility
• Problematic medical test => care management
• Car smog test
• Slightly higher risk factors mean a prisoner attends a high-security prison, not a low-security prison. (Chen and Shapiro 2007)
• Regulatory rules with cut-offs

• such as employment of at least 10 or 25

  Slightly higher average air pollution can lead to more stringent regulations
  under the Clean Air Act (Ken Chay, Michael Greenstone, JPE 2005)

• Regulations on staffing

• 41 children in a grade means 2 classrooms of 20 & 21, not 1 classroom of
  40 (Angrist & Lavy, QJE 1999).

  Similar rules sometimes hold for daycare, nurse staffing, etc.

• Time
  
• Unemployment insurance runs out at 6 or 9 months.
• Welfare programs with various time requirements and cutoffs
• Laws and union contracts with employee probationary periods of N days
• School attendance
• In late June they were medical students and interns, while in early July they are interns and medical residents. What happens to medical care at graduation?

Age

Large shifts in behavior at a certain age are plausibly related to rules that affect that age.

• 5 or so and school starting
• Religious celebrations

• Confirmation for Catholics

  Bar & Bat Mitzvah for Jews

• Some welfare rules in Quebec depend on age (Lemieux and Milligan 2007)
• 16 or 18 by state: eligibility for a driver's license
• 18, formerly 21, voting
• 21, often formerly 18: Drinking legally
• 40: ADEA eligibility prohibiting age discrimination
• Medicare eligibility

Space

• 20 mile EEOC definition of “labor market”
• 100 km. rule on Maquilas in Mexico
• 300 mile law of the sea exclusive economic zone
• Special economic zones and export processing zones
• Boundaries of school districts (esp. when they do not match city limits)
• Others?
  

Elections have a discontinuity at 50% + 1 (approximately)

• How do Democratic and Republican winners with 50.01% of the vote behave and get re-elected? [David Lee and Enrico Moretti]
• Unions => firm survival (Ken Chay and David Lee)

N+1 place in a contest

• Second place winners are often not much different from first place winners. If the contest is a signal (but not valuable in and of itself), then there is some causal effect of the contest.
• Second place in

• Procurement contracts

  Musical contests

  Competition to land a large employer [Moretti and xx]

• Organizational rankings

• N+1th best when N get listed. For example, 101st best firm in Best 100 places for a woman to work

  Fortune 501st leads to fewer getting analyst cover it, lower stock market
  volumes, etc.

  11th most volatile or highest volume stock when 10 biggest increases and declines get listed and are visible

  CalPERS list of 10 firms to target for improvements in corporate governance

  Region chosen to host a factory (“$million plants") (Moretti and studied)

  Corporation that loses vs. wins a big procurement contract

Two heuristics for normal science:

1. Find a cutoff others have studied. Find additional outcomes of interest and some theory linking the cutoff and the outcome. Write a paper.

2. Consider a theory of interest to you. Find a relevant intervention that has some allocation rule. Most of that rule makes sense, so you will have endoengous treament (e.g., for the most needy). At the margin, there is usually some arbitrariness. Study that arbitrariness!

Natural Randomized Experiments

A theme of this blog is the challenge and pleasures of finding convincing ways to measure what causes what. While qualitative research, lab experiments, field data, and all other methods have their place, the most convincing statistical evidence of causality comes from experiments with randomization. When randomization is in the lab, it is less convincing that it applies outside the lab. Thus, the sweet spot is naturally occurring randomization.

One of the most famous example among social scientists is the Vietnam draft lottery that affected the probability of military service in a nearly random fashion. Barry Staw used that natural randomized experiment to find convincing evidence of cognitive dissonance effects. Josh Angrist (1990) has used it to study how Vietnam-era service affects education, earnings, and other outcomes.

Others have studied:

• Winners of public lotteries (e.g., Lindahl 2002)
• Private school voucher programs that use lotteries to choose from a waiting list (e.g., Angrist and others, 2001)
• Several nations use lotteries for some immigrants (e.g., Stillman, Gibson & McKenzie, 2007)
• Each year TV offers a host of new game shows, many of which have some randomization within them (e.g., Blavatskyy and Pogrebna 2007)
• Sometimes the order on a ballot is chosen randomly (Ho & Imai, 2004)
• Some colleges select roommates somewhat randomly(e.g., Sacerdote 2001)

Many other random or near-random allocation rules exist, most of which have been studied by at least some social scientists (so please send your favorite citations):

• Any other times people line up and there is a lottery for who goes first.

• This procedure is used sometimes when non-profits or government programs are
  giving out free or discounted goods or services. (A recent, as of 2007 example, of apartments in San Francisco)
• Do any transplant waiting lists have lotteries?
• Administrative rules often have randomization, e.g.:

• Select which three judges hear a case from a panel of Appeals Court
  judges or referees in a sporting event

  Kick-off in football and other sports events

  Who is next to whom in some geographic allocations

  Who is in your class or who is your teacher

  Who competes with whom in one round of a race with qualifying heats, sports
  tournament, bridge, or golf tournament

What are other truly randomized allocations that social scientists should use to create convincing field studies?

Why Blog?

This blog will collect my thoughts, with an emphasis on projects I hope others might carry out.

I have lots of ideas. I have many more ideas than I have good ideas. I am hoping if I post lots of ideas, others will find the good ones and implement them.

Many of my ideas are not original. This blog will be short on citations, as I will not look up each predecessor. At the same time, I greatly appreciate citations and links that help flesh out ideas, show where someone has pushed this idea far past my tentative thoughts, or perhaps just the source from which I stole an idea (perhaps inadvertently).

If anyone pursues any of these ideas, please send me an update on how it is going.