Quasi-Experimental Design

A quasi-experimental design is one that looks a bit like an experimental design but lacks the key ingredient – random assignment. My mentor, Don Campbell, often referred to them as “queasy” experiments because they give the experimental purists a queasy feeling. With respect to internal validity, they often appear to be inferior to randomized experiments. But there is something compelling about these designs; taken as a group, they are easily more frequently implemented than their randomized cousins.

I’m not going to try to cover the quasi-experimental designs comprehensively. Instead, I’ll present two of the classic quasi-experimental designs in some detail and show how we analyze them. Probably the most commonly used quasi-experimental design (and it may be the most commonly used of all designs) is the nonequivalent groups design. In its simplest form it requires a pretest and posttest for a treated and comparison group. It’s identical to the Analysis of Covariance design except that the groups are not created through random assignment. You will see that the lack of random assignment, and the potential nonequivalence between the groups, complicates the statistical analysis of the nonequivalent groups design.

The second design I’ll focus on is the regression-discontinuity design. I’m not including it just because I did my dissertation on it and wrote a book about it (although those were certainly factors weighing in its favor!). I include it because I believe it is an important and often misunderstood alternative to randomized experiments because its distinguishing characteristic – assignment to treatment using a cutoff score on a pretreatment variable – allows us to assign to the program those who need or deserve it most. At first glance, the regression discontinuity design strikes most people as biased because of regression to the mean. After all, we’re assigning low scorers to one group and high scorers to the other. In the discussion of the statistical analysis of the regression discontinuity design, I’ll show you why this isn’t the case.

Finally, I’ll briefly present an assortment of other quasi-experiments that have specific applicability or noteworthy features, including the Proxy Pretest Design, Double Pretest Design, Nonequivalent Dependent Variables Design, Pattern Matching Design, and the Regression Point Displacement design. I had the distinct honor of co-authoring a paper with Donald T. Campbell that first described the Regression Point Displacement Design. At the time of his death in Spring 1996, we had gone through about five drafts each over a five year period. The paper includes numerous examples of this newest of quasi-experiments, and provides a detailed description of the statistical analysis of the regression point displacement design.

There is one major class of quasi-experimental designs that are not included here – the interrupted time series designs. I plan to include them in later rewrites of this material.