Urban Institute analysis of longitudinal data in education research
A program of research by the Urban Institute with Duke University, Stanford University, University of Florida, University of Missouri-Columbia, University of Texas at Dallas, and University of WashingtonUrban Institute



CALDER Research Methods

CALDER capitalizes on the richest new source of information about schools, teachers, and students in the United States, state administrative databases based on census files that include all districts, schools, teachers, and students over time.

Policies instituted at one time may target different districts, schools, teachers, or students than policies instituted at a later time. Schools and districts in different situations may respond in different ways. Because census files do not confine us to a sample selected for a specific research question, we can estimate the effects of policies as they emerge and target our analysis to wherever we expect consequences to be felt.

Our primary data come from Texas, Florida, North Carolina, New York, Missouri, and Washington—states at the vanguard of developing comprehensive education data systems. Data from other states and districts supplement our work as they offer opportunities to study particular topics. Our data allow us to conduct the following types of studies:

Replication studies:
Analyses conducted in one state are confined to the data available in that state, and interpretation is confined to the conditions associated with that state. Replication in a different state, with perhaps different measures and with a different context, can pinpoint intermediary factors and the robustness of effects.

Longitudinal studies:
CALDER's data extend into the past as well as the present, enabling us to take history into account in our analyses. Because we have longitudinal data, we are particularly well positioned to examine how changing student demographics may, for example, influence the flow of teachers in and out of schools and school districts, as well as new policies to attract and retain quality teachers.

Our analytic strategies include:

While experimental data is the gold standard of statistical inference, it is hard to come by, in part because the costs (both pecuniary and political) of experiments often result in small samples that may not have enough statistical power or external validity to quantify effects with any confidence. Other statistical techniques, including the quasi-experimental methods listed above, offer the potential of identifying the treatment effect in a non-experimental setting. All quasi-experimental methods benefit from the availability of longitudinal data, since the outcome variable can be measured in changes instead of levels, and in this way, other confounding factors can be eliminated from our estimates.