Publications
 CALDER researchers are tapping into state administrative longitudinal databases- what is emerging as the richest new source of information about schools, teachers, and students in the U.S.- to address some critical questions about educational policy.CALDER research focuses most centrally on how teacher policies (hiring practices, compensation, and certification); governance policies (accountability and choice) and changing social and economic community conditions (changing student demographics and resources) affect outcomes for both teachers and students. We explore these questions across different states. Below you'll find all working papers and policy briefs. Research by state: Florida | Missouri | New York | North Carolina | Texas | Jump to: Working Papers Policy Briefs Do Disadvantaged Urban Schools Lose Their Best Teachers? Policy Brief 7 Author(s): Eric A. Hanushek, Steven G. Rivkin This research brief examines differences in teacher effectiveness by school transition status and school characteristics in a large urban school district in Texas, using estimates of teacher effectiveness based on teacher contributions to student learning outcomes across classrooms. This research finds little or no evidence to support the view that more effective teachers have higher exit probabilities. In fact, the study finds that teachers who exit are significantly less effective, on average, than those who stay.
This important research explores the effects of district policy interventions on the distribution of teacher qualifications and student achievement. Authors use a 5-year span of individual teacher- and student-level longitudinal data from New York City (NYC) from 2000 through 2005 to estimate the differences in the effectiveness of teachers entering NYC schools through different pathways to teaching. The study finds that the gap between the qualifications of NYC teachers in high-poverty and low-poverty NYC schools has narrowed substantially since 2000, mostly ensuing from the city's concentrated effort to match exceptionally capable teachers with very needy students and the virtual substitution of newly hired uncertified teachers in high-poverty schools with new hires from alternative certification routes: NYC Teaching Fellows and Teach for America. The Texas FERPA Story Policy Brief 5 Author(s): Dan M. O'Brien This research brief describes the legal and operational structure of the Texas longitudinal data system related to recent changes in the Family Educational Rights and Privacy Act of 1974 (FERPA)—which establishes the rights of parents to access their children's educational records and protects the confidentiality of student information—that more closely align law and practice. The U.S. Department of Education's FERPA Final Regulations Amendments took effect January 8, 2009.
There is little doubt that teacher quality is a key determinant of student achievement, but finding ways to identify and reward the best teachers has proven illusive. This research brief considers the stability of value-added measures of teacher effectiveness over time and the resulting implications for the design and implementation of performance-based teacher compensation schemes. Published: November 2008 | Availability:
Using individual teacher and student-level longitudinal data from North Carolina, this research brief presents selected findings from work examining the stability of value-added model estimates of teacher effectiveness, focusing on their implication for teacher tenure policies and making high stakes personnel decisions. Findings show year-to-year correlations in teacher effects are modest, but pre-tenure estimates of teacher job performance do predict estimated post-tenure performance in both math and reading, and would therefore seem to be a reasonable metric to use as a factor in making substantive teacher selection decisions. Published: November 2008 | Availability: Overview of Measuring Effect Sizes: The Effect of Measurement Error This research brief estimates the overall extent of test measurement error and how this varies across students using New York City student- level longitudinal data across grades 3-8 from 1999- 2007. Results reinforce the importance of accounting for measurement error, as it meaningfully increases effect size estimates associated with teacher attributes. There are important differences in teacher effectiveness that are systematically related to observed teacher attributes. Such effects are important in the formulation and implementation of personnel policies. Also, effect sizes as traditionally measured have led analysts to understate the magnitudes of effects because the standard deviation of observed scores overstates the dispersion of true achievement in the student population. Published: November 2008 | Availability: Value-Added Analysis and Education Policy Policy Brief 1 Author(s): Steven G. Rivkin This brief describes estimation and measurement issues relevant to estimating the quality of instruction in the context of a cumulative model of learning. The discussion highlights the importance of accounting for student differences and the advantages of focusing on student achievement gains as opposed to differences in test scores.
This brief calculates graduation rates for the state of Florida using longitudinal data. We describe our measurement strategies and compare them with the state’s official measurement procedures. We calculate the diploma and GED attainment rates of six separate cohorts of Florida 9th graders who began high school between 1995/96 and 2000/01. We then present rates of both diploma receipt and GED receipt at four years and in later years. The results show an increasing trend in graduation rates in the state over the period studied and a substantial bump at five years, with growth flattening out after that time. Who Leaves? Teacher Attrition and Student Achievement Teacher attrition has attracted considerable attention as federal, state and local policies- intended to improve student outcomes, increasingly focus on recruiting and retaining more qualified and effective teachers. But policy makers are often frustrated by the seemingly high rates of attrition among teachers earlier on in their careers. This paper analyzes attrition patterns among teachers in New York City elementary and middle schools and explores whether teachers who transfer among schools, or leave teaching entirely, are more or less effective than those who remain. Findings show first-year teachers who are less effective in improving student math scores have higher attrition rates than do more effective teachers. This raises important questions about current retention and transfer policies.
This paper describes the school mobility rates for elementary and middle school students in North Carolina and attempts to estimate the effect of school mobility on the performance of different groups of students using student fixed effects models. School mobility is defined as changing schools at times that are non-promotional (e.g., moving from middle to high school). We used detailed administrative data on North Carolina students and schools from 1996 to 2005 and followed four cohorts of 3rd graders for six years each. School mobility rates were highest for minority and disadvantaged students. School mobility rates for Hispanic students declined across successive cohorts, but increased for Black students. Findings on effects were most pronounced in math. School mobility hurt the math performance of Black and Hispanic students, but not the math performance of white students. School mobility improved the reading performance of white and more advantaged students, but had no effect on the reading performance of minority students. “Strategic” school moves (cross-district) benefitted or had no effect on student performance, but “reactive” moves (within district) hurt all groups of students. White and Hispanic students were more likely to move to a higher quality school while Blacks were more likely to move to a lower quality school. The negative effects of school mobility increased with the number of school moves.
No Child Left Behind judges the effectiveness of schools based on their students' achievement status. However, many policy analysts argue that schools should be measured, instead, by their students' achievement growth. Using a ten-year student-level panel dataset from North Carolina, we examine how school-specific pressure related to two school accountability approaches (status and growth) affects student achievement at different points in the prior-year achievement distribution. Achievement gains for students below the proficiency cut point emerge in response to both types of accountability systems. We find little or no evidence that schools in North Carolina ignore students far below proficiency under either approach. Importantly, we find that the status, but not the growth, approach reduces the reading achievement of higher performing students, with the losses in the aggregate exceeding the gains at the bottom. The distributional effects of accountability pressure depend on the type of pressure for which schools are held accountable and the tested subject.
There are fierce debates over the best way to prepare teachers. Some argue that easing entry into teaching is necessary to attract strong candidates, while others argue that investing in high quality teacher preparation is the most promising approach. Most agree, however, that we lack a strong research basis for understanding how to prepare teachers. This paper is one of the first to estimate the effects of features of teachers' preparation on teachers' value-added to student test score performance in Math and English Language Arts. Our results indicate variation across preparation programs in the average effectiveness of the teachers they are supplying to New York City schools. In particular, preparation directly linked to practice appears to benefit teachers in their first year.
Value-added models in education research allow researchers to explore how a wide variety of policies and measured school inputs affect the academic performance of students. The impacts of such interventions are typically quantified in terms of effect sizes. We estimate the overall extent of test measurement error and how this varies across students using the covariance structure of student test scores across grades in New York City from 1999 to 2007. Results reinforce the importance of accounting for measurement error, as it meaningfully increases effect size estimates associated with teacher attributes. There are important differences in teacher effectiveness that are systematically related to observed teacher attributes. Such effects are important in the formulation and implementation of personnel policies.
Using a unique longitudinal dataset covering all Florida public school students in grades 3–10 over a five-year period, we analyze the impact of classroom peers on individual student performance. Unlike many previous data sets used to study peer effects in education, our data allow us to identify each member of a given student’s classroom peer group in elementary, middle and high school as well as the classroom teacher responsible for instruction. As a result, we can control for individual student fixed effects simultaneously with individual teacher fixed effects, thereby alleviating biases due to endogenous assignment of both peers and teachers, including some dynamic aspects of such assignments. We find some sizable, significant peer effects within nonlinear models, but not with linear specifications. We find peer effects depend on a student's own ability and on the ability of the peers under consideration. Peer effects tend to be smaller when teacher fixed effects are included, a result that suggests co-movement of peer and teacher quality within a student over time. We also find that peer effects tend to be stronger at the classroom level than the grade level.
Teach for America (TFA) selects and places graduates from the most competitive colleges as teachers in the lowest-performing schools in the country. This paper is the first study that examines TFA effects in high school. We use rich longitudinal data from North Carolina and estimate TFA effects through cross-subject student and school fixed-effects models. We find that TFA teachers tend to have a positive effect on high school student test scores relative to non-TFA teachers, including those who are certified in-field. Such effects exceed the impact of additional years of experience and are particularly strong in math and science.
This paper uses administrative data for the public K-12 schools of North Carolina to measure racial segregation in the public schools of North Carolina. Using data for the 2005/06 school year, the authors update previous calculations that measure segregation in terms of unevenness in racial enrollment patterns both between schools and within schools. They find that classroom segregation generally increased between 2000/01 and 2005/06, continuing, albeit at a slightly slower rate, the trend observed over the preceding six years. Segregation increased sharply in Charlotte-Mecklenburg, which introduced a new choice plan in 2002. Over the same period, racial and economic disparities in teacher quality widened in that district.
Since the 1996/97 school year, North Carolina has awarded bonuses of up to $1,500 to teachers in schools that exhibit test score gains above certain thresholds. This article reviews the details of the bonus program, describes patterns of differences between schools that qualify for bonuses of differing amounts, and presents basic data to address the question of whether the bonus program has improved student achievement, or has led to a narrowing of racial or socioeconomic achievement gaps. There is some evidence to suggest an improvement in overall test scores, particularly in math, but less evidence to suggest that achievement gaps have narrowed.
This paper uses evidence from Durham, North Carolina to examine the impact of school choice on racial and class-based segregation across schools. The findings suggest that school choice increases segregation. Furthermore, the effects of choice on segregation by class are larger than the effects on segregation by race. These results are consistent with the theoretical argument—developed in sociology and economics literature—that the segregating choices of students from advantaged backgrounds are likely to outweigh any integrating choices by disadvantaged students.
This paper examines the effect of accountability policy on school practices and student outcomes with remarkably comprehensive and detailed data that include a multi-wave five-year survey of the census of public schools in Florida and administrative data on individual student performance over time. The authors show that low-performing schools facing accountability pressure changed their instructional practices in meaningful ways. In addition, they present medium-run evidence school accountability promotes improved student test scores, and find that a significant portion of these test score gains can likely be attributed to the changes in school policies and practices uncovered in these surveys.
Most studies that have fueled alarm over the attrition and mobility rates of high-quality teachers have relied on proxy indicators of teacher quality, which recent research finds to be only weakly correlated with value-added measures of teachers' performance. We examine attrition and mobility of teachers using teacher value-added measures for early-career teachers in North Carolina public schools from 1996 to 2002. Our findings suggest that the most-effective teachers tend to stay in teaching and in specific schools. Contrary to common expectations, we do not find that more-effective teachers are more likely to leave more-challenging schools.
One of the first papers to ever estimate teacher effects at the secondary school level, this groundbreaking work presents evidence that teacher credentials affect secondary school student success in systematic ways and to a significant, policy-relevant extent. We use data on statewide end-of-course tests in North Carolina to examine the relationship between teacher credentials and student achievement at the high school level. We find compelling evidence that teacher credentials affect student achievement in systematic ways and that the magnitudes are large enough to be policy relevant. As a result, the uneven distribution of teacher credentials by race and socio-economic status of high school students--a pattern we also document--contributes to achievement gaps in high school.
This important paper explores how the distribution of teacher qualifications and student achievement in New York City have changed from 2000 through 2005 using data on teachers and students. We find: the gap between the qualifications of New York City teachers in high-poverty schools and low-poverty schools has narrowed substantially over this period, and that this gap-narrowing associated with new hires has been driven almost entirely by the substitution of teachers entering through alternative certification routes, for uncertified teachers in high-poverty schools, these changes resulted from a direct policy intervention eliminating unlicensed teachers, and perhaps most intriguing, much larger gains could result if teachers with strong teacher qualifications could be recruited.
This paper explores the relationship between teacher testing and teacher effectiveness using a unique dataset that links teachers to their individual students. My findings show a positive relationship between some teacher licensure tests and student achievement. But they also suggest that states face significant tradeoffs when they require particular performance levels as a precondition to becoming a teacher: some teachers whom we might wish were not in the teacher workforce based on their contribution toward student achievement are eligible to teach based on their performance on these tests, while other individuals who would be effective teachers are ineligible.
This paper is the first to systematically document the relationship between individual teacher performance incentives and student achievement using United States data. We combine data from the National Education Longitudinal Survey with original survey data regarding the use of teacher incentives. We find that test scores are higher in schools offering individual financial incentives for good performance. Moreover, the relationship between the presence of merit pay and student test scores is strongest in schools that may have the least parental oversight. The association between teacher incentives and student performance could be due to better schools adopting teacher incentives or to teacher incentives eliciting more effort from teachers.
This paper is the first to explore the effects of school accountability systems on high-achieving students' long-term performance. Using data from a large state university, we relate school accountability pressure in high school to a student's university-level grades and study habits. We find that an accountability system based on a low-level test of basic skills apparently led to reduced performance by high-achieving students, while an accountability system based on a more challenging criterion-referenced exam apparently led to improved performance in college on mathematics and other technical subjects. Both types of systems are associated with increased "cramming" by students in college. The results indicate that the nature of an accountability system can influence its effectiveness.
Defined Benefit pension plans often generate odd time patterns of benefits. One typical pattern exhibits low accrual in early years, accelerating in mid-late years, followed by dramatic decline, or even negative returns in years that are relatively young for retirement. We consider four states for specific analysis: Arkansas, Missouri, California and Massachusetts. There are interesting variations among these states’ formulas, which affect the incentive to retire early. We identify key factors in the defined benefit formulas that drive such patterns and likely consequences for employee behavior. We examine the efficiency and equity consequences of these systems and lessons that might be drawn for pension reform.
This paper examines late career mobility and retirement decisions for a cohort of mid-career Missouri public school teachers. The rate of accrual of traditional defined benefit pension systems is highly nonlinear and back-loaded with most of the gain occurring in the final years prior to retirement. Also, these pension systems have rules that introduce kinks or discontinuities in the rate of accrual after 30 years. This paper explores the effect of these pension rules on retirement patterns. Missouri permits teachers to continue teaching part-time while collecting benefits. Teachers can also retire from one pension system and begin teaching in another. The paper examines both types of behavior.
This study considers the efficacy of a certification system for teachers established by the National Board for Professional Teaching Standards (NBPTS). The authors utilize a four-year span of the longitudinal data from Florida to determine the relationship between teacher NBPTS certification and student test scores on low-stakes and high-stakes exams. They find evidence that NBPTS certification provides a positive signal of teacher productivity in some cases, but it is highly variable. The process of becoming NBPTS certified does not appear to increase teacher productivity nor do NBPTS-certified teachers appear to enhance the productivity of their colleagues. Read commentary on this working paper by Mary Dilworth of the NBPTS, Daniel McCaffrey of the RAND Corporation, and CALDER researchers, Helen F. Ladd and Tim R. Sass. Teacher Training, Teacher Quality and Student Achievement Working Paper 3 Author(s): Douglas N. Harris, Tim R. Sass Using longitudinal data from the state of Florida, this study examines the effects of various types of education and training on the ability of teachers to promote student achievement. It suggests that teacher training generally has little influence on productivity. One exception is that content-focused teacher professional development is positively associated with productivity in middle and high school math. In addition, more experienced teachers appear more effective in teaching elementary and middle school reading. There is no evidence that either pre-service (undergraduate) training or the scholastic aptitude of teachers influences their ability to increase student achievement.
In this paper, the authors use a ten-year span of longitudinal data from North Carolina to explore a range of questions related to the relationship between teacher characteristics and credentials, on the one hand, and student achievement on the other. They conclude that a teacher's experience, test scores and regular licensure all have positive effects on student achievement, with larger effects for math than for reading. Taken together the various teacher credentials exhibit quite large effects on math achievement, whether compared to the effects of changes in class size or to the socio-economics characteristics of students. Read commentary on this working paper by Mary Dilworth of the NBPTS, Daniel McCaffrey of the RAND Corporation, and CALDER researchers Helen F. Ladd and Tim R. Sass.
The central question for this study is how the quality of the teachers and principals in high poverty schools in North Carolina compares to that in the schools serving more advantaged students. A related question is why these differences emerge. The consistency of the patterns across many measures of qualifications for both teachers and principals leaves no doubt that students in the high poverty schools are served by school personnel with lower qualifications than those in the lower poverty schools. Moreover, in many cases the differences are large. Additional evidence documents that the differences largely reflect predictable outcomes of the labor market for teachers and principals.
| |
 | |
