by Daniel F. McCaffrey
The RAND Corporation

The studies by Harris and Sass (2007) and Clotfelter, Ladd, and Vigdor (2007) both seem to have great potential for identifying the effects that a National Board of Professional Teacher Standards certified teacher (NBCT) has on his or her students. Both studies use large datasets with multiple cohorts of students where individual students are tracked over multiple years. The authors of both studies use aggressive econometric modeling to account for the nonrandom assignment of students to classes that tends to result in more advantaged students having a greater probability of being taught by an NBCT. The studies even cover similar time periods. However, despite all the similarities, the two studies come to different conclusions about NBCT.

Across five varied model specifications, Clotfelter, Ladd, and Vigdor (2007) find fairly consistent positive statistically significant effects for NBCTs in both mathematics and reading. The effects are very small—on the order of 0.02 or 0.01 standard deviation units—after controlling for other teacher characteristics. Harris and Sass (2007) find no statistically significant results when looking at the entire population and the signs of their estimated effects depend on the test used as an outcome. However, they find significant effects for reading on one test for some periods when they disaggregate NBCTs by time period (pre-application, application year, or post application).

The source of the differences between these studies may be a result of the sample sizes and estimation methods. Clotfelter, Ladd, and Vidgor have a very large sample from North Carolina with many NBCTs and this might explain some of the differences. The very small effects might not be estimated precisely by the analyses of Harris and Sass due their smaller sample of NBCTs in their sample from Florida. In addition, the model considered by Harris and Sass, which includes both student and school fixed effects, is not considered by Clotfelter and colleagues. The Clotfelter, Ladd, and Vidgor model most similar to the model of Harris and Sass uses gain scores as outcome and includes only student fixed effects rather than both student and school fixed effects. However, in similar contexts the inclusion of school effects did not have a great impact on estimates, so the difference in models might not be the source of the difference.

Thus, it might be that the effects, not just the estimates, are truly different in Florida from what they are in North Carolina. Such a difference might be related to the nature of the tests. Estimates in Florida are sensitive to the specific test, with estimates based on the low-stakes FCAT-NRT smaller than those based on the high stakes FCAT-SSS. The tests in North Carolina are also high stakes. It might be that NBCTs are better able to align their teaching with state standards covered by the high stakes tests.

Difference between the states might also depend on unmeasured characteristics of teachers who apply for NBPTS certification in each state. It could also depend on subtle factors, such as the relationship between NBPTS standards and state standards. NBPTS has a long history of support in North Carolina, and this could have an impact on the teachers who apply and how they perform relative to the state standards. Additional studies that probe these and other hypotheses would be very useful for providing greater understanding of the meaning of NBCTs for student achievement.

It is important to note that the effect of context on the estimated effect of NBCTs is not limited to difference by state or subject; grade level appears to matter as well. The Florida database allows for estimation of effects for elementary, middle school, and high school teachers. In an earlier study of NBCTs in Miami-Dade County Schools, Cavalluzzo (2004) found significant positive effects for NBCTs on ninth and tenth grade students' mathematics achievement and achievement gains as measured by the FCAT-SSS. Harris and Sass (2007) find similar results as reported in their appendix on FCAT-SSS. This suggests that the effects of NBCTs on mathematics might be larger in high school than in elementary school. This finding is consistent with other studies that find certain teacher qualifications had greater effects on high school students where teachers' content knowledge might be particularly important. This interesting finding needs further investigation to confirm that the results in Florida are not spurious, and to determine if they replicate in other contexts.

The disparity of the various findings suggests that we might need to refine the question about NBCTs from "Does board certification signal quality?" to "Under what circumstances does board certification signal quality?" Future research will need to investigate the differences between states to determine the ways in which NBCTs differ, and find creative ways to isolate factors that could be interacting with NBCT effects. This could be very exciting research that could greatly enhance our understanding of teacher qualifications and teachers' performance on increasing student achievement, especially if the future research is as careful, thorough, and thoughtful as these two CALDER reports.

Dr. Daniel F. McCaffrey is a senior statistician with the RAND Corporation. This commentary does not necessarily reflect the opinion of RAND research sponsors.

References

Cavalluzzo, L. C. 2004. Is National Board certification an effective signal of teacher quality? Alexandria, VA: The CNA Corporation.

Clotfelter, Charles T., Helen F. Ladd and Jacob L. Vigdor. 2007. "How and Why Do Teacher Credentials Matter for Student Achievement?" CALDER working paper no. 2.

Harris, Douglas N. and Tim R. Sass. 2007. "The Effects of NBPTS-Certified Teachers on Student Achievement." CALDER working paper no. 4.

Read additional commentary by Mary Dilworth and Helen F. Ladd & Tim R. Sass.