Urban Institute analysis of longitudinal data in education research
A program of research by the Urban Institute with Duke University, Stanford University, University of Florida, University of Missouri-Columbia, University of Texas at Dallas, and University of WashingtonUrban Institute



Mixed Results Reflect Complexity of Measuring the Impact of National Board Certification

by Mary Dilworth
National Board for Professional Teaching Standards

Two recent studies on National Board Certification posted on the website of the National Center for Analysis of Longitudinal Data in Education Research found mixed and, in some cases, opposite results regarding the impact of National Board Certified Teachers (NBCTs) on student achievement.

"The National Board for Professional Teaching Standards (NBPTS) is pleased that these two studies are contributing to the rich and growing body of knowledge about the impact of National Board Certification on the complex process ofteaching and learning. Accordingly, it is important to look more closely at these studies to understand why they reach different conclusions. The mixed results, and the different approaches behind them, also support our position that it is unwise to use any one study to gauge the success of teachers on individual student achievement."

The National Board for Professional Teaching Standards (NBPTS) is pleased that these two studies are contributing to the rich and growing body of knowledge about the impact of National Board Certification on the complex process of teaching and learning. Accordingly, it is important to look more closely at these studies to understand why they reach different conclusions. The mixed results, and the different approaches behind them, also support our position that it is unwise to use any one study to gauge the success of teachers on individual student achievement.

Research by Charles T. Clotfelter, Helen F. Ladd, and Jacob L. Vigdor examined the effects of National Board Certified Teachers (NBCTs) on student achievement in North Carolina. While NBPTS did not directly commission this research, we provided data used to study the research questions. In the study, students of NBCTs generally outperformed those of other teachers at statistically significant levels when a wide range of other factors are held constant. The authors of this study noted that their findings were consistent with those of another study by Dan Goldhaber and Emily Anthony (2006), which found similarly positive findings for North Carolina NBCTs.

Conversely, Tim Harris and Doug Sass, in a study commissioned by NBPTS, found the students of NBCTs in Florida only performed better than other teachers in some grades and subject areas, while in other cases they performed worse than other teachers. These findings are counterintuitive, since a great deal of other research has found that NBCTs not only elicit better test scores from their students, but also foster deeper learning outcomes.

Some of the differences in the two studies are not surprising, since each covers different states, different demographics in types of teachers, different incentive systems for becoming certified, and different sets of student standards. In addition to these differences, methodological factors, the types of tests used to measure student achievement, and the policy environments in which teachers work could also explain differences between the studies.

Both teams of researchers applied value-added methodologies. This methodology is "quasi-experimental," meaning that it uses complex statistical techniques to simulate a controlled environment. However, value-added analysis faces the same basic challenge as all statistical measures of student achievement: it is very difficult to distinguish the impact that a single teacher is having on his or her students' achievement—especially student achievement as measured by standardized tests. Hundreds of factors weigh on a student's performance on a particular test. Demographics, school characteristics, even whether the child had breakfast on test day have all been shown to have noticeable effects on test scores.

Harris and Sass, for example, used two different tests to measure teacher effectiveness. Results varied depending on whether the test used was the criterion-referenced Sunshine State Standards test or the norm-referenced Stanford 9 assessment. Not surprisingly, NBCTs performed better on the state assessment, which is aligned to curriculum standards and may be a better assessment of teaching abilities. Here, teachers who are NBCTs at any point in their teaching career demonstrated significantly better performance than other teachers. In contrast, the Stanford 9, which is not aligned with state standards, might not reflect teachers' direct impact on student learning.

Ladd, Clotfelter, and Vigdor's study used a test that has been administered for more than 10 years as part of North Carolina's accountability system and is tightly linked to the state’s Standard Course of Study. As in Florida, NBCTs performed better when evaluated using a test linked to the material that the state wants students to know.

Given these differences in findings between—and within—these two studies, and among the scores of other studies on student achievement and National Board Certification, we are not prepared to take the results of any one study as the final word. Studies of student achievement based on test scores are only one aspect of the research on National Board Certification.
Other studies have documented the impact of National Board Certification on teacher retention, mentoring, professional development, and depth of student learning. The National Board applauds these researchers for their worthy contribution to the growing body of knowledge about National Board Certification.

Mary Dilworth is Vice President of Higher Education and Research for the National Board for Professional Teaching Standards.

References

Clotfelter, Charles T., Helen F. Ladd and Jacob L. Vigdor. 2007. "How and Why Do Teacher Credentials Matter for Student Achievement?" CALDER working paper no. 2.

Goldhaber, Daniel and Emily Anthony. 2006. "Can Teacher Quality Be Effectively Assessed? National Board Certification as a Signal of Effective Teaching." MS 9832.

Harris, Douglas N. and Tim R. Sass. 2007. "The Effects of NBPTS-Certified Teachers on Student Achievement." CALDER working paper no. 4.

Read additional commentary by Daniel F. McCaffrey and Helen F. Ladd & Tim R. Sass.