Statistical analysis is one of the best toolboxes we have for finding out what works and what doesn’t. It seems reasonable to use statistics to determine the best teachers and the best schools.

After all, parents want the best teachers for their children, and citizens want the best schools for their tax dollars.

But a lot of the art of statistics is knowing what questions to ask. For example, suppose you want to know what cars are the cheapest to operate. If you ask the wrong question – do red cars get higher gas mileage than blue cars? – the results probably won’t be very helpful.

Educational assessment is in a similar swamp. Assessment has become almost a code word for conducting periodic standardized exams and using them to determine teachers’ pay and continued employment.

Many politicians and pundits assume the way to fix education is to get rid of incompetent teachers. A principal should hire teachers and check up on them by conducting frequent standardized exams. The entire system can be administered by someone who doesn’t know anything about teaching at all. This should set off alarm bells in anyone familiar with business controversies over the last few decades.

There’s an elephant in the room neither side talks about. Classroom teaching is unnatural (compared to the more traditional tutorial, the approach for homeschooling, piano lessons, and Ph.D. preparation.) Classroom teaching is a lot cheaper than the tutorial, but making classrooms work is an ancient problem. You may know all the material and still not teach much if you don’t know how to teach, and that’s why school districts insist that teachers learn how to teach.

In fact, fixing or finessing the classroom is a major focus of educational psychology, and there’s a lot of research into new teaching techniques. The same teacher can get radically different results by using different teaching techniques.

This suggests that while we are learning what works and what doesn’t, we should be assessing teaching techniques, not the teachers trying them out. Turning to a success story, medicine made enormous progress during the last two centuries not by assessing doctors, but by assessing techniques and therapies. Does bleeding cure a fever? Does penicillin cure pneumonia? Does laetrile cure cancer? Does aspirin reduce inflammation? Doctors – not politicians and pundits, but doctors and allied scientists – used statistical analysis to determine what worked and what didn’t. And now almost all children live to adulthood, pneumonia is largely segregated to the very old, and lack of a cure for an infectious disease is a scandal and not a fact of life.

Focusing on teachers rather than the techniques they use has poisoned the debate and made assessment itself the issue. Recent studies suggest that teacher reward-punishment systems based on frequent assessments are not very successful in raising scores. Many solutions that have little to do with the debate – like abolishing unions or privatizing schools – have only convinced many educators that this isn’t about education at all, which only undermines the credibility of assessment as a tool.

If we are going to make American classrooms work, we must focus on what teachers do, and how to improve what they do. And that is what the colleges of education are doing.

Gregory McColm is an associate professor of mathematics & statistics at USF. He may be reached at [email protected].