A publication by Kansas Association of School Boards (KASB) touts “Kansas student proficiencies in reading and math have increased 40 percent over the past decade and exceed 80 percent at every level.”
Kansas public school officials also boast that Kansas does well compared to other states on the National Assessment of Educational Progress (NAEP), called the “nation’s report card.”
But the confidence of Kansas public school leaders has been shaken a bit with the recent release of test results for the recent school year. Results are down. The Kansas Commissioner of Education will form a task force to investigate. Already excuses are being proffered, with some officials saying they always knew that scores would reach a plateau.
(Later it was discovered that the reported decline was due to a mistake in tabulating results, and that overall, test scores were mostly unchanged.)
While Kansas Education Commissioner Diane DeBacker is conducting an investigation, here’s two questions she ought to ask.
One: Why have scores on Kansas assessments risen, while scores on the NAEP haven’t? Some know the answer, but it would be refreshing to hear the bureaucracy admit it: The Kansas State Department of Education has lowered standards on the tests it controls. When a state board of education member tries to ask questions about this, he isn’t allowed to have his questions answered.
Second question: Let’s understand why Kansas NAEP scores are high, relative to other states. Here’s a table comparing Kansas with Texas (shaded cells indicate the state with the highest score):
Notice that for all students, Kansas has the highest scores, except for one tie. But when we look at subgroups, all the sudden the picture is different: Texas has the best scores in all cases, except for two ties.
How can this be? The answer is Simpson’s Paradox. A Wall Street Journal article explains: “Put simply, Simpson’s Paradox reveals that aggregated data can appear to reverse important trends in the numbers being combined.” (When Combined Data Reveal the Flaw of Averages: In a Statistical Anomaly Dubbed Simpson’s Paradox, Aggregated Numbers Obscure Trends in Job Market, Medicine and Baseball.)
In this case, the confounding factor is that the two states differ greatly in the proportion of white students. In Kansas, 69 percent of students are white. In Texas it’s 32 percent.
Texas, by the way, spends much less per student, and has a higher pupil/teacher ratio.
Kansas liberals and those who support more spending on schools say we don’t want to be like Texas. I wonder if they are aware of Simpson’s Paradox.
Will DeBacker and her task force will be interested in the answers to these questions? Kansans should be, especially as we wait the verdict on the school finance trial.
Leave a Reply