Jacki and I attended a panel at the American Enterprise Institute yesterday on the Jay Greene report I blogged about earlier this week. Marcus Winters, co-author of the report, was presenting along with Jane Hannaway from the Urban Institute and David Figlio from the University of Florida. It was a very collegial affair, even though the report attracted substantial criticism. Figlio lauded the researchers for the positive trend they found in science proficiency. But he added that the report didn’t prove that the increase in science test scores was causally linked to the increase in math and reading scores. He characterized as “modest” the strength of the authors’ correlation of science scores with those in reading and math. That correlation is the key evidence the report puts forward to assert that testing in high-stakes subjects doesn’t hurt students’ performance in low-stakes subjects in the state of Florida.
When the Q and A began, I suggested that calling science a “low-stakes” subject in Florida was misleading (which was sort of a position the panel had coalesced to in the course of its deliberations). After all, a school’s overall performance on the FCAT (Florida’s Comprehensive Achievement Test), which tests students’ reading, math, and science proficiency is a key factor in determining what grade the school receives under the state’s A+ assessment system. Science scores started counting toward the A+ determination just this year but—as Hannaway pointed out—”the writing was on the wall” for some time with regard to science scores carrying real stakes in Florida. And Figlio, who lives in the state, noted that students’ performance on the science part of the FCAT has made front page news since the test began. Marcus didn’t disagree with my assertion that science was a high-stakes subject in Florida, but suggested that reading and math were “higher” stakes.
A little later Amber Winkler, research director for the Thomas B. Fordham Institute, questioned the validity of using prior reading and math scores as a proxy for prior science scores. Basically, in order to gauge whether or not an increase in science scores had occurred, Greene and Winters constructed a faux science “score” using 2001-2002 math and reading results so that they would have a measure to compare with 2002-2003 science scores. Asked by Winkler how commonplace it was in studies such as these to construct prior scores in one subject from those in another, Winters responded that he “wasn’t sure as they’d never had that problem before.” Winkler told me she wondered why the researchers hadn’t just waited until they could compare science scores to science scores.
NOTE: When I wrote about this report earlier this week, I criticized Greene for asserting in a blog he’d written promoting the report that its findings would “debunk” and prove “unfounded” statements made by CC co-chair Diane Ravitch and Fordham President Checker Finn. He’s now complained (see comment below) that he never used the words quoted above. I of course never said he used those particular words and hence did not put quotes around them when I used them. Any reader of his original posting (which he’s copied into his comment below) can see that that was precisely the inference he was trying to make when he quoted Diane and Checker at length on the narrowing issue and then asked people to read his report “to find out whether these concerns are supported by the empirical evidence from Florida.” Well, we’ve read the report and our concerns are far from allayed. Enuf said.