Failing Fordham and the Journal Sentinel

(Bumped from the misc. list below.)

The state's largest daily paper followed up yesterday's love note to the MPS superintendent--literally re-imagining him as the hero in a spy movie--with another in its series on how the Wisconsin DPI is failing miserably. I found this part the most interesting:

In the Fordham report, Wisconsin's academic standards in five areas were given poor grades: A "C" in English, a "D" in math and "F's" in U.S. history, world history and science. The overall grade of "D-" was down from "C-" when Fordham did a similar report in 2000--although [Deputy State Superintendent Tony] Evers noted that Wisconsin's standards have not themselves changed over that period.

That's my emphasis there on the sentence that should have sent up red flags all over the newsroom and editors' desks. It clearly raises questions about the Fordham's study's methodology. I'm not sure what they are specifically; Xoff, on the other hand, has a pretty good idea. He cites Gerald Bracey's analysis of the 2000 study:

The problem with the evaluations is a simple one: the states’ rankings for quality of standards are inverse to their performance on the National Assessment of Educational Progress (NAEP) and on the Third International Mathematics and Science Study (TIMSS). That is, the same states that have done the best job in the eyes of the Fordham report’s authors in implementing high standards have shown the poorest performance on widely accepted national tests for student achievement, and vice versa.

Seth does a bit of an update of Bracey's work:

Out of the fifty states plus the District of Columbia, there were 9 that received a grade of B- or higher in the Fordham study. [. . .] Wisconsin was given a D- in the Fordham study. Yet, when we take a look at the NAEP reports [. . .] on student performance for 2005, Wisconsin ranks higher than all of the nine states [with a B- or higher] except Massachusetts.

Seth's got the links to prove it, so click through if you doubt me.

I commented on this series about the DPI's supposed failure back in June, doing some of this same kind of research to show that the doom-and-gloom proclamations from out-of-state "think tanks" don't match reality here on the ground. This should not be new news to anyone. (Note that the other stories in paper's series are conveniently linked in the sidebar of the current story.)

Alan Borsuk has access to all of the same data--probably a lot more--that Xoff, Seth, and I do; he should have been able to put the Fordham and other studies into much better context than he did. Instead, he protests that his story is balanced (my emphasis again):

But the three education advocates interviewed Monday [. . .] come from varying places on the political spectrum [. . .] Fordham is politically conservative, but the Education Trust and Education Sector are each harder to peg politically. However, both generally favor the approach to raising student achievement that underlies No Child Left Behind, which became law in 2002.

Yes! We talked to three people on the same side of the issue! But they were different people! That's balance!

It's crap, is what it is.

I'm not suggesting that there is no room for improvement, either in the Milwaukee schools or state-wide (we need to recapture the number one ACT spot back from Minnesota!). If I thought we were as good as we were going to get I'd quit teaching and move on to something else that sorely needs improvement (like journalism?).

Instead, I'm suggesting that biased or unreliable reports about Wisconsin's performance don't deserve front-page, unrebutted status. In June, I noted that the week before the study saying how badly Wisconsin was doing--another front-page story--the paper had buried on page 8B the story of our state's fourth-place score on the NAEP science exam. Some of us think that our successes are front-page news, not these questionable-on-their-faces think-tank "studies."

Public School Teachers and Private Schools

From the mailbag:

I am a retired Madison teacher and formerly a 25 year member of the MTI negotiating team.  I have been trying to find out some information for some time.  Maybe you can help me.  On public radio this week, there was an advocate of choice schools reciting some statistics.  He said that a study by MTEA found that 29% of Milwaukee teachers send their kids to voucher school.  Do you know that the actual numbers are?  I have heard various numbers like that ranging from 14 to 50%.

The program he heard was the Joy Cardin show with George Mitchell, the one I called into and wrote about Tuesday. And it's true--Mitchell said that "29% of MPS teachers with children send their children to private schools," calling it a long-term tradition.

But, because I am attentive to my readers, I took this fellow up on his question, and did some looking into it. As I found in my own conversation with Mitchell, it turns out he was not telling the whole story.

The 29% figure is real, if debatable. The number comes from a study by pro-voucher researcher Denis Doyle, published through the Fordham Foundation. The figure is based on Doyle's analysis of 2000 census data; just to give you an idea of where he's coming from, Doyle's analysis of 1980 census data was published by the conservative American Enterprise Institute, and his work on the 1990 data was funded by the pro-voucher Center for Education Reform.

Doyle's analysis had two components, studying the difference between public school teachers and the general public nationwide, and teasing out the data for America's 50 largest cities, including Milwaukee. Doyle found that in the nation's 50 biggest cities, 21.5% of public school teachers send children to private schools, about 4 percentage points more often than parents overall. Milwaukee's 29.4% of teachers sending kids to private schools was six percentage points higher than Milwaukee parents in general. Milwaukee ranked 20th of 50 cities in terms of difference between parents and population, and the study made no distinction between private schools and schools participating in the Milwaukee Parental Choice Program.

But there are some other things going on here. Gerald Bracey, a crusader on the side of public schools and teachers, gave Doyle and Fordham one of his coveted Rotten Apple awards in 2004--the “If at first you don’t succeed: Fudge, Fudge Again” award, in fact:

When Doyle finally got around to showing the data, the 1990 data revealed that 12.1% of public school teachers use private schools compared to 13.1% of the general public. In his analysis of the 2000 census, the figure is down to 10.6%, and it’s only 7.9% if you don’t count teachers who send their kids to both public and private schools. For the general public it’s 12.1%, 9.4% dropping out parents who use both. [. . .]

Doyle calls public school teachers education “connoisseurs.” One would expect private school teachers to exhibit the same level of expertise, yet fewer than a third of private school teachers pack their children off to private schools, 29.5% in 2000, down from 32.7% in 1990. In fact, although 54.1% of private school teachers in 2000 had family incomes of more than $84,000 a year, only 31.1% of this group had children in private schools.

Of course, Bracey can't break down the private-school number by city, so we cannot make the direct comparison between Milwaukee's public and private school teachers, specifically. But it is important to note that the 29% number would probably be lower if you take out parents who send their children to both public and private schools, as Bracey found for the study overall. Plus, when George Mitchell said that it is 29% of Milwaukee Public Schools teachers, this is not what the report says; it refers only to public school teachers who live in the city, who may not all teach for MPS. More importantly, the number for Milwaukee is not significantly different than the number for private school teachers nationwide. If the public schools are good enough for 70% of private school teachers, why can't they be good enough for 70% of public school teachers without its being an issue?

But there's more to it than that: If you go back to discussions (see here and here, for example) about the creation of the Milwaukee Parental Choice voucher program in 1990, two proponents, Annette Polly Williams and Howard Fuller, were claiming--based on what, I admit I don't know--that fully half of MPS teachers were sending their children to private schools. These claims were a big part of the push behind the program's creation. If that 50% number is true--and, again, I'm just taking their word for it--that means that between 1990 and 2000, there was a 40% drop in the number of MPS teachers sending their children to private schools. I have no idea what the number might be now, but the trend seems to be downward.

Finally, and, cautiously, given my discussion of sampling the other day, a note on Doyle's methodology. It's actually worth reading the "Methodology" section in the report, since it talks about the census data being used. The Census Bureau releases 5% of the data for an area--like the city of Milwaukee--for study; the authors say that the incidence of teachers with children in the population is 2% of households. In other words, all the data on public school teachers in this study comes from just 0.1% of households in Milwaukee.