Friday, September 30, 2011

Incompetence

Officials stop paying for test cheating audit

High-stakes education testing in California is operating on the honor system because two years ago state officials eliminated the system used to catch cheaters.

The routine checking of tests for erasures or other anomalies was eliminated in California in September 2009, when the the budget for testing was slashed by $17 million. The cost of security audits and running hand-bubbled tests through a machine to check for excessive erasures and suspicious marks: $105,000.

That means California isn't likely to catch major episodes of cheating like those that rocked city schools in Atlanta, Washington, D.C., and Baltimore, and school districts in Pennsylvania and New Jersey in recent years.

In those cases, teachers have been accused of changing test answers or prompting students to change them.

Without computerized assistance, California officials rely on school districts to self-report, submitting testing irregularity reports for everything from inappropriate test preparation to cheating.

Some question how effective that is.

"The district has a conflict of interest," said Robert Schaeffer of the National Center for Fair and Open Testing. "They want scores to be high and they want to look like their staff is honest and aboveboard."

High test scores mean more than just accolades for schools; they can keep schools from being labeled failures or, in the worse cases, being taken over by the state.

Meanwhile, the state Department of Education doesn't encourage whistle-blowers to call, officials said. If someone calls the agency to report cheating by teachers or administrators they are told to call the school district involved, said John Boivin of the California Department of Education's Standardized Testing and Accountability office.

But can the state rely on school districts to police themselves?

The Los Angeles Unified School District is an example of a district that policed itself. Earlier this month California education officials tossed out the test scores of two Los Angeles-area schools after several teachers were found to have doctored test results.

As a result of the alleged test fixing, Los Angeles Unified conducted investigations and notified the state Department of Education, which struck each school from this year's Academic Performance Index, which is used to measure how well a school is doing.

In the high-profile cases of cheating in other states, the cheaters were caught by erasure marks on the tests.

The equipment used in erasure analysis is generally programmed with a formula that allows for more erasures for second-grade math, for example, than for ninth-grade English. If there is a large percentage of stray marks or erasures in any one school or district a notice is sent to the district.

Fewer than two dozen states analyze erasures, according to Schaeffer. "But, it's changing very rapidly because of all the news coverage of misbehavior."

California's Boivin said that there has been some discussion at the State Department of Education about reinstating erasure analysis after the recent reports of cheating, but that nothing official has been done.

The question of re-funding the erasure analysis program has not come before the state Board of Education, said Mike Kirst, its president. "There has been a problem, but there is no evidence there is a huge problem we don't know about," Kirst said.

Despite the pressure to do well, reports of irregularity from the state's schools have increased in the last three years. Between 2009 and 2011, the reports of testing irregularities grew from 35 to 121.

If a school reports even a minor irregularity it is no longer eligible for state awards, like the Distinguished School Award, Boivin said. If the report shows that more than 5 percent of the students tested at the school could have been affected by the irregularity, the school's API is tossed out for the year.

Florin Elementary in the Elk Grove Unified School District lost its API this year after a math table had been left partially uncovered on a wall. A student brought it to the teacher's attention, said Elizabeth Graswich, district spokeswoman. Because the 22 students in the class made up more than 5 percent of the students tested at the school, the API was invalidated.

Many of the irregularities reported are these sorts of inadvertent mistakes, but cheating is still a concern.

Schaeffer said there are ways to manipulate scores that don't involve erasing wrong answers and replacing them with right ones. He said schools can teach to the test, limit curriculum to only that taught on the test, exclude students likely to have low scores and hold kids back a grade in critical years and promote them after the test.

"That's why we can't rely on test scores," he said. "There are too many ways to game them."


No comments: