KATU Investigation: Are school test scores too good to be true?

KATU Investigation: Are school test scores too good to be true? »Play Video
FILE - Earlier this year, students demonstrated against the state's standardized tests. The walkout was part of a nationwide movement.

PORTLAND, Ore. – Oregon schools are under more pressure than ever to improve student test results and tie them to teacher performance.
With the state at risk of losing its federal education waiver, control over classrooms and funding is at stake, careers too.  But On Your Side Investigator Anna Canzano has uncovered some local schools' scores that seem too good to be true.

Her findings, the result of a two-month investigation of school and district scoring results, now has the attention of the Oregon Department of Education, which has already disciplined four teachers for cheating on standardized tests over the last two years.
"The commission takes every case very seriously," says Milt Dennison, superintendent of the Clackamas Education Service District and a member of the Oregon Teacher Standard and Practices Commission.

The commission keeps Oregon teachers accountable, and punishes them when they cheat.

"The typical explanation was, 'I felt like I had to do something to save my job,'" Dennison says. "If a teacher is on a plan of improvement, and it's focused on student improvement in terms of test scores, it's pretty high pressure."

Cheating Teachers

An Oregon teacher who was caught cheating agreed to talk to KATU – on the condition we not reveal their identity. We asked if this teacher felt their career was on the line: "Absolutely, they're looking at those numbers and saying, 'Why aren't your kids getting it, when this other teacher gets it. We need to put you on a plan of assistance.'"

The teacher said the scrutiny and pressure from above makes it easy to put scores first, education second.

"As a teacher the most important thing for me is that my students can do as well as they can on the test, one to show that they can do it, but two, also, so that I am doing my job," the teacher said.

Recent investigations by the Commission include examples of teachers supplying students with inappropriate aid materials, and in one case a teacher went so far as to tell students "no, wrong answer" when they entered incorrect information.

Protesting the Test

How did things get this way? How did they get to the point where students are protesting - as they did earlier this year - refusing to take the state's standardized tests? The walkout was part of a nationwide movement, characterized by groups such as The National Center for Fair & Open Testing.

And the scores – and the role they play in measuring school achievement – may become even more controversial. Oregon is rolling out a new school report card system this academic year.  It not only shows parents how the test scores at their kids' school stacks up against others in the state, it shows how those scores compare to schools similar to theirs - in terms of demographics. At a glance, you can see if your kids' school is doing better or worse than others with similar numbers of poor children, kids with disabilities and those for whom English is a second language.

U.S. Education officials have also labeled Oregon – along with Washington –as "high-risk" of losing their federal education waiver, forcing a return to strict No Child Left Behind standards, because they haven't incorporated students test scores as part of teacher performance evaluations. 

Unusual Test Scores

But are those scores a true measure of teacher competency? Does a good score mean you're a good teacher? Or a bad score mean the opposite?  The On Your Side Investigators analyzed four years’ worth of standardized test scores at schools in the Portland metro area.

We found a dozen cases of schools posting unusual score trends in back-to-back years of testing: instances where a school that had scored near the bottom of the state on a particular test one year, suddenly became one of the best performers on the next year's test. Or vice versa: a school that had been ranked among Oregon's best in a grade-category, suddenly dropped to the bottom of the list on the next year's exam.

For example, third-graders at Ardenwald Elementary in Milwaukie were among the worst performers in math in 2010 - only 50 percent of them passed the standardized test.  But in 2011, 80 percent of the third-graders there met or exceeded expectations – a 30 percentile point swing – while performance in the district as a whole on that test declined.

In another case, eighth-graders at Aloha-Huber Park went from being on the worst list in reading with only 60 percent of them passing in 2010, to 89 percent of them passing the next year, a swing of 29 percent.

We noted sudden drops as well. At Trillium Charter School in 2010, 85 percent of eighth-graders passed the reading test. In 2011 that number dropped to 63 percent.  Alliance Charter Academy also experienced a double dip decline: its eighth-graders were among the best math test takers in Oregon in 2010 with 78 percent meeting or exceeding expectations. In 2011, the eighth-graders were among the worst performers, with only 59 percent passing.

(Search for your school's test scores using The Oregon Department of Education Data Explorer)

A National Problem

The state of Oregon does not currently have a system in place that automatically flags questionable test scores for further inquiry. But these cases reflect similar findings brought to light by an Atlanta Journal Constitution report. The newspaper's analysis flagged suspicious test results in nearly 200 districts nationwide, including Hillsboro, North Clackamas, and Portland in Oregon.

(Search The Atlanta Journal Constitution Database to see flagged results in your district)

"As a superintendent I can't recall a school ever having that kind of significant shifts," says Rob Saxton, the Deputy Superintendent of Public Instruction for the state of Oregon, appointed by Gov. John Kitzhaber.
In all these cases Saxton was quick to note that a sudden change in scoring performance does not prove cheating occurred. Several factors might explain the wildly different results: district boundaries being redrawn, new curriculum (for example, standards were overhauled in 2010-11), or the inclusion of new student populations. But particularly large shifts can spell trouble and Saxton says our findings are worth a deeper examination.

"It may well be our assessment (that) folks have already looked at that and asked some questions, but I'd be happy to have them," he said.

We shared what we found with the Department of Education, but a spokeswoman later told us the data staff there was unable to do any in-depth analysis of our findings until after the new report card system is in place.

We'll keep checking back with them. Meantime, we’ve assemble the most recent state test scores for some of the largest school districts around Portland:

  • PPS (Excel spreadsheet)

You can search for other the test results for other Oregon schools here: http://www.ode.state.or.us/search/page/?=3764

And standardized test score results in Washington are available through this searchable database:  http://www.greatschools.org/test/landing.page?state=WA&tid=149