The agency responsible for overseeing and analyzing public education data in California removed “bad” Advanced Placement test results from its website last month, acknowledging it had published the wrong scores from 2016.
inewsource found the discrepancies in the California Department of Education’s 2016 AP test scores, and while waiting for the department to respond, found inconsistencies in prior years, as well. State officials said two data analysts each had grabbed the wrong file for 2016 and posted it. That file contained 350,000 more tests than had actually been taken, throwing off pass rates.
Officials dismissed concerns about prior years’ data, saying it can be off by hundreds to thousands of tests every year prior. They described the numbers as “pretty close and pretty consistent.”
The erroneous AP data was on the state website for more than two months. The state department removed the scores on May 29. They also removed the 2015-2016 SAT and ACT scores “as a precaution,” although officials said there was no indication that the data was incorrect.
School officials and policymakers rely on data posted to the California Department of Education’s website to compare school performance. Parents say test scores factor into their decisions about where to send their children to school.
The San Diego Unified School District independently publishes AP scores by school, allowing inewsource to understand how the discrepancies affected local scores.
According to its report, Point Loma High administered the most AP exams in the district in 2016. Its students took 1,113 AP tests, with 616 resulting in a passing score. But state data showed 2,203 exams taken with 1,290 passing.
Steve Green, who uses state data to track school progress and publish annual reports for the San Diego County Office of Education, was taken aback to hear about accuracy issues.
“Wow,” Green said. “I wasn't aware. When we look at those numbers, we presume they’re accurate."
The California education department publishes annual AP, SAT and ACT results by school as well as dozens of other files that local educators use to run reports. inewsource found the discrepancies while comparing state AP data to results published by the College Board, the national organization that administers the tests.
Datasets published by the state also filter into its Dataquest application and new California School Dashboard, which allow the public to run quick searches such as school demographics, test scores and college readiness rates.
Several third party websites, such as greatschools.org, use the data to rank neighborhood schools – a tool for parents like Karen Calderon who said she consulted the data to decide a school track for her son in the San Diego Unified School District.
“It was helpful to see the student body makeup, test scores, etc.,” Calderon said, although much of her decision was based on touring schools and speaking with other parents.
San Diego Unified reported students at her son’s future high school, University City, took 887 AP tests in 2016, with 510 resulting in a passing score. But state data showed 1,727 exams taken with 1,075 passing.
When errors in data are corrected by the state, those inside the education industry – like Green – receive a notification. The public does not.
“All of the schools will be receiving a notification about the data and the correction, which is kind of a standard operating procedure,” said Keric Ashley, deputy superintendent of the state department’s district, school and innovation branch.
“How would we notify the public other than a note on our website indicating that this data has been updated?” he said. “I'm not sure what else we could do.”
A majority of the downloadable test files on the California Department of Education’s website are accompanied by the word “revised,” and a date, meaning the file has been updated at least once. But details are not included in a file explaining what was changed or why.
“There will be times when districts have made an error getting data to us and other times, like this time around, which was clearly the fault of our programmers,” Ashley said.
Jenny Singh, an administrator with the department’s data reporting division, told inewsource June 2 that a corrected AP file will be uploaded in the next two weeks.
She said other problems can occur when processing AP data. For example, when the College Board shares results with the department, it must match a unique code to each school’s CDS code. Singh said school codes can sometimes be inaccurate, meaning those results get left out.
But it’s often up to the districts to notice.
There was also a recent error found in Smarter Balanced test results – an end-of-year assessment that measures students college and career readiness. The department confirmed to inewsource that the mistake was corrected.
