“Shock plunge in kid test scores,” screamed the New York Post in late July 2010. “New York City test scores plummet year after officials makes statewide exams tougher,” was the Daily News headline, while the sober New York Times went with, “Standards Raised, More Students Fail Tests.” At the time I was working for the United Federation of Teachers, the labor union that represents New York City’s public school teachers, so I remember the story well. In a way, it’s a simple cause-and-effect: after New York State education officials raised the standards for proficiency in math more kids failed the math test. But thanks to NYC’s commitment to open data we can dig into the test data and hopefully learn a little more about what happened in 2010.

Above we have the New York State math test results for New York City students for the years 2006 through 2011, grouped into “proficient,” meaning at or above grade level, or “not proficient,” which can be glossed as “failing.” Students are actually assigned scores from 1-4, where Levels 1-2 are not at grade level and 3-4 are at or above grade level. See this sample score report for more details on math test scoring. We can clearly see the sharp drop in the number of students graded proficient in math from 2009 to 2010, and the concomitant rise in failing grades.

If we look at the percentage of students who failed the math test, sometimes called the failure rate, by borough — this time using percentages to control for student population — we see that the Bronx consistently has the highest failure rates in the city. We can also look at this data on a map.

Click the right arrow next to the Year dropdown, or use the slider, to see the change in failure rate from year to year represented as color intensity. This map was created in Tableau Public 8.3 by using the NYC Open Data results by borough, assigning the borough value the Geographic Role of County, and mapping the percentage of students by borough who failed the test. But we already know that Bronx consistently has high failure rates. Was the spike in 2010 any more pronounced in that borough?

Here we see the same borough data now computed as a difference in failure percentage from year to year. Indeed, the effect of the raised proficiency standards was more pronounced in the Bronx than in the other boroughs. To generate this graph, we used the Quick Table Calculation feature in Tableau.

Let’s look at a different set of the same data, this time parsed by gender. We often hear that girls have stronger math skills than boys, and this is certainly borne out in NYC; boys consistently fail the math test at a higher rate than girls. But the tougher standards seemed to affect both genders about the same, as we see below in the failure rate difference in 2010 from the previous year.

Which schools fell furthest in the “shock plunge” in test scores in 2010? We took the test data parsed by school and performed some transformations and calculations in Open Refine and Open Office. We removed all entries for District 75, since those are NYC’s special education schools. In Open Office we filtered the data to show only failing students in the years 2009 and 2010, then computed the percent difference between the two years. In Tableau we used a “Top 10” filter to display only those 10 schools with the greatest percent increase in failures from 2009-2010, sorted by percent increase in failures. NYC school codes are formed by taking the two-digit district number, followed by a borough letter, followed by a three-digit school number. Just looking at the borough letter, we see that five out of the 10 individual schools that experienced the biggest increase in failing grades on the math test from 2009-2010 are in the Bronx (X). Three Brooklyn (K) schools and two Queens schools round out the list.

New York State math test scores dropped dramatically among NYC students in 2010. Students in the Bronx, already struggling, were affected more than their peers, while girls and boys citywide were affected about equally. As states increasingly rely on student standardized testing data to evaluate students as well as their teachers and schools, we can expect to see more and more data-driven education stories in the news, and continue to debate the merits of this approach to public education. Thanks to open data initiatives like New York City’s, citizens can actually check the numbers and analyze the data themselves, and possibly bring new stories to light.