Every March and April, students in school districts across Pennsylvania sharpen their pencils and prepare to take on the Pennsylvania System of School Assessment tests.
The state-mandated exams, administered in grades three through eight and grade 11, test students in reading and math (and, depending on grade level, science and writing). According to the state Department of Education's website, the tests are designed to measure how many students are capable of knowing "what a student should know and be able to do at varying grade levels."
But in school districts like Pittsburgh and Philadelphia especially, it's common for students to flunk the exams … and then go on to graduate the next year anyway.
Take Westinghouse High School. In 2009, only 25 percent of the Homewood school's 11th-graders scored at least "proficient" -- meaning at or above average -- in reading. Only 13 percent of Westinghouse juniors scored at least proficient in math. And yet when that class moved on to 12th grade the following year, Westinghouse graduated 88 percent of its students.
Such disparities have some education experts scratching their heads. While test scores are just one of many measures of student performance, some worry that the large gap between low test scores and high graduation rates suggests districts are handing students diplomas without the education to back it up.
"It's so incongruous," says Johnson Martin, an educational consultant and former Pittsburgh Public Schools administrator. "Kids are graduating without the appropriate education."
Not every Pittsburgh school suffers from the disparity. CAPA, the Downtown performing-arts school, graduated a perfect 100 percent of its senior class in 2010. The year before, 95 percent of CAPA 11th-graders scored at least proficient in reading, while 89 percent of them scored the same in math.
Nor are gaps between test results and graduation rates unique to Pittsburgh. Statewide, the graduation rate last year was 91 percent. Yet when the class of 2010 took its PSSA exams the year before, only 65 percent scored at least proficient in reading. Fifty-five percent scored the same in math.
Even charter schools -- publicly funded alternative schools -- aren't immune to the problem. Career Connections Charter High School, in Lawrenceville, had an 88 percent graduation rate in 2010. But in 2009, only 36 percent of its 11th-grade students scored at least proficient in reading, and just 32 percent scored the same in math.
Still, the disparity is most pronounced in big-city public schools. Pittsburgh posted an 89 percent graduation rate in 2010, despite the fact that only half of its 11th-graders scored at least proficient in reading the year before, and just 43 percent scored the same in math. (Philadelphia's gap is almost identical to Pittsburgh's.) At the North Side's Oliver High, just 30 percent of the school's 11th-graders scored at least proficient in reading in 2009, while 21 percent scored the same in math. The next year, however, Oliver had a 74 percent graduation rate.
That kind of gap "is very troubling, very frustrating," says Sherry Hazuda, president of the Pittsburgh school board. "It suggests we're not doing a good enough job for our kids. They're sure not prepared to go to college."
"I'm alarmed at the disparity," adds Marge McMackin, an educational consultant and former member of the Pennsylvania Department of Education. "The number of students not prepared for college or careers who are currently graduating … is astounding and unconscionable."
So what is causing the disparity? Are the state exams too hard? Are districts' academic standards too low? Or are the numbers that measure student performance skewed?
"It's a good question," says Carey Harris, executive director of A+ Schools, a local education watchdog group. She doubts, however, that the tests are too demanding. "The test isn't flawed. It's just that kids aren't doing well on them."
Indeed, test results seem like an objective measure of student ability. But some educators say they must be taken with a grain of salt.
For one thing, students have little incentive to take the tests seriously. While scores can be used to target students who need additional help, or to help schools improve their curriculum, it's not as if anyone's grade depends on them.
Tim Eller, a spokesperson for the state Department of Education, says that the department "is concerned about the possible disparity" between test scores and graduation rates. But in an e-mail, he notes, "In nearly all instances, the PSSA is not a high-stakes assessment for students."
And even good students sometimes don't test well on standardized exams.
Jerry Longo, an education professor at the University of Pittsburgh, agrees that it's important to investigate why graduation rates and test scores diverge so sharply. But he cautions against putting too much stock in exams.
"State tests only deal with reading and math, and obviously there's more to a high school curriculum than that," says Longo, a former school superintendent. "A student might be a brilliant artist or musician, but that test isn't measuring any of that."
Some counter that grades, unlike test scores, are easy to doctor.
"You can always inflate grades," says Martin. "But you can't [inflate] state tests."
Martin says Pittsburgh school officials regularly tout their high academic standards. But, he says, if they are so high, the district's graduation rate should more closely resemble its proficiency rate on state test scores. The gap suggests "kids are being passed that don't deserve to be passed."
School districts do have some incentive for keeping graduation rates high: Such rates determine whether a district meets federal standards outlined in the No Child Left Behind Act, passed by former President George W. Bush in 2001. Student scores on the PSSA are also a factor.
Schools "are not going to keep 75 to 100 kids in high school," Martin says. "They're just going to pass them on."
"There's no way a district is going to fail over half of its kids," adds McMackin. "So they're going to find a way out of that dilemma."
Pittsburgh school officials dispute that the district is passing students who haven't earned their diploma. Lisa Fischetti, the district's chief of staff, points to the success of the Pittsburgh Promise, a scholarship program that provides students up to $40,000 in college education funding if they graduate high school with at least a 2.5 GPA. Since it launched in 2008, roughly 3,200 Pittsburgh graduates have received tuition grants from the program.
"What you need to look at is how many students are Promise-ready," says Fischetti.
Pittsburgh school officials say there's another reason for the apparent gap between test results and graduation rates: The way the state calculates graduation rates, they say, overstates the ease with which students get diplomas.
It's easy to measure the number of diplomas handed out each year, of course. What's not so simple is counting students who didn't get one. Between the time 11th graders take their PSSAs and their graduation a year later, for example, some of their classmates will transfer to other schools, have to repeat their junior year or drop out entirely. Other students, meanwhile, will transfer in.
School officials say that the state's current calculation of graduation rates, known as the "leaver rate," doesn't adequately account for such transfers, or for students who required more than four years to graduate high school.
"The Pennsylvania leaver system puts all districts at a more inflated rate," says Jerri Lippert, executive director of the Pittsburgh school district's Office of Curriculum, Instruction and Professional Development. "We're looking forward to more accurate data."
Beginning in 2012, the state will change to what officials consider a more precise "cohort rate," which will take into account student transfers and other factors. Using the cohort approach, Lippert estimates, the district's graduation rate should drop to the 65 percent range. That would bring it closer in line with the district's proficiency scores on 11th-grade PSSA exams.
In the meantime, Lippert says, the district does require all graduates to demonstrate proficiency on their reading and math PSSA tests. If they fail, they must retake the tests they failed in October of their senior year. Should they fail again, the district requires students to take, and pass, a remedial course on the material before they can graduate.
Lippert, however, agrees that the poor test results reflect a problem. Poor performance on the PSSAs, she says, is one reason the district is carrying out a variety of high-school reforms.
"You're not going to find anybody in the Pittsburgh Public Schools to say that we're serving students the best we can," Lippert says.
In the meantime, the state has prepared another method of trying to shrink the gap between test scores and graduation rates: making high school students take yet another test.
Beginning with the class of 2015, high school students will begin taking Keystone Exams as a graduation requirement. The exams will replace the last round of PSSAs, and test students in English, math, science and social studies. Unlike the PSSAs, students must pass the exam in order to graduate.
The tests have already been controversial. Opponents have criticized them as an unnecessary obstacle to graduation that could increase the number of high-school dropouts. Arguing that low-income and minority students often enjoy fewer educational opportunities than their more affluent, white classmates, officials from the Pittsburgh chapter of the NAACP have worried that the Keystones wouldn't test all students fairly.
"When you have different expectations for students and then you want to put them all in the same testing, that's unequal treatment," Marilyn Barnett, who chairs the education committee of the Pittsburgh chapter of the NAACP, told the Pittsburgh Courier in 2009.
Still, supporters say the Keystones will ensure students have the necessary knowledge to succeed after high school.
"It's a better way to assess students," says Lippert. "They're better-constructed tests and more rigorous."
"We are interested in producing high school graduates who are college- and career-ready," agrees Eller, of the state education department. "The Keystone exams provide us an opportunity to do that."
