Analyzing Keystone Analytics

From the outset, I’ve been a little wary of the poll findings from Keystone Analytics, an outfit owned by the state Realtors Association who has been providing almost all the polling data in this year’s mayoral race. It’s nothing personal; I just never heard of them, and they have no track record in predicting races at all.

And then last week, as I noted in a series of tweets, it turned out that the Realtors are backing one of the mayoral candidates, Jack Wagner. Judging from Wagner’s campaign-finance reports, a political committee associated with the Realtors — the “Alliance for a Better Pennsylvania” — spent $$8,500 on the mailing. (I’ve seen a copy, though I don’t have the wherewithal to post it here. Suffice it to say it touts Wagner for saving taxpayers money through audits and so on. Ryan Barton, who represents the association’s Pittsburgh chapter, says the Realtors felt Wagner “had a good understanding of our issues,” especially on such key topics as eminent domain and taxation.)

It might be worth nothing that Team Wagner says they had nothing to do with the mailing, and weren’t aware of it until the Realtors sent them a notice that it was going out. (This isn’t the first time Wagner has benefited from such spontaneous largesse, apparently: Although Wagner was recently endorsed by a pro-life group, the campaign says it did not fill out a questionnaire or do anything else to seek that support.) If that’s the case, it’s not clear to me that Wagner had to report the mailing at all. So more power to ’em for erring on the side of openness.

But this raises the obvious question: If the Realtors have a dog in the fight after all, can their Keystone polls be trusted?

That question might seem a bit less pressing today, now that Keystone has shown Peduto with a 7-point lead over Wagner. And Jennifer Shockley, who handles press queries for Keystone, says “Everything’s on the level.” And yet this is just another wrinkle in a campaign that has already been notable for the way third parties are getting mixed up in the race. First we had Luke Ravenstahl paying for outside advertising to attack Bill Peduto; now we have another third party — one not affiliated with any candidate, media outlet, or university — releasing its own polls. I’ve never seen that before in a local race … but because there’s been a dearth of other polling, Keystone has been getting a lot of attention.

The treasurer of the Alliance is David Phillips, who is also the CEO for the state Realtors association and is thus Shockley’s own boss. But Shockley told me that while Keystone Analytics is “a wholly owned subsidiary of the [the Realtors association], the Alliance is behind another firewall. Those two entities don’t talk to each other. Whatever we send out to the public is what the alliance gets, and they don’t get anything else.”

Other association members told me that there is some interaction between the Alliance and Keystone. Barton and Kim Shindle, the association’s media-relations contact, told me Keystone targets likely voters for the political mailings, among other things. But obviously the real concern here is whether Keystone’s numbers are being skewed to help Wagner. And I don’t see any real evidence of that sort of book-cooking.

The only other independent poll — an early-April survey conducted by Susquehanna Polling and Research at the behest of the Tribune-Review — came up with results that were statistically identical to those of Keystone poll released at the same time. Both those polls showed Wagner out in front by 7 points.

On the other hand, Susquehanna’s own brand got dinged last year, when it was just about the only polling outfit in Pennsylvania predicting a close race between Barack Obama and Mitt Romney. And there’s always a chance that Keystone could — without meaning to do so — be understating Peduto’s potential margin of victory.

Back when the numbers were bad for Peduto, his backers consoled themselves with the belief that Keystone was undersampling his base. For one thing, they claimed, the poll didn’t sample enough young voters. And you could still make that argument with the new data: As with the previous survey, only 7 percent of voters sampled were in the 18-to-34 age group. Only 8 percent were from Peduto’s homebase of council district 8.

Shockley said she’d put me in touch with someone who could explain Keystone’s methodology, though that hasn’t happened. But suffice it to say that Keystone didn’t compile that sample randomly. It’s based on previous historical patterns: My own non-scientific-but-still-time-consuming check showed turnout in district 8, for example, was actually pretty close to the Keystone model. But then, in the races I was looking at, there wasn’t a viable mayoral candidate from district 8. Would having a hometown hero boost turnout there? I’ll leave it to Chris Briem — and the voters, if he’s too busy — to be the ultimate arbiter, but the argument doesn’t seem implausible. And whenever a poll measures “likely voters” — as Keystone does — you can always question the screen being used.

My guess, in fact, is that people will have their suspicions no matter what. Today’s numbers were good for Peduto, but I’ve already seen some online speculation that Keystone may be trying to make his supporters complacent. Had the numbers come out in Wagner’s favor, by contrast, I suspect Keystone would be accused of trying to incite despair. That’s part of what makes this all so strange — Keystone is a new player, and accordingly people are wondering whether it’s running some kind of game.

And if I were to say that Peduto might actually have a bigger advantage than Keystone suggests, someone would probably log on to Twitter and accuse me of encouraging complacency. So let’s just leave it at this: So far, I don’t see any reason to think Keystone is skewing data. But it’s polling assumptions seem based on the belief that the future will be like the past. Which means that if Peduto’s boosters can break with history, they have a chance to make some history.

And with 18 percent still undecided, there’s obviously plenty of work for all the candidates to do.

This article appears in May 8-14, 2013.