In March 2001, as the Senate readied for a showdown on campaign finance legislation, Kenneth Goldstein, a University of Wisconsin political scientist, received an urgent call on his cellphone while standing at the baggage carousel at the West Palm Beach airport in Florida.

Officials at the Brennan Center for Justice were scrambling to complete a new report on the impact of so-called “sham” issue ads in time to influence the Senate debate. But they ran into a problem with some of the data put together by Goldstein and his students that examined numerous television ads in the 2000 elections.

Goldstein’s students had looked at storyboards for a number of ads and classified them as either “genuine” issue advocacy or thinly disguised electioneering messages. The Brennan Center authors, racing to send the report to press, worried about some of the students’ judgment and called Goldstein. As Goldstein waited for his luggage, the Brennan Center writers read to him the text of the ads and asked his opinion on whether they were genuine or sham issue ads. Goldstein, unable to view the ad storyboards at the airport, nonetheless overruled many of the conclusions of his students and ordered that the ads his students believed were examples of genuine issue advocacy instead be coded as sham issue ads intended to affect an electoral contest.

The airport scene, tucked deep within the thousands of pages of court documents in the campaign finance litigation that reignited this month with the bitterly splintered ruling from a three-judge panel, dramatized the fierce behind-the-scenes battle over the reliability of two widely cited Brennan Center reports that played a remarkable and pivotal role in the drive to enact campaign finance reform. The stakes involved some of the most fundamental questions about the role of speech and money in the political arena.

A Powerful Tool

Reform advocates struggled for years to craft rules that would curb the explosion of electioneering ads paid for with unregulated soft money while still allowing legislation to pass constitutional muster. To demonstrate that the law’s regulations would have a minimal effect on the ability of interest groups to run genuine issue ads, reform advocates armed themselves with the two “Buying Time” studies produced by the Brennan Center that focused on the role of television ads in the 1998 and 2000 elections.

In a nutshell, the reports concluded that the vast majority of advertisements aired in the 60-day window before Election Day — 93 percent in 1998 and more than 99 percent in 2000 — were not what reform advocates viewed as “genuine” issue advocacy but were instead what became commonly known as “sham” issue advocacy.

In March 2002, Sen. Olympia Snowe

(R-Maine) trumpeted the Buying Time reports’ “most compelling” findings on the Senate floor.

“They found that just 1 percent of all those ads run during the year that were viewed as actual genuine issue ads and mentioned federal candidates were captured by our provision,” Snowe said. “In other words, of all the so-called issue ads that ran last year and mentioned federal candidates, 99 percent of those that ran in the last 60 days were seen as electioneering ads,” Snowe said.

“If you had any test that was accurate 99 percent of the time, I believe you’d say that was a pretty good test,” the Maine Republican added.

A Numbers Game

As soon as their lawsuit was filed last year, opponents led by Sen. Mitch McConnell

(R-Ky.) and a diverse set of liberal and conservative interest groups set out to show that the test used by the Brennan Center wasn’t reliable at all. And they succeeded at least in convincing each of the three judges that the numbers cited by Snowe and other reform advocates including Sens. John McCain

(R-Ariz.) and Russ Feingold (D-Wis.) on the Senate floor were simply incorrect.

Through subpoenas and lengthy depositions, the plaintiffs uncovered e-mails, documents and mind-numbing testimony from an array of experts and officials who examined every aspect of the Brennan Center studies from start to finish. Since the court trial of the new campaign finance law was conducted entirely on paper under a compressed schedule, the dispute over the Brennan Center reports escaped public attention.

But the plaintiffs say their unrelenting scrutiny proved conclusively that the Brennan Center reports were fundamentally flawed, riddled with errors and unreliable.

Whacking the Piñata

The sprawling, fractured ruling proved that the three judges on the panel could not agree on much at all. But a close reading of the opinions shows that each of the three agreed that the central finding of the Buying Time reports, the one cited by Snowe and other reform advocates to prove that the restrictions in the law would have little effect on the flow of genuine issue advocacy, was incorrect.

Despite their misgivings, two of the three judges — Richard Leon, a 2001 Bush appointee to the U.S. District Court for the District of Columbia and his colleague Judge Colleen Kollar-Kotelly — ultimately accepted the thrust of the Buying Time findings.

But all three judges took official note of the quarrel fought in the briefs.

“The effort is not unlike that of a piñata party: If one hits the piñata enough, it will eventually crack apart,” noted Kollar-Kotelly, who while ultimately defending the reports’ findings observed that “some of these hits have merit.”

Leon, in a conclusion that has confounded conservative opponents of the law, delivered the most detailed and comprehensive defense of the Brennan Center studies.

“After reviewing the expert reports, I find that although the Buying Time studies contain some flaws and shortcomings … those shortcomings do not detract from the studies’ credibility and reliability,” Leon wrote.

U.S. Circuit Judge Karen LeCraft Henderson reached the opposite conclusion, writing that “neither study has any significant evidentiary weight.” And she disparaged the work of the officials and academics who oversaw the two studies in a blistering set of legal findings.

“The Brennan Center and the authors of the Buying Time reports sought to achieve a certain result and therefore sacrificed scientific objectivity,” Henderson concluded.

In addition to noting Goldstein’s airport recoding scene that affected the 2000 Buying Time report, Henderson also excoriated the 1998 study, finding that its conclusion that the campaign finance law would restrict just 7 percent of genuine issue ads was erroneous and, perhaps even more damaging, that the authors of the report knew the 7 percent figure was wrong.

Basically, the study was conducted by showing the storyboards of campaign ads to groups of political science students who were then asked to classify their perceptions of the purpose of the commercials. If, after looking at the storyboards and the ad text, they believed it was intended to support or oppose a candidate, it would be coded as an electioneering ad. Students could also choose to classify the commercial as genuine issue advocacy in cases in which, while referring to candidates, the ad was relatively neutral on supporting or opposing candidates at the ballot box.

Henderson cited e-mails and memos sent back and forth between the Brennan Center authors revealing that they considered the 7 percent figure to be “either false or so vague as to mislead the reader.” One of the authors put the correct percentage of “genuine” ads affected by the law at 11.38 percent, while the other calculated it to be 13.8 percent. Depositions also showed that, at times, the authors disagreed with each other over whether a particular ad was genuine or an attempt at electioneering.

Despite their debate about whether the numbers published in the 1998 study were inaccurate, the authors and the Brennan Center decided against correcting the record with their new calculations that would not have been as helpful to the cause of campaign finance reform. Two other experts for the defendants conceded that the 7 percent figure, under a formula used in the 2000 study, would actually be as high as 14.7 percent. Plaintiffs, meanwhile, countered with their own expert who calculated the number of genuine ads that would have been prohibited to be as high as 64 percent after finding that a number of student evaluations had been changed by Goldstein.

Those changes, referred to as “recoding” in the testimony, clearly bothered Judge Kollar-Kotelly.

She noted, “I am troubled by the fact that coders in both studies were asked questions regarding their own perceptions of the advertisements’ purposes, and that these perceptions were later recoded.” She went on to point out that the “principal casualty in this regard are the conclusions the Buying Time studies make regarding the percentage of ‘genuine’ issue advertisements ‘captured’ by” the campaign finance law. The record, in her view, was so contentious that she simply gave up attempting to figure it out.

While seemingly a futile, tedious back-and-forth academic exercise between feuding social scientists, the statistics actually carry significant importance when it comes to deciding whether restrictions on political ads are in fact constitutional.

The point was driven home by Judge Leon, who, despite the dizzying number of statistics measuring the number of genuine issue ads that would be prohibited, concluded that the law reached too far to be constitutional. Leon settled on very different numbers than the figures used in the Buying Time reports.

“Percentage discrepancies aside, I find that 14.7 percent and 17 percent of the ads run in the months leading up to the 1998 and 2000 elections, respectively, represents a ‘substantial amount’ of protected speech and renders the primary definition defective as constitutionally overbroad,” Leon wrote, referring to the main provision in the campaign finance law that banned ads using corporate or labor-union money within the weeks before an election.

Leon joined with Henderson to strike down the provision that had been advocated by Snowe and others. And again, in a move that has astonished some conservative legal observers and become a flashpoint for appeals to the Supreme Court, Leon rewrote the law himself, crafting an alternative restriction on issue ads that many believe is far more restrictive and comprehensive because it would regulate all corporate or union-funded ads that support or oppose any candidate.

‘No Preconceived Notions’

Despite the judicial rejection of one of the key elements of their reports, Brennan Center officials stood by their work, pointing to the passages in Leon’s and Kollar-Kotelly’s opinions that embraced the overall findings of the two studies.

“There was an enormous effort on the other side to undermine these reports,” said Frederick Schwarz, interim president of the Brennan Center. “But I think they hang together pretty well and that the two judges are correct.”

Still, the attacks registered. “If you are asking me, was the research perfect, the answer is certainly not,” conceded Brennan Center spokeswoman Amanda Cooper. “Because it never is and things have real-life pressures.”

Contacted by telephone in Israel last week, Goldstein defended his academic research and methods while expressing some frustration with the strategy to undermine his work on the part of the plaintiffs.

“I hate to sound like the naive political scientist, but I really came into this with no preconceived notions. And when I started this process, I had no idea I would be giving depositions and going through discovery of all my e-mails for the last three years,” Goldstein said.

He shrugged off the airport scene painted so dramatically by the plaintiffs as ultimately inconsequential.

“I was on vacation and they in fact did call me at an airport. It sort of cracks me up how they make a big deal that I was on a cellphone at an airport. It makes it sound like the most unusual thing in the world.”

He said that even if he had not changed the evaluation of his students, the percentage change would have been minimal. Instead of finding that 0.6 percent of genuine issue ads would have been unfairly captured under the law’s provisions, the number would have been 2.3 percent, he said.

And, he pointed out that the two reports involved more than 40 million data points and that there were bound to be a few mistakes in such a large universe.

“I realize the stakes are big. I’m not naive. Honorable people can disagree about issues. I think perfectly honorable people can be against [the campaign finance law]. I think there are many perfectly honorable people for it. But to, I think, unfairly critique these findings is not proper,” he said.

“If it quacks like a duck, it’s a duck,” Goldstein said of the interest group ads that clearly appear to be aired to influence elections. “Honorable people can disagree. There can be mistakes in data. But the question is: Are there systematic mistakes? Are there bias? And are any changes changing major substantive conclusions? I think the answer is clearly no,” Goldstein said.