Monday, January 08, 2007

Election Report analysis notable findings all around

ESI's Report on the VVPAT
Friday, August 18 Dan Tokaji

Earlier this week, the Election Science Institute (ESI) released this report regarding the use of an electronic voting system with a contemporaneous paper record. The focus of the report is the implementation of a Diebold system in Cuyahoga County, Ohio, but the report is essential reading for anyone who's interested in the ongoing debates regarding electronic voting. It's received a fair amount of coverage, including stories in the Columbus Dispatch and Cleveland Plain Dealer, as well as this cogent analysis from Dan Seligson of

So what was the cause of the major problems with the VVPAT records? Was it mechanical problems or inadequate poll worker training? It's hard to answer this question definitively, but I suspect that it's some combination of these and perhaps other factors.

In any event, the ESI findings suggest that there are serious practical problems with the implementation of present-generation VVPAT systems. We simply don't know whether VVPAT systems in other places have the same problems found in Cuyahoga, because no one's done a comparable analysis.

ESI's report certainly provides additional reasons for doubting the assumption that has been a prime motivation for making the VVPAT the official ballot of record -- namely, that paper records are necessarily more accurate than electronic records. The report suggests the potential for very serious problems in states that have passed such laws. Unless the mechanical and/or training issues found in the ESI report are resolved, it is quite likely that reliance on the VVPAT in recounts will lead to the wrong result in some future election.


I. In Focus This Week

News Analysis: The Coming Paper-Trail Debacle?
Ohio report finds challenges abound in evaluating voter-verified paper audit trails
By Dan Seligson

A 240-page report on failures and foibles during Cuyahoga County’s May primary raised more questions about the accuracy and reliability of touch-screen voting machines which researchers say failed to match up electronic ballots to paper versions of the vote.

Perhaps equally significant – and noteworthy – are the details of the considerable woes that plagued the voter-verified paper audit trail (VVPAT) system through careless election administration, printer failures or both.

Buried some 93 pages into the report, which was commissioned by county leaders and produced by the San Francisco-based Election Science Institute, are details of errors that included poll workers loading thermal paper into VVPAT printers backwards, blank audit trails, “accordion-style” crumpling of ballots, long blank spaces between ballots that could have represented missing or unprinted VVPATs, torn and taped-together VVPATs and missing ballot text.

ESI researchers found that nearly 10 percent of VVPAT ballots sampled were in some way compromised, damaged or otherwise uncountable, an alarmingly high proportion for a state that requires that paper be used as the ballot of record in the event of a recount.

That led ESI to the ominous conclusion that “in the event of a recount or election contest, the risk of legal challenges is exceptionally high if no significant modifications are made to the current election system.”

“The VVPAT is only as reliable as the administration of the system that produces the paper trail,” said Tracy Warren, the ESI researcher who led the manual VVPAT recount....

To critics, the high percentage of damaged or uncountable VVPATs damaged rated as significantly worse than ‘imperfect.”

“Ten percent is a complete disaster and totally defeats the purpose of a VVPAT,” said David Dill, a Stanford University computer science professor and founder of Verified Voting. “You can blame it on poll worker training, but there are ways to design equipment that makes user error less likely. There are indications that Diebold has done a less than adequate job in design. The company has adopted a generally reluctant and unenthusiastic stance to paper trails and it shows in the design.”

The answer to VVPAT problems, Dill said, would be precinct-counted optical-scan units.

“There are fewer questions about it,” he said. “We know with appropriate care and poll worker training, we can run a good election on optical scan.”..

Brad Blog disputes the pair

Cuyahoga Co. OH: ESI Report on Diebold Primary Election Failure Bastardized By Anti-Paper Trail Pundits
'Experts' Ignore All But a Few Pages of a 240-Page Report
Guest Blogged by John Gideon

Election Science Institute (ESI) of San Francisco California was hired to investigate and report on problems with the May primary election in Cuyahoga County, Ohio. The 240-page report was just released this month (August), and some in the elections community have ignored a large majority of the report and zeroed in on a single aspect that supports their thesis that voters don't need any paper, even when voting on a Direct Recording Electronic (DRE) voting machine.

Daniel Tokaji is an Assistant Professor of Law at the Ohio State University’s Moritz College of Law and the Associate Director of Election Law at Moritz. He has testified against a voter verified paper audit trail (VVPAT) to a US House committee and the Election Assistance Commission..

In his paper on the ESI report, Tokaji ignores many significant findings of the report and distorts the overall findings by highlighting only the compromised VVPAT. As pointed out by VotersUnite.Org in their analysis of the ESI report

Avi Rubin Blog

...I find it interesting that different people on different sides of the issue have used this report to back up the claims they've been making all along. One thing that is absolutely clear to me, and something I believe pretty much everybody would agree on is that such studies are extremely valuable, and we need more of them.

...I will take this opportunity, as I have in the past, to respectfully disagree with Dan Tokaji, although not entirely. I will concede that the machines used in this study clearly did not implement an ideal paper audit trail. In fact, if you read the study, it is pretty clear that there were many faults with the paper audit trail. Where I part ways with Tokaji's is in his conclusions. I do not believe that the concept of a voter verified paper audit trail should be thrown out just because there was a poor implementation of it. In fact, if you consider a ballot marking system, where there is no electronic tally, such a system qualifies as a VVPAT, and would by its nature avoid many of the problems that arose in Cuyahoga County, Ohio...

Cuyahoga report highlights VVPAT challenges
While much of the discussion in the wake of the second Cuyahoga County report has focused on the questions about the report's audit of the voting devices (and the vendor's response to those questions), our friends at Electionline hit the nail on the head this afternoon.

A great piece in today's Electionline Weekly by Dan Seligson points to the data in the ESI-lead report on the problems the report highlighted regarding the voter-verified paper audit trail (VVPAT). It's worth quoting extensively from Seligson's piece, as it is the first media report so far that I've seen that has highlighted the fact that the report found that almost 10% of the VVPAT ballots studied were problematic...



For Immediate Release: Monday, April 19, 2004
Electronic Frontier Foundation Honors Pioneer Award Winners
EFF to Honor Kim Alexander, David Dill, and Aviel Rubin at the Thirteenth Annual Pioneer Awards Ceremony...

The judges for this year's EFF Pioneer Awards were: Herb Brody (Senior Editor, Technology Review), Beth Givens (Founder and Director of the Privacy Rights Clearinghouse), Moira Gunn (Host, "Tech Nation," National Public Radio), Donna Hoffman (Associate Professor of Management, Vanderbilt University), Peter Neumann (Principal Scientist, SRI Intl.; Moderator, ACM Risks Forum), Drazen Pantic (Media & Tech. Director, NYU Center for War, Peace & the News Media), Barbara Simons (past President, Association for Computing Machinery & U.C. Berkeley Distinguished Alumnus), and Karen Schneider (Director, Librarians' Index to the Internet).

Bev Harris Vote Here

Also see Saul Iversen post

No comments: