|
Today we had our most productive meeting so far. We now have a fairly concrete plan to succeed. We reviewed our single proposal, reviewed the other options I proposed, and settled on a compromise solution. But first, the background:
Introducing Dr. Ben Adida
 I have introduced Dr. Kernighan and Dr. Felten before in this blog, but Dr. Adida is a new and most welcome addition to the team of experts I've been in touch with about the project. I requested an email be sent to all Princeton Computer Science undergrads and grads, and one of the handful of responses I received was a suggestion to look into the Helios Voting System developed by Dr. Ben Adida. Although I had come across a slew of run-of-the-mill online voting systems in our previous research, as I explored Helios, I recognized that this voting system was different from the rest. It's the difference between an abacus and a TI-89; other voting sites count things, but Helios brings some sophisticated computing power.
Dr. Adida (who asks to be just called Ben) is a graduate of MIT who is a postdoctoral fellow at Harvard's Center for Research on Computation and Society. He researches security and privacy in voting, health records, and web applications. He is a member of the program committee for the Workshop On Trustworthy Elections (WOTE) and the USENIX/ACCURATE Electronic Voting Technology Workshop (EVT). In fact, he and Dr. Felten will be brushing shoulders on August 10th at the EVT/WOTE 2009 in Montreal. Could you imagine a better team of experts to advise our project?
Dr. Adida was kind enough to offer to speak with me by telephone last night. I had only begun emailing him the day before. We discussed his Helios system, Princeton's predicament, and the MedForward proposal. Below are a few highlights from the conversation:
- All elections have hiccups. What differentiates successful elections from failures is how these hiccups are handled.
- Our best bet with our elections project is to be "iterative and conservative." Rather than completing the system in one shot, it is in our best interest to make realistic goals and continuously develop the system for some time until it is ready.
- Although Princeton currently uses Single Transferrable Vote for certain election needs, Range Voting could be a more straightforward alternative.
- With the Helios Voting System, voters get the freedom to observe the election and make sure everything is okay.
The Helios Voting System
So I've mentioned the Helios Voting System, but not yet what it is. So what is it? In short, it is a cutting edge online election system that puts the power to audit the election in the hands of the voters and observers. Helios allows voters to make sure their vote counted, to modify their vote before the close of the election, and to audit the entire election once the scores are tallied. It leverages advanced cryptography to ensure vote security and voter anonymity, while simultaneously improving overall election transparency.
Learn about cryptography and voting in a Google TechTalk video presentation given by Dr. Adida.
This all sounds great, but does it actually work? To my delight, Helios actually works very well. In fact, the Université catholique de Louvain (UCL) used Helios to conduct an election for their university president, and it was no small deal. Below is an excerpt from their site (warning: it is a Google automatic translation cleaned up for enhanced English readability by me)
It is apparent that, with our 24,000 voters, electronic voting is the best solution, and that this system must be reliable and ensure the highest degree of security and confidentiality. Fortunately, the election commission has the help of Professor Jean-Jacques Quisquater, an expert in cryptography. After studying a variety of voting systems with his team, he suggested Helios, a system designed by a Harvard professor (Ben Adidas).
Three reasons motivated this choice, says Professor Olivier Pereira. First, voters have a high level of control at each stage, and we can verify that everything is operating correctly, and thanks to the internet, a voter can verify that her vote has been recorded. This system produces the weighted count of the ballots submitted by voters, nobody (nor even a computer) can ever determine who voted for whom. Finally, the system makes it virtually impossible to vote incorrectly: first, the voter is prompted to confirm his vote, and then he has the opportunity to vote again if he thinks he is having a problem or made a mistake (that is one of the great peculiarities of the system). Helios has, of course, been adapted to the needs of the UCL. This has been the work for several months of Olivier Pereira and Olivier de Marneffe in collaboration with Ben Adidas and the General Information System (ISMS) [presumably the equivalent of Princeton's OIT]. Among the challenges, they had to ensure "the soundness of the system given the large amount of data to be processed and the risk of attacks." That is why he suggested service providers, namely Amazon and Google: Amazon hosts the interface designed by UCL, and Google will store the data. "The best way to ensure the availability of the voting system for the duration of the election was to have extremely powerful servers arranged across the world. That is what we use Google and Amazon," explain two members of the technical team.
Source: UCL Website via Google Translation
During the first round of voting, one of the candidates missed the majority by just 2 weighted votes (faculty votes counted more than student votes in this election). An independent company also audited the election, and certified that Helios had correctly tallied the votes. A second election was held, and that candidate finally achieved the majority he needed. In a paper detailing the outcome of the election, Dr. Adida and his colleagues at UCL write:
The deployment of an open-audit voting systems brings a number of small changes in the voting process, which
inevitably makes voters suspicious. Numerous presentations were given and discussions fostered make voters more familiar and comfortable with the system. The features that were the topic of significant questions and debates were: (i) the possibility to revote, (ii) the identification of the voter at the time of ballot submission (instead of before the ballot creation), and (iii) the point of the bulletin board audit day (which delayed the announcement of the election results). From anecdotal evidence and the small number of complaints, it appears eventual acceptance of the system was quite high.
What Can Be Done with Helios?
Helios will soon be released as an open source project written in Django, a rapid web development framework based on Python (An older version of Helios is open source too, and is available here). The Django connection is particularly exciting, given the number of students at Princeton who have worked with Django and know how to use it. In fact, the group of students that came together under the assumption that they would be building a voting system from scratch had planned to use Django anyway. So this means that rather than starting from scratch, this group could simply extend the work of Dr. Adida, and modify Helios to meet our own specific needs. All of the hard work of ensuring the system tallies votes correctly has been done! All that remains to be added is the administrative and registration component, where candidates and referendum advocates can sign on and register for the election, and where the elections managers can manage these positions. Amazing, right?
The MedForward Proposal
When we sat down for today's meeting, we began by reviewing our one and only response to our RFP. I highlighted some of the benefits and drawbacks of this proposal:
| Positive |
Negative |
On Time
MedForward promises to deliver the system on time. |
Lack of Elections Expertise
MedForward specializes in building websites for medical professionals, not building secure election systems. |
Meets Most Requests
They have committed to satisfying most of our requests for the implementation. |
Use of ASP.NET and Windows Server 2003
The platform has a steep learning curve if students ever need to modify it, and it will be difficult to obtain affordable server space running Windows Server 2003 and to keep running this older technology for up to 10 years. |
Potential for Future Support
MedForward will likely be able to support the system after it is built, through the lifetime of the system. |
MedForward Retains Intellectual Property
After Princeton pays to have the system built, MedForward will retain ownership of the code and related intellectual property. |
On Budget
The proposed price is within our original expectation of $7,500 to $15,000. |
Potential to Become Expensive
The price does not include the Single Transferrable Vote system, which will cost $2,000 more to develop, and combined with the additional costs of support and maintaining the required server, the lifetime costs could approach $20,000. |
After weighing these positives and negatives, we decided against choosing the MedForward proposal. Chief among our concerns was how developing voting systems is not something MedForward has done before. After consulting with Dr. Kernighan and Dr. Adida, both expressed concerns about the complications inherent in developing online voting systems, and were unsure how well MedForward, or anyone, could implement the system we designed with just over one month to do so. Dr. Adida recalls one voting system he built taking two years to complete. And Dr. Kernighan wrote to me via email, "MedForward is presumably a fine company, but they claim no experience with voting systems, which have different constraints than most commercial systems. For instance, voting systems have to authenticate the user, permit him or her to vote only once, and make it impossible to determine how the user voted." It's far more important to me that we come upon an airtight system that will last for up to 10 years, rather than settle quickly and potentially regret making a hasty decision. So although I was extremely pleased to receive the proposal by MedForward, and we appreciate their offer to help us, in the end we felt that it was too risky to select them. This decision was unanimous among the committee members in attendance.
UPDATE 9/29/09: At the request of MedForward, I have redacted the price of the project in their proposal.
Download the MedForward Proposal
What Now?
Considering that we decided to decline the proposal from MedForward, we now found ourselves without a developer, and no guarantee of a new system by September. This looks bad, huh? Actually, I think we're in even better shape than we would have been if we had begun developing the system with a professional developer. Let me explain.
After speaking with Dr. Adida, I know that our September deadline to develop a complex voting system may have been too optimistic. I was cautiously optimistic last May, but it looks like my cautious choice of words, "it would be 'ideal' to have the new system in place for the Class of 2013 elections next fall," was duly necessary. This isn't as much an "I told you so" as it is a recognition that this project was more complicated than we had hoped. So even though we won't have a new system in use by September, we have accumulated crucial insights from the panel of experts and have discovered Helios. In the end, the result will be better than if we made hasty decisions in order to have the system ready by September.
One Suggestion
I suggested to the group that we may be able to use a minimally adapted version of Helios in order to run the class of 2013 elections this fall so that we can at the very least have something new and reliable in use to tally the votes. Helios doesn't currently support Single Transferrable Vote, but the upcoming election doesn't require that. In fact, Helios can do everything WebSurvey (our previous election system) does for the fall election, plus Helios offers the advanced security and transparency features. All that would have to be changed would be to allow Helios to use our CAS netID authentication system, and to communicate with the Registrar's database in order to verify the class year of the voter.
So, considering that I hope to use Helios as the backbone of our new system we will develop, I suggested we begin using Helios for the fall election instead of WebSurvey. This would be an ideal election to begin using Helios: it is the smallest and simplest of all of the elections. However, the committee was uncomfortable using Helios right now. Instead, they suggested that we continue using WebSurvey, so this was the option we are compelled to take.
More WebSurvey?
You may be saying to yourself, "More WebSurvey? That's the system we had so much trouble with in the first place! How can you claim that we've made any progress if we're still using WebSurvey?" That's a good question, and the answer requires understanding exactly why we've had trouble with WebSurvey to realize that WebSurvey didn't cause the problems we've had, it only contributed. As Daisy says in The Great Gatsby, "it takes two bad drivers to cause an accident" (not a direct quote). So who was the other "bad driver?" Sub-par elections administration.
Pointing fingers is a poor way to make progress, but we need to point the finger so we know where to make improvements. So I'll review some of the reasons why previous elections have been problematic:
Winter 2008
Problem 1: Admin Access
In the news
What happened?
Because of unusual circumstances in which the USG President openly supported a candidate and spread misinformation about another candidate's endorsement, the senate was tasked with deciding whether to conduct a revote after the original vote had already taken place. There was suspicion that the USG President based his sway on whether to conduct a revote on knowledge of the turnout of the first vote that he wrongfully obtained via logging into the voting system to obtain the results.
What can be done to prevent it?
It was suspected that the USG President accessed the election results by logging in with the USG@ shared netID, which, in addition to the USGVote@ netID, had access to the voting system and the results. Immediately following this allegation, changes were made to block the usg@ netID from gaining access to the system. Therefore, this type of problem has been addressed. However, the USGVote netID could also potentially be shared, which would grant access to anyone with the password. Instead of using these shared netIDs to grant access, I suggest using personal netIDs, which are less likely to be shared and provide additional security.
Problem 2: Incorrect Vote Count Published
In the news
What happened?
The source of the problem was a quotation mark in one of the candidate's names. This quotation mark caused the system to ignore votes for two candidates. Every time a vote was disregarded, an error message was printed to the log where the vote counts are displayed. There were 392 of these error messages. There were other, negligible error messages, for when someone chose the same candidate twice for a senate seat. The elections manager and the Registrar did not notice the unusual error messages, nor notice that the sum of all votes counted was remarkably less than would be expected. The result based on the erroneous vote counts was published, but vote counts were not.
What can be done to prevent this?
First, the elections manager can more carefully inspected the system output to check all error messages and ensure that they don't affect the outcome of the election. Next, the elections manager can compare the total number of votes cast with a calculated expected voter turnout, where this expected value is based on previous elections. Then, the Registrar can double check the elections manager's checking. And finally, the vote counts can be published, so that the public can also double check that votes were reported for every candidate, and that the vote numbers generally seemed to follow public preferences.
Spring 2009
Problem 1: Election start delay and erased votes
What happened?
Students began voting before the USG president had announced that voting had begun, and the ballot they were voting on was premature. The USG president announced the actual start of voting when the ballot was ready, and all votes cast before then were not counted.
What can be done to prevent this?
Voting should not begin until the USG president's official announcement when it has been confirmed that the ballot is ready.
Problem 2: Incorrect scoring on Referendum I
In the news
What happened?
There was a mismatch with how the elections manager created the XML file that creates the election, and how the election system tallied the results. As a result, some responses were inverted.
What can be done to prevent this?
The XML file that creates the election (for WebSurvey only) can be verified before it is used. And for a future system, there would be no need to manually map the database values with the choices -- it should be completely automatic so as to avoid any possibility of confusing them.
Notice a Trend?
Given these examples of problems that happened, the trend you may notice is that there were no problems that couldn't have been avoided were there some more comprehensive election oversight by humans. Without a doubt, the technical system seeded some of the problems, but none of the final problems that occurred weren't preventable. A better election protocol, including more time built in to allow for careful verification of all parts of the election, could compensate for these issues.
So what this means is perhaps the best change we can make is not to the technology but to ourselves. The "ideal" voting system I designed, and we will still design, will help alleviate some of the issues that would require human oversight. But that only makes it easier. As Dr. Adida commented, all elections have problems. The difference between a successful election and a failed election is how the administrators handle them.
Advice from the Expert: Dr. Ed Felten
 As I mentioned in a previous post, Dr. Felten has become a part of our team of expert consultants. When I mentioned the class of 2012 senator debacle, Dr. Felten offered:
Having read about the past botched election, my first thought was that the error would have been evident if the system reported results more fully and to a broader audience. Just seeing no votes for certain candidates would have been a strong clue that something is wrong -- and the fact that the number of votes reported was many fewer than the number of voters would have been another giant red flag.
Therefore, I think it's in the best interest of everyone for the USG to commit to reporting all vote counts to the public. This means every election, every position, and every candidate. But Dr. Felten also agrees that the technology needs to change, too.
The people who saw the debugging output could have caught the error, in principle, but putting the blame on them seems too easy, and in my view that would miss some serious problems in the technology and -- at least as important -- in the human processes used in the election. The problem would have been detected if
(a) the system had shown voters the choices tabulated for them, at the time they voted, or
(b) vote totals for each candidate had been reported to the public or even to the candidates, or
(c) the error messages had been intelligible (it was a big problem that the real error messages looked very similar to messages that would normally occur), or
(d) the system had displayed a count of the number of errors of various types, in easy-to-read format, or
(e) the error messages had occurred somewhere other than a giant file of random-looking debugging output, or
(f) the system had been programmed to raise a red flag when the number of votes recorded was far less than the number of voters, or ....
Safeguards against error are the most important part of an election system. Often the best safeguards are not technological at all, or involve only easy technical changes such as providing clear error reporting and building simple consistency checks into the system.
If you're wondering what the election output looks like, download the election output from the original class of 2012, December '08 election below. This is an excerpt from the entire output, which includes the rest of the positions and referenda. The header was added by my text-editing program when I made the PDF file, but the actual text as shown is unedited. This is what the elections manager reads in order to determine the winner, and this is the text the Registrar reads to certify the election. A little confusing, right? No wonder there have been so many problems!
Download the original election output from the class of 2012, December 2008 ballot
Making WebSurvey Work for the Fall
So we still can use WebSurvey to run the election in the fall, as long as we implement a new, more thorough election protocol in order to prevent and mitigate any hiccups that can (and likely will) occur. Given this additional time, we can then develop the system to make the technology work more effectively, too. Although I would have liked to have had something more tangible to show for our work by September, this seems to be an acceptable option, and frankly the best that we can do.
What's Next?
Although we won't be using Helios for the fall election, and we won't have a new system developed by then, we will be simultaneously developing our own system that works with Helios and continuing to accept proposals from commercial developers.
By November 1, we will stop taking proposals and then compare the proposals we receive from commercial developers with the system developed by Princeton students that interacts with Helios. We may then choose the system developed by the students, or we may select a commercial developer to execute the system.
What's the Timeline?
Starting immediately, the IT Committee and the Election System Developers will begin developing an administrative component that will interact with Helios (I'll continue describing this below). By the first USG senate meeting, I will have a proposed protocol for the election. Immediately after this protocol is adopted, we will begin planning for the election, which will include a meeting with the OIT technician responsible for WebSurvey, the elections managers, certain USG representatives, and the Registrar. The election will take place in the beginning of October, using WebSurvey. Then, in late October, using whatever parts of the Helios-USG collaborative project that is ready, we will conduct a pilot election -- perhaps a non-binding survey, a Mr./Mrs. Princeton contest, or just a mock election. Based on that election, we will have gained valuable insight to how we can modify the system to be even more airtight. Then, by November 1 (or around then), we will stop taking proposals and convene to review these proposals. Then, we will compare the Helios-USG system with the top proposals (assuming there are any), and choose the best system. If we decide to choose a commercial developer, then we will begin developing with them. If we choose to use the Helios-USG system, then we will continue modifying that to be ready for the winter elections. And then we will continue tweaking the system between elections until it is officially completed.
What will the new system look like?
If you haven't already visited www.HeliosVoting.org, check it out now! In fact, create your own sample election, and vote in it. You'll have to login with some Google ID, like a GMail account, since the system runs on Google's servers.
 Helios already conducts the actual voting part very well, but it lacks the candidate/advocate self-registration aspect we would like to include. The thought I have is that the student developers will create a standalone system (I'm informally calling this part "Luna." And yes, I know Luna is Roman, not Greek like Helios, but I think "Luna" is more mainstream than "Selene," the Greek counterpart.) that allows candidates/advocates to register for the election and the elections manager to manage these candidates. Then, once all candidates have registered and are verified, the elections manager essentially "flips a switch" that sends all of the information over to Helios for Helios to then create the ballots and prepare for voting. This will create a special barrier between the two systems that will keep Helios as protected as possible.
However, before we actually decide what we are going to do, I have to arrange a phone call with Dr. Adida so we can carefully plan this.
As the wise Kel Mitchell once said, and the ever-sagacious Coolio sang, "Ah, here it goes!"
|
Two weeks ago I delivered my third presentation to the senate regarding the elections project. This was the first presentation to the senate since we started working with Helios. The purpose of the presentation was to explain what Helios is, how it works, and what we can do to use it.
