The poll questions are kind of arbitrary and nonsensical because I’m focused on testing the behavior of this widget inside of a WordPress post. Right away it’s evident that if I want the poll to be properly sized, I have to either squeeze, move, or outright eliminate the sidebar widget.
Data Driven Design
Visualizing Political Polarization in the USA
This interactive graph published by the Pew Research Center, tells a couple of important stories about the political mood in the Untied States. Neither is a big surprise. One is that the parties are more polarized, especially among the politically active. The second is that the country as a whole is not polarized (though not coalescing around an ideological consensus either), and tends to be moving slightly left overall.
Matthew Dickinson at Politico has a good analysis.
New RKEY Logo
I’ve been playing with logos, getting help from here and there. Still tweaking, but pretty soon I’ll be able to move on and print up new cards.
Ranked Choice Polls and Sygnol Analytics
I first became aware of ranked choice mathematics in mid 2000, when the Internet Corporation for Assigned Names and Numbers (ICANN) used Instant Runoff Voting to elect 5 at-large members to the Board of Directors. It’s hard to think of a better approach to support a massively scalable deliberative decision-making system for the Web.
Finally, in 2007, I decided to try building the system I had been thinking about. A lot of time and effort has gone into this since then, under the aegis of what I call the “Indaba Application Network.” IAN’s ranked choice polling sites have included Indaba.org, ChoiceRanker.com, WeVote.net, AmericanQuorum.com, and Mayor2011.com (the last three were also Facebook apps). These were all PHP sites that shared the same underlying MySQL database.
I’m currently in the process of refactoring the database to support proxy voting and better scalability. I’ve also been pursuing a branding strategy for a hybrid liquid democracy/opinion research platform called Sygnol. It’s going to take some time, but the goal is to redeploy the database in SQL Azure, and to scale the service as an API across a broad spectrum of mobile, game, and website channels.
I’d rather be coding, but I also have to put time into publicizing this in order to attract some helpful force multipliers. So here’s my first video, which focuses primarily on the interactive ballot tool and the very cool analytics that ranked choice data make possible.
Comparing Ranked-Choice and Pairwise Evaluation User Experiences
Here’s a handy formula that expresses a key virtue of ranked-choice interfaces over head-to-head pairwise evaluations… N(N-1)/2.
Head-to-head interfaces are typical of “Whose hotter?” sites such as candobetter.com. The point is to simplify the task of evaluating a long list of options by asking people to view a randomly selected pair of images from a large set, and then select one. The choice is recorded and another pair of images is displayed immediately. It’s fine for idle entertainment since it’s so easy. Of course, the typical user gets bored and quits long before all the options are exhausted. Why? N(N-1)/2.
The reality of N(N-1)/2 boils down to this: As the number of candidates to be evaluated increases by 1, the possible number of pairwise evaluations grows at an increasingly higher number. The growth is less than half the rate of an exponential increase, but significant nonetheless. The underlying math is simple and straightforward (as Larry Bowen demonstrates), with important implications for user interface design.
One of the most advanced head-to-head crowdsourcing applications currently online is AllOurIdeas.org, a Google-funded project led by Princeton-based Sociologist Matthew Salganik. It’s a serious, text-oriented approach toward what Salganik calls “Bottom-Up Social Data Collection.” It’s been used to help small groups design their web sites and to help US citizens discuss national priorities. But even a site as sophisticated as that can’t escape the fundamental challenge of N(N-1)/2.
With just two choices per page, hot-or-not sites and AllOurIdeas benefit from having lots of room to display big, engaging pictures or crisp easy-to-read text boxes. On a per page basis, that space-saving approach leaves little room for confusion. But problems arise as the list of choices increases, taxing respondent patience and undermining the accuracy of results.
Imagine a comparison of just 6 candidates in a hot-or-not styled contest. According to N(N-1)/2, the various names would be seen a total of 30 times because someone would have to click through 15 pages of pairs before a full ordering of preferences is possible. Even worse, someone could easily enter contradictory rankings, for example choosing A over B, B over C, and C over A (This “rock, paper, scissors” problem is known to mathematicians as intransitivity). This happens for a variety of reasons, starting with failure to remember how prior choices were ranked.
Things gets worse as lists get longer. Contests such as the ongoing GOP Presidential debates, which typically present 8 candidates, would require 28 pages. A 12 candidate reality show such as a season-starting American Idol would require 66. A top 25 list such as college football’s Bowl Championship Series would require 300.
The bottom line is that a hot-or-not interface might be ideal if the only two choices are, say RCV versus hot-or-not, or if one’s main interest in publishing a site is to run up page views. But it’s a very different story when serious multi-candidate evaluations are called for, accurate responses are important, and an efficient user experience is desired.