What’s Wrong with Dot Voting Exercises
Or, what NOT TO DO (and what to watch out for) next time you decide to use dot voting
I used to love dot voting.
I don’t recall the first time I participated in a dot voting exercise, but I’m sure I felt a sense of wonder at how a skilled facilitator was able to take a jumble of recently minted ideas and get us all to coalesce around just a few of these…
Alas, I rarely feel that way anymore.
All but two of the dot voting exercises I’ve participated in over the last few years have left me feeling — more than any other emotion — frustrated.
To be clear about what I am critiquing, it’s the flavor of dot voting popularized within design thinking circles, where groups will:
- Generate a bunch of ideas —“One idea per sticky!”
- Cluster these ideas into groups or themes (sadly, this step is sometimes skipped).
- Dot-vote on these clusters/individual stickies. Everyone is typically given three sticky dots with which to vote (you’d think there’s something magical about this ‘three dots per person’ pattern, as groups rarely stray from this default).
There’s nothing wrong with this per se. As with most things, it’s in the details where things break down. Tools are tools. Some tools or more or less suited to a problem. A good tool can be poorly or expertly used. And we can always invent and modify the tools we use.
If I’m being honest, part of my frustration is that I see really great tools — such as dot voting — being wielded poorly, with lackluster results. This eventually leads to faulty conclusions about the value of such tools. This is true of tools as well as the entire practice of design. In the democratization of design methods and mindsets, I see a lot of adolescent fumbling with new things, by non-skilled practitioners . This is good and to be expected. But, this calls for those who are skilled to step into coaching roles, and model best practices. Or, as is the intention of this article, to be more explicit about things might have come naturally and never have been articulated. I have zero interest policing what people do or trying to tell people the “right” way to do something (we’re all learning and figuring things out!), but there are some things to share that might level-up usage of this tool for all of us.
Bottom line: Despite the negative nature of this title, look at this is as a provocation to make things better.
So, back to my frustrations. Why am I frustrated? My issues are mostly with the quality of objects being voted on, though there are also issues with the act of dot voting itself, including the human biases inherently built into this activity. All of these issues are solvable. Let’s look at these in turn…
Problems With ‘What’ is Being Voted On
Let’s start with the things being voted on. More specifically, how ill-structured these things are.
I think of these as information architecture problems with Dot Voting, because, well… the ideas, concepts, proposals and other things generated on the fly often have little to no thought about how they relate to each other. Let me explain with some exaggerated examples. And while these are exaggerated and easy to laugh at, these are the very problems that I repeatedly see in everything from product brainstorms to team off-sites.
Bad when there isn’t parity between things being voted on
Quick! We need to decide whether to have cookies or a fruit bowl for our afternoon snack. This, is a good and fair vote.
Sadly, I don’t see this kind of clarity very often. More often, I see something resembling this:
We get four kinds of cookies and the fruit tray to decide between. Because there are so many variations of one general idea, and one representative of another general direction, this creates a “split-vote problem” which leads to entirely different voting outcomes.
Bad when basic expectations are mixed in with differentiators
Similar to the split-vote problem described above, I often see non-negotiables mixed in with negotiables. Let me explain: Do you want windows on your new house, or a ceiling fans? I’m willing to bet you want both (especially if you live in a place that gets hot, like Texas). But, windows on the house —like the roof, doors, and plumbing —are a basic expectation.
In software design, authentication isn’t very sexy and won’t differentiate your app. But, you probably need some kind of login, right—right!? So why is this “voted on” alongside differentiating or value-add features? They’re not in the same class. If you’re trying to decide where time and effort should should go, at the very least cull out the critical, non-negotiables, first.
Bad for pieces in a system
Dot voting fails when the things you’re voting on all work together as part of a larger system. Easy example: There are nine ingredients in this recipe. Let’s vote on which three ingredients we should prioritize!
Good luck making that chocolate chip cookie with only the three ingredients you just agreed to prioritize.
This one is harder to spot, but usually shows up across many of the ideas generated. A team might decide to focus on recruitment, when that should be considered as one of the six stages of the employee lifecycle (attraction, recruitment, onboarding, development, retention, and separation). Or, to improve team performance, a group may decide to focus solely tools and processes, when we also need to talk about things like the organizational structure and psychological safety.
Bad for pace layered priorities
“Let’s fix the UI. The UI is terrible.” But what if beneath these user interface frustrations is the real behemoth — the underlying tech structure that really needs to be fixed first. And if everyone understood this, then they’d also understand that fixing the user interface issues now would be throwaway work when the foundation is eventually repaired. But everyone sees the visible problem, and cares about the visible problem, so the visible problem is what gets voted on.
Years ago, I started thinking about product and feature prioritization through the lens of pace layering. For the uninitiated, ‘pace layering’ is essentially a way to discuss different layers in a system, and how each layer changes at a different pace, from the fastest layers to the slowest layers in the system. Pace layering is often shown like this:
Pace layering is also explained using a house metaphor, where rearranging furniture (changing “Stuff”) is far easier than adding an extra bedroom (changing the “Structure”).
This metaphor is especially relevant for software design, where we have technology stacks and deep — and slow changing — layers of infrastructure sitting behind things that are far easier to change. Editing a misspelled word is far easier to change the the underling technical architecture everyone has committed to. If we think of user interface changes through this lens, it’s a good way to prioritize things according to where they sit in the stack.
Bad when there’s a relationship between things being voted on:
If there’s a parity between entities being voted on, great! But what if there’s a relationship, say ‘cause-and-effect’, between the things being voted on?
It’d be awkward to walk out of such a meeting having prioritize B and C, knowing A is dependency for both items.
Example: A team identifies a dozen or more frustrating problems. However, one of these may be the root cause of these frustrations and the best thing to tackle. Sadly, if this isn’t discussed, then you end up with a scattershot of votes, mostly addressing cascading problems, without getting to the core, triggering issue.
Bad when solutions are mixed in with problems (and outcomes)
See if you can spot the difference between these things:
Start a Friday culture club
and…
Fix problems with employee morale
While they both start with a verb, and may seem like similar things, the first is a proposed solution to the other, broader articulation of a problem.
- If you just generated possible solutions to a narrowly defined problem, that might be okay.
- If you were only brainstorming problems (or desired outcomes), that might be okay.
Dot voting on a mix of problems and solutions is very bad. ‘I want to fix that problem, but I don’t think that’s a viable solution, but that other thing over there is a solution I do like to the problem, but I have limited votes, do I vote on the the solution I like or the problem, knowing we might or might not test that solution… Yikes!
Bad when we mix easy and hard options.
We are often asked to make a binary judgement on things that differ in complexity.
Quick, cast your vote! We’re forced to vote on things with very different levels of effort. Or unknown levels of effort. Or unknown value. Maybe we shouldn’t be dot voting on these at all, but seeking to understand them better by placing them into a simple matrix or canvas?
Of course, if you must persist with voting, I’ve seen this mixed-difficulty problem solved: Classify your votes!
A Good Dot-Voting Example #1:
In this example, you see six dots in total, two for each type of vote.
Sidenote: In nearly every dot voting exercise I’ve been in, it’s always three dots, and they always signal the same thing. I love that this simple idea examines the dot as our voting unit, and uses color coding to introduce a classification. Moreover, this exercise broke the implicit “three votes” rule. Call the dot voting police! But seriously, think of all the things we could do with our votes:
- Play with the number of votes people get
- Use color coding to classify the nature of vote
- Use color coding for traceability (the engineers, product managers, and designers all get different colors)
- Use differently sized sticky dots to signal intensity
- Use two differently colored dots as votes ‘for’ and ‘against’
- And so on…
While we’re at it, voting need not be limited to dots, as John Willshire suggests:
Bad when there are redundancies and overlaps
‘How is this idea different from that one? Wait, are these the same idea? And these two ideas, they seem to overlap. And what does this have to do with anything.’ All too often, classifications are downright muddy.
Unfortunately, this kind of muddy mix — bulldog, tiger, cats, canines, pets, poodle, clown, etc. — is all too common. Especially, especially, where the brainstorming topic was very broad.
Before voting, examine the things that have been generated. Are some the same? Is the distinction between things clear? Is there an overlap in some things? Like I stated at the outset, most of my frustrations have been with the lack of information architecture in the things we vote on. This mixed information is a classic problem example.
Bad when there is a poor understanding of proposed concepts
“Wait, I thought I was voting on…” This happens a lot. And, it’s a language issue. Not a Mandarin-to-English kind of translation issue, but a good, old-fashioned linguistics ‘that’s-what-I-said-but-not-what-I-meant’ issue. What I mean when I describe something, and what you imagine in your head, are not the same. And vice versa. And know we have to assess and vote on these ideas.
Bad for Bold, Visionary Ideas
If your goal is bold, visionary ideas, dot voting is terrible, because of familiarity bias. Simply stated: We have a preference for things that are familiar. By definition, a bold, unconventional idea is going to be unfamiliar, scary, less likely to be understood, and… most likely to get fewer votes.
Exception: Unconventional ideas do get support when the person advocating for this idea is really compelling. Which gets to my next set of issues: The Human Element.
Human Problems with Dot Voting.
Let’s confront the bigger challenges in all this, those pesky traits that make us all human.
Let’s the get the biggest, most obvious issue out of the way first:
Voting is Not Blind
We’ve all been there, holding back, deliberating, wondering where we should cast our limited votes. And then, you start to see a cluster of votes on that thing you were going to vote for. Whew, you think. Now that that idea has some traction, I can cast my vote elsewhere.
Or, the opposite.
That idea that’s really important, it’s not getting any votes — quick, I’ll cast my vote, and maybe others will follow.
The live nature of dot voting also means votes are easily skewed (and often are!).
Fortunately, there are ways around this.
- If the things being voted on are numbered, you can have people number their votes (privately), before getting up to make their votes public.
- Tools such as Feedback Frames keep all votes hidden, until ready to be revealed.
- And recently, I witnessed another great (and simple) example of this of making the voting blind…
A Good Dot Voting Example #2:
Several teams — each with a competing concept — needed to align behind one concept. Much of the afternoon was spent helping each team actively listen to the other, to fully understand what the other was proposing. That said, the day needed to end with some consensus building. For the voting exercise, the facilitator did several things really well:
- Ahead of time, she prepared a structured, objective rubric that would be used to assess five different considerations.
- For each consideration, each person could only vote for one of the three concepts.
- Rather than live vote, voting was done blind, on a handout. It took a bit of extra time to ‘tally’ the votes, but the results were much more useful.
In this, the facilitator addressed both the structural issues (mentioned above) and the issues with live voting. Best of all, when a decision was made, there was alignment. Everyone agreed with the decision, and (most critically) felt they had been listened to —even where ‘their’ idea wasn’t chosen. This is powerful.
Penmanship and clarity skews votes
As silly as it may sound, communications play a role in all this. Writing clearly, or being able to articulate your idea concisely — these affect people’s rash evaluation. Maybe something innocuous as a little illustration or a star means more people slow down on that specific idea. A low contrast color makes something that much harder to read. The human brain is a perceptual organ, one that can be easily hacked, whether that hacking is intentional or not.
You can skew votes by throwing all of your dots on one thing
You have three votes. There’s one idea you care deeply about, but it’s not getting any votes. Or, not enough votes to rise to the top. In a brash act of defiance, you cast all your votes on that one idea. In the end, ‘your’ idea comes out on top. With a twinge of guilt, you know that had you cast your votes differently, the results would be quite different.
Conclusion? The best or right idea isn’t necessarily the one that gets the votes. In fact, the idea that gets the most votes is — often, not always — the idea most advocated for (‘squeaky wheel’) or the idea most easily understood (familiarity bias).
While many of the things listed above are tactical, and can be fixed, let’s get to the heart of the matter: How people feel in the end?
Dot Voting Alienates & Isolates:
To be honest, the emotions I’ve felt (and observed) lately in dot voting exercises have not been healthy: Misunderstood. Victorious. Manipulative. Threatened. Marginalized. Defeated. Relieved. Tense. Frustrated. These aren’t good feelings.
I love how Austin Govella articulated this:
I never use [dot voting]. You spend all that time aligning everyone, and then turn around and marginalize a bunch of people.
That idea you were so ardently advocating for, no one voted for it. No one. But you still believe it’ s important. Where does this leave you at the end of it all?
I’ve found the value of dot voting primarily as a discussion tool, not a tool for making decisions. Let me repeat that: The value of dot voting is as a discussion tool, not a tool for making decisions.
Unless you take some time beforehand to prepare a tight structure, and you carefully manage for human biases, I’d be cautious about using dot-voting to make a final call or cull out ideas.
A Final Thought:
Do not take this as an admonition to never again use dot voting. Dot-voting is a great tool, but one that needs to be used with care. My hope here has been to expose problem areas, so we can all get better at using the tool — so that we all benefit from more rewarding outcomes.
While writing this, I felt called out — in a good way — by my friend Kate Rutter. She challenged me to:
“Focus on the helpful/good parts and briefly address the limitations. Kinda sad at all the “wrong” being hyped right now.”
True.
There’s way too much “wrong” being hyped right now. In writing this, it was born out of years of mounting frustrations, but tearing down is not my intent. Rather I’d love see everyone build up, develop, and elevate their own abilities. Dot voting can be a great and useful tool, if used thoughtfully.
