Learn how the Iowa Caucus incident highlights the importance of good product research and development practices. Discover valuable lessons for any industry.
Table of Contents:
- Did a Mobile App Break the Iowa Caucus?
- Do We Have to Develop a Mobile App?
- Product Research and Load Testing Avoids Disasters
- Piloting Pre-Development For Safety
- Research and Development Budget Matters
- Product Development Takeaways
Did a Mobile App Break the Iowa Caucus?
Well, no, not exactly. Bad product development practices did. Maybe, or maybe not, this could be true. According to some, the sky is falling, and according to others, it's just taking a minute to double-check results for consistency, and everything else is fine; but is everything actually fine?
Either way, this incident with the Iowa Caucus mobile app is a lesson in product development and in user experience design.
Why Use Mobile App Development?
Politics laid aside, the DNC wanted to develop a mobile app to do their Caucus voting because it should be quicker and more trustworthy than the 2016 method of literally flipping a coin to allocate a delegate's vote because they couldn't be located. The problem statement was pretty clear. It was definitely a bad process in need of a solution.
Going straight to "We should make an app!" is super logical today. Nearly anything I can think of, I can find an app for. But, good product research and development would have, should have, asked: Does it have to be a mobile app?
Do We Have to Develop a Mobile App?
Well, no. It's secure, which is good. But, it's hard to update if anything goes wrong. Plus, you have to make multiple versions for different phones. You also have to support "Phil" from up the road who doesn't have a smartphone, so now you have to buy it for him, get him a data plan, or install Wi-Fi in his house.
Poor Research Is Costly
The complexities of a mobile app add up. Good product design and research would have focused on the users and the very real truth that a web app would have been fine. A web app would be easier to update if anything went wrong, and Phil could just pop down in the public library to do it. Challenge every assumption about what we think we need, and then design digital solutions that get out of the way and let us accomplish the goal.
Product Research and Load Testing Avoids Disasters
It's not hard to do deep product research or load testing. It's costly, but it's not hard, it just depends on how much you care about your customers. When your entire digital software product is designed to work for a ton of people all at once, but then never to be used again, you absolutely have to perform load testing. There's no way around it. I'm not saying they didn't do it, because I don't know the answer to that. However, if they did, then either the results weren't acted upon, or the remediation wasn't also tested.
We've Been There, Done That
This is common, we've seen it at Anthroware multiple times. When you test a product for scale (or for security), research shows you have to do it multiple times. The amount of times depends on a variety of factors. You test digital product performance over and over because every time you fix one performance problem, you place the remaining load on some other part of the software, which may or may not be up to the challenge.
For example: Say you made a hypothetical app for, I don't know, voting. The basic system would be to present several choices to the user. The user taps one, the phone prompts them to confirm: "You selected Jason's World-Famous Apple Pie as Best-In-Show. Is this correct?" The app hasn't sent any data anywhere yet. All of the computation is on the phone itself. So far so good (phones usually perform pretty well, even Phil's loaner phone). Then, the user confirms the prompt. Once the user confirms it, the data is sent off to a cloud server, data is moved around, stored, aggregated, etc.
Unseen Development Errors
Performance problems could happen anywhere in the process of the mobile app executing the users' commands. Maybe there's only 3G connectivity and it's a foggy day. If you alleviate the 3G problem, you make more requests happen at once. So, now maybe the API can't handle that many requests. So, maybe you enable Elastic Growth (thank you AWS, we love you for this). Now you can handle the requests, but can your reporting aggregations keep up? See, eliminating product performance bottlenecks just moves the problem down the line until finally, you eliminate enough problems to make the product viable. Having a solid product strategy ahead of time is critical.
Piloting Pre-Development For Safety
Something as important as a federal election mobile app probably needs a pilot program, at the same scale as the real thing. Something with low stakes, nationwide, so you know the limits. I would have happily participated in a "Vote for Your Favorite Sandwich" pilot if it meant ensuring a smooth rollout at the Iowa Caucus. There are always kinks in a new product post-development. Always. I have never seen a rollout that didn't have at least one kink; I thought I did once, but it turned out I needed to wait like 30 seconds longer. Would you start a business without a solid business-level strategy? I doubt it.
Digitizing a Complex Process Does Not Ensure Success
This is the biggest lesson here: Digitizing a complex process does not ensure success. Period.
The Caucus system is old and made sense once. However, reading about it today to ensure I understood it took more than 6 paragraphs to explain, and I still don't really get it. It is a complex system that made sense when it was instituted. Software development for an app that simply replicates the system doesn't fix the problem, it makes it worse. It adds authentication, logging, support, phones, wireless carriers, SSLs, support staff, demands for immediate results, reporting solutions and costs, etc. It's complexity on top of complexity. Good product research and development should have caught this and demanded a change in the process itself, resulting in a seamless product design with a great user experience.
Digital transformation doesn't mean "do everything digital". It means that we challenge every assumption about what we think we need, and then design digital solutions that get out of the way and let us accomplish the goal.
Research and Development Budget Matters
According to The Wall Street Journal, this Iowa Caucus mobile app cost $63,000. Christopher Lee for The Wall Street Journal. “Iowa's Tally-by-App Experiment Fails.” The Wall Street Journal, Dow Jones & Company, 5 Feb. 2020, https://www.wsj.com/articles/iowa-caucus-results-delayed-by-apparent-app-issue-11580801699.
That's a really low budget, and it would be surprising if adequate load testing or QA happened; the tools and time simply cost too much. Rigorous testing couldn't be done. As the old adage goes, you get what you pay for. Adding to this, Nevada paid $58,000 for a similar system. Don't be surprised if we see similar results. Not saying it's certain, but let's not be surprised.
Understanding Your End-Users
Real product development thinkers understand that 'user experience' encompasses the entire system, not just the buttons that you tap. Reading about this "app failure" is really interesting. Some of the biggest failures here are, not understanding the low-tech audience, not training people to be able to easily download and use the app, and trouble logging in. Tech issues aside that's a frustrating list of misses that could have been solved with proper product research.
Product Development Takeaways
How is all this a lesson in product research and development? Well, each of these points above is basically critical to the end-user and design process. Does the problem really exist? Does the solution match the problem, or is there a better one? Does it need to pilot? How will this product get rolled out? What are the consequences of the app failing? Are we making it as simple as possible for our users, or are we simply passing on the complexity? Is this software going to hit the mark with our users? Have we made any baseless assumptions about our users?
The Evidence is Clear
Based on what I'm seeing in the news, one or more of these critical questions got missed. And that's really unfortunate when the stakes are so high. The process and approach to product research and development truly matter.