Archive | January, 2017

Dot Voting – How PMs Can Avoid Wasting Time on Ideas Everyone Hates

24 Jan

You’re leading a brainstorming and prioritization session, and there are a dozen or more ideas on the white board. You could ask someone to pitch each idea, you could encourage debate of each idea’s merits, or you could quickly eliminate most of the ideas with a single step, and focus the discussion: ask people to vote on their top 3 favorite ideas.

Dot Voting helps me focus everyone’s time on the ideas that have the most promise, by simply forcing people to narrow in on the ideas they’re most passionate about. Also sometimes called Dotmocracy or Multi-voting, dot voting is a technique I learned from Intuit’s founder Scott Cook while I was a PM leader at TurboTax. It’s been around since at least the 2000s, and is fairly well documented as a meeting facilitation technique.

Dot Voting at Couchsurfing.jpg

As a Product Manager these are the reasons I’ve come to love dot voting:

Helps get buy-in on priority. When you give an idea in a brainstorm and you never hear about it again afterwards, it can be kind of frustrating. Was your idea ignored? It can be especially maddening if, later, your great idea turns out to be extra great because the situation has changed. If, however, you collect Dot Votes at the end of a meeting, and you see that one of your ideas didn’t get a single vote (especially if you didn’t use a precious vote on it), you can understand why it doesn’t get picked.

Saves time. In prioritization sessions, it sometimes feels like you need to really speak up for your favorite ideas and defend them in front of the group. As the meeting facilitator, you can save a lot of time by pre-empting these monologues with a vote.

Focuses energy on a small set of ideas. Allows you to focus any further discussion with the group around the ideas that have the consensus of the group, and allows you to focus your follow-up research on a smaller set of ideas.

Dot voting isn’t, of course, the perfect tool for every situation. If the ideas you’re considering are new or are difficult to understand on the surface, or if the participants lack context on the problem or customer, it might be the right time to use it.

As a facilitator, there are a few things to watch out for: you can speak up during the meeting to make sure newer ideas are understood, you can use your own votes to help make sure some ideas are considered, and you can make sure the team knows which ideas have customer insights or data to back them. It’s also important to remember that businesses aren’t democracies – sometimes leaders will need to pursue a different direction than the team would favor. Even in these cases, however, it’s helpful to know if you’re fighting an uphill battle or if the team already has your back.

Taking Dot Voting Further: I’ve experimented with using a live Google Spreadsheet to take the Dot Voting concept further, and with each person giving a rank vote for their favorite few ideas (e.g. 1 is best, 5 is worst) in a single session. The collaborative nature of Google Sheets, and easy tabulation and rank-ordering of ideas at the end are appealing. Within a few minutes of starting, you have the beginnings of a priority-ranked set of initiatives.

A few things I like less about it however: everyone’s focused on their laptops instead of their colleagues or the board, it’s very tempting to make live edits to your votes to try and influence the final scores (“oh, idea X isn’t doing well, I’ll bump up my vote on that one because idea Y looks to be solidly in the lead”), and the rank order can give an overly scientific feeling to the outcome.

Do you have experience with Dot Voting? Any tips to share? What other techniques have you tried?

Another “Concierge MVP” Example, from TurboTax

17 Jan

I recently wrote about how Concierge MVPs work, and shared an example from my startup, GoodApril.

TL;DR: Concierge MVPs help prove out a new software product by offering customers a service, and providing it, without building out all the software to deliver it in an automated way.

The GoodApril Tax Checkup was actually a form of “Wizard of Oz” MVP, where the user didn’t know that my Co-Founder and I, not our software, were doing some of the work.

For this second example, I share an example of how a Concierge MVP can work even at a mature software product like TurboTax. This example is of a true Concierge MVP, where the user knows they aren’t interacting with software.

Example of Concierge MVP: The TurboTax Health AnswerXchange

After GoodApril was acquired, my Co-Founder and I joined the Intuit TurboTax team, responsible for the company’s response to the Affordable Care Act (ACA). Investment analysts had downgraded Intuit’s stock because of the risk that consumers would flee Do-it-Yourself (DIY) tax software in the face of this complex tax law change, and rather prefer the “expertise” of tax stores and accountants. As leaders of this team, however, we weren’t sure what about the law was most vexing to consumers, and how we could best alleviate their concerns and keep them as TurboTax customers. To find out, we built a way to gather data quickly: a question and answer forum on what would become TurboTax Health.

Starting with a Concierge MVP is a great way to uncover customer needs without building tons of product up front. You offer to help customers with their problem in a manual (and likely unscalable) way, and in doing so, gain much deeper understanding of their specific needs and what kinds of solutions might work best at scale.

For ACA, our idea was pretty simple: Let’s just ask TurboTax users to post their questions about Obamacare, and we’ll offer to answer them. We could then analyze the most frequent questions, and build tools or content to help give them confidence that TurboTax had them covered for ACA.

Our plan required two things: users with actual questions, and someone to answer them.

Finding customers was easy enough: We created a branch off the TurboTax homepage asking users if they had questions about how Obamacare would affect them in the year ahead, and directed them to our new Question-and-Answer product for healthcare, which we called the AnswerXchange.

Giving customers accurate answers at scale was a somewhat tougher challenge, but luckily we had a major asset on our side: an existing question-and-answer forum called the TurboTax Live Community. The community was built to help answer in-product questions for users. While some Intuit employees answered questions in the community, most of the answerers were actually just regular people who enjoyed helping others with tax questions.

Since we anticipated we might get a large volume of questions, and some folks at Intuit had already built a basic calculator that helped customers determine if they would face a penalty under the law, or be eligible for a subsidy, we incorporated it into the AnswerXchange. First, they could step through that calculator, then they were prompted to browse the existing questions or ask their own:


Gathering a deeper understanding of customer needs by answering their questions

The volume of questions that started to roll in was pretty overwhelming, and it turned out our community experts couldn’t totally keep up, so our whole team – product people, engineers, designers and marketers alike – began researching the answers for customers’ questions and posting them in the community.

Within just a couple of weeks, we had amazing results. Here are a few of the ways the Concierge MVP benefited us:

  • We were able to develop a basic solution to our customer’s problems before any competitor did, helping us establish credibility and generate early press mentions on our expertise on the ACA.
  • We learned the most common kinds of questions people had about the ACA: how much subsidy they could get, whether their insurance qualified, and how they could enroll.
  • We also learned that even for basic questions like these, the answers could be surprisingly complex based on small nuances of a family’s situation. For instance, what if your kids were covered under free government insurance (CHIP), but you as parents weren’t?
  • Our entire team developed a much deeper understanding of the law, its many complexities, and the impact it was going to have on our customers

With what we learned, we were able to roll out a much more robust solution, called TurboTax Health, to serve customers throughout the 2013 tax year. It incorporated a step-by-step guide to understanding the subsidy, what kinds of insurance would avoid the penalties, and even connected you to an online health insurance marketplace partner to purchase insurance if you needed it.


TurboTax Health was built with the knowledge developed in our Concierge MVP

The “Concierge MVP” – An Example: GoodApril’s Tax Checkup

8 Jan

Do you want to know if customers really want what you’re building? Try selling it to them before you’ve built it, and you’ll know for sure. That’s the idea behind the “Concierge MVP” or “Wizard of Oz MVP.” Unlike a simple “Click Test,” however, you actually deliver the value you’re promising to your customer, just not in the way you intend to when you build your full-scale software solution.

It’s one of the most powerful ways I’ve learned to test a product hypothesis and gain real customer insights quickly. I’ve used it in startups, like GoodApril, and even at a big company like TurboTax.

I’ll share two specific examples to help you learn how to create your own Concierge MVP test. You can also check out other examples of MVPs.

Background: What is a Concierge MVP?

An MVP is a “Minimum Viable Product”. It’s a fundamental concept in Product Management and software development. Build only the bare essential elements of a solution, otherwise you’re likely wasting development effort or worse, making your product harder to use by overloading it with functionality. There are two flavors of Concierge MVP:

Example of Wizard of Oz MVP: The GoodApril Tax Checkup

Wouldn’t it be amazing if your tax software would tell you if you could be paying less in taxes, or warn you if you were going to owe the IRS next April? My Co-Founder and I were pretty sure it was possible to build, and that people would pay for it if we could, but before we tried to raise investment in our startup, GoodApril, we needed to be sure. So we built a test to find out: The Tax Checkup.

The thesis behind the Tax Checkup was that if people actually cared about finding ways to pay less in taxes, they would be willing to pay to find out how. Rather than actually charge users, our currency would be asking them to provide us with a copy of their most sensitive financial document: their tax return. Not charging money also allowed us to avoid building any payments tools, a big time savings since we were just a two-man startup at the time.

Here’s how it worked: We generated some traffic to our site using traditional Growth Hacking approaches. One example was that I would find tax questions people asked on Quora, and link to our website in my answer as a way to find other savings tips. Our website:


Testing the value we would deliver: save money on your taxes

We would then lead them through a short online process of creating an account (so they could login and securely see their report later), and then uploading their tax return. To get users to create an account, we provided a quick demonstration the kind of analysis we could provide using three simple questions.


Providing value before asking for an email address or tax return

After someone had created an account, we asked for their tax return. While we did say that it could take up to 48 hours to get your report back, we never implied GoodApril wasn’t a fully software-based solution.


Asking for a user’s tax return

And that’s where the software stopped, and the people began.

Since we wanted to make sure it would actually be possible to build repeatable tax advice with software, we went ahead and built the logic and UI around a few savings opportunities for the Tax Checkup. We didn’t, however, invest in building the OCR technology that would be necessary to actually extract the data from the user’s tax return, to generate their report, or any kind of transactional email system notifying people when their reports were ready. That’s where the “Concierge” part really came in: my co-founder and I would read the necessary input values from the customer’s tax return, manually input it into our internal analysis tool, publish the report, and then email them that the report was ready. The user could then login and see the report we generated for them:

GoodApril Tax Checkup - Cropped.jpg

The GoodApril Tax Checkup – Example for a family earning $306K

Each report took us almost 10 minutes to generate, and that was only after we got into the rhythm of it. We were OK with that, however, because spending a few hours of manual work allowed us to launch many weeks earlier than we could have otherwise.

We ran the Tax Checkup for a few weeks, and received hundreds of tax returns. We shut it down once we felt we had the evidence we needed to prove to ourselves and to investors that there was real demand for the kind of service GoodApril would ultimately offer.

So, a quick summary of why this Concierge MVP test was so helpful for us:

  • It helped us prove demand for tax advice without building a fully functional site
  • It allowed us to test demand much faster than we could have otherwise
  • We gathered evidence of customer demand using real customer behavior (sign ups and uploads of tax returns), not just self-reported interest from a survey or interviews
  • We learned some of the challenges in building the ultimate solution we aimed to provide: for instance, how a tax return alone didn’t give us as much information as we needed to give many kinds of advice
  • We learned what it would take to acquire customers. Our growth hacking helped us drive a few thousand visitors to our site, and we got to start testing approaches to see what would work for us down the road

Do you have examples of how you’ve built a Concierge MVP to test a new software offering before launch? I’d love to learn about it!