Evidence strength
Relevant metrics: Jobs to be done ranking, Ranking needs, wants, desires, pains, Net promoter score, Sean Ellis Score
Validates: Viability, Desirability
How: Ask closed-ended questions validating previous suggestions from qualitative research. Avoid ambiguity and false positive results by asking questions about existing customer behavior rather than about opinions and preferences.
Why: They are easy to use, but also potentially misleading. Be careful. Surveys are faceless, without empathy, and assume knowledge of the right question to ask. Plus, you can't ever understand the full context of a written response.
This experiment is part of the Validation Patterns printed card deck
A collection of 60 product experiments that will validate your idea in a matter of days, not months. They are regularly used by product builders at companies like Google, Facebook, Dropbox, and Amazon.
Get your deck!Before the experiment
The first thing to do when planning any kind of test or experiment, is to figure out what you want to test. To make critical assumptions explicit, fill out an experiment sheet as you prepare your test. We created a sample sheet for you to get started. Download the Experiment Sheet.
Be wary of false positives: don’t ask for opinions
Asking for opinions, you get opinions. People have opinions on everything, and it doesn’t necessarily reflect what people are actually doing in the real world. Whenever you hear yourself asking question that begin with “Would you…” or “Do you think…”, you will want to stop youself.
It’s better to ask for facts. Questions like “When was the last time you googled…” shows whether potential customers thought about it before you triggered them. If you ask “Would you” or “Do you think”, those same people will be very likely to answer positively. “When was the last time…?” refers to a concrete instance. You want to go for the facts and the instances when you ask.
Don’t force your own perspective
You do not want to focus on the solution because you will be framing the conversation around what you think is important. Instead, you want to go for uncovering the pains and gains of the potential customer.
Instead, you want to first go for validating the problem. Go for situations - concrete instances in the past life of the potential customer you’re asking. “Tell me the last time you thought about…” is a great starter. After that, you can ask follow up questions like “Why did it bother you?”.
After the experiment
To make sure you move forward, it is a good idea to systematically record your the insights you learned and what actions or decisions follow. We created a sample Learning Sheet, that will help you capture insights in the process of turning your product ideas successful. Download the Learning Sheet.
Popular tools
The tools below will help you with the Closed-Ended Surveys play.
-
Typeform
Survey tool focused on lowing friction
-
SurveyMonkey
Popular free survey tool
-
Survey.io
Specialized Product-Market Fit survey tool
-
Qualaroo
Context-aware micro-surveys
-
HotJar
Heat-maps, microsurveys, etc. in one package
Related plays
- What Type of Lean Startup Experiment Should I Run by Tristan Kromer