Also called: Sean Ellis test
See also: Closed-Ended Surveys, Net Promoter Score (NPS)
Difficulty: Easy
Requires existing audience or product
Evidence strength
Relevant metrics: Jobs to be done ranking, Ranking needs, wants, desires, pains
Validates: Desirability
How: If over 40% of customers say they would be very disappointed, there is a product/market fit. Allow for open-ended responses to understand why. If you are close to 40%, learn what product or segment tweaks can get you above the line and what solutions customers consider as viable alternatives to their problems.
Why: While a product/market fit survey provides an excellent way to gauge customers' dependency and fondness for an existing product, it does not validate actual product/market fit. If you have to ask whether you have product/market fit, the answer is simple: you don't. Product/market fit surveys can give false positive results Ð especially in the early stages of a product.
This card is part of the Validation Patterns printed card deck
A collection of 60 product experiments that will validate your idea in a matter of days, not months. They are regularly used by product builders at companies like Google, Facebook, Dropbox, and Amazon.
Get your deck!Before the experiment
The first thing to do when planning any kind of test or experiment, is to figure out what you want to test. To make critical assumptions explicit, fill out an experiment sheet as you prepare your test. We created a sample sheet for you to get started. Download the Experiment Sheet.
Scale when it’s time
The timing of when you decide to scale your idea can make or break your business. Scale too early, and users will leave. Scale too later, and a competitor will beat you to the punch.
To help determine when it is time to push ‘GO’, the Product-Market fit survey invented by Sean Ellis, can come in handy.
Ask a simple question…
The product-market fit (PMF) survey asks the customer one very specific question:
How would you feel if you could no longer use [this product]?”
Let participants answer with one of these mulitple choice responses:
- Very disappointed
- Somewhat disappointed
- Not disappointed (it isn’t really that useful)
- N/A – I no longer use [product]
By comparing nearly 100 startups, Sean Ellis, who came up with the survey format, found that if over 40% of users responded that they would be “Very dissapointed” to stop using the product, there’s a great chance that the solution had found its Product-Market fit. He found that those companies that scored below 40% all struggled to reach traction.
And follow up…
Receiving a PMF score is a great metric to gauge your progress on your road to scale. However, if you don’t follow up with more qualitative questions or more experiments, it can be hard to understand exactly what impacted your score to move from one plateau to the other.
“If you have to ask whether you have Product/Market Fit, the answer is simple: you don’t.” – Eric Ries
You ultimately want to put yourself in a position where you can observe real user behavior that proves that you have indeed found Product-Market fit.
How many responses do I need?
Buffer conducted a study using the PMF survey and found that they only needed 40-50 respones for the results to carry significance.
Sean Ellis recommends to survey a crows with the following characteristics:
- People that have experienced the core of your product offering
- People that have used your product at least twice
- People that have used your product in the last two weeks
After the experiment
To make sure you move forward, it is a good idea to systematically record your the insights you learned and what actions or decisions follow. We created a sample Learning Sheet, that will help you capture insights in the process of turning your product ideas successful. Download the Learning Sheet.
Popular tools
The tools below will help you with the Product-Market Fit Survey play.
-
PMF Survey
A free tool by Sean Ellis to measure Product-Market fit as defined by the man himself.
Examples
Slack
Slack issued an open research project for defining its own Product/Market fit
Buffer
Buffer determined that 40-50 responses should carry statistical significance
Source: A Simple Guide to Measuring the Product-Market Fit of Your Product or Feature
Related plays
- How to find Product/Market fit by Ryan Law
- False Positives and Product Market Fit by Tristan Kromer
- Product/Market Fit Survey by Tristan Kromer
- Part 4: The only thing that matters by Marc Andreessen
- A Simple Guide to Measuring the Product-Market Fit of Your Product or Feature by Leo Widrich
- The Real Startup book - Product/Market Fit Survey by Tristan Kromer, et. al.