Idea Validation: Market demand, Product

Product-Market Fit Survey

Ask users how they would feel if they could no longer use a product

Illustration of Product-Market Fit Survey
Run a Product-Market Fit Survey play

Also called: Sean Ellis Test, "40% Test"

Difficulty: Easy

Requires existing audience or product

Evidence strength
15

Relevant metrics: Jobs to be done ranking, Ranking needs, wants, desires, pains

Validates: Desirability

How: If over 40% of customers say they would be very disappointed, there is a product/market fit. Allow for open-ended responses to understand why. If you are close to 40%, learn what product or segment tweaks can get you above the line and what solutions customers consider as viable alternatives to their problems.

Why: While a product/market fit survey provides an excellent way to gauge customers' dependency and fondness for an existing product, it does not validate actual product/market fit. If you have to ask whether you have product/market fit, the answer is simple: you don't. Product/market fit surveys can give false positive results Ð especially in the early stages of a product.

This experiment is part of the Validation Patterns printed card deck

A collection of 60 product experiments that will validate your idea in a matter of days, not months. They are regularly used by product builders at companies like Google, Facebook, Dropbox, and Amazon.

Get your deck!

Before the experiment

The first thing to do when planning any kind of test or experiment, is to figure out what you want to test. To make critical assumptions explicit, fill out an experiment sheet as you prepare your test. We created a sample sheet for you to get started. Download the Experiment Sheet.

Scale when it’s time

The timing of when you decide to scale your idea can make or break your business. Scale too early, and users will leave. Scale too later, and a competitor will beat you to the punch.

To help determine when it is time to push ‘GO’, the Product-Market fit survey invented by Sean Ellis, can come in handy.

Ask a simple question…

The product-market fit (PMF) survey asks the customer one very specific question:

How would you feel if you could no longer use [this product]?”

Let participants answer with one of these mulitple choice responses:

  1. Very disappointed
  2. Somewhat disappointed
  3. Not disappointed (it isn’t really that useful)
  4. N/A – I no longer use [product]

By comparing nearly 100 startups, Sean Ellis, who came up with the survey format, found that if over 40% of users responded that they would be “Very dissapointed” to stop using the product, there’s a great chance that the solution had found its Product-Market fit. He found that those companies that scored below 40% all struggled to reach traction.

And follow up…

Receiving a PMF score is a great metric to gauge your progress on your road to scale. However, if you don’t follow up with more qualitative questions or more experiments, it can be hard to understand exactly what impacted your score to move from one plateau to the other.

Typical follow-up questionns

The followinng can serve as inspiration to what kind of questions you might ask your customres. As with all research, if you can find this evidence looking at real behavioral data you will want to do that. Real evidence, observing what people do, will always trump opinions and what people say. You build better products by asking better questions.

  • Please help us understad why you selected this answer. (Open ended question)
  • Have you recommended this product to anyone? (No, Yes: ___)
  • What type of person do you think would benefit most from the product? (Open ended question)
  • How can we improve the product to better meet your needs? (Open ended question)
  • How often do you use our product? (Never, Once a year, A couple of times a year, Once a month, A couple of times a month, Once a week, Every day)
  • How would you feel if you could no longer use our product? (Very disappointed, Somewhat disappointed, No reaction / neutral)
  • Have you ever recommended us to friends or family? (Yes / No)
  • What do you think sets us apart from our competitors? (Open ended question)
  • What could we improve on? (E.g. Customer Support, Reliability of Service, Design, Functionality, Other)
  • Could you tell us a bit about yourself? What’s your employment status? (Open ended question)
  • How did you discover/come across this product? (Blog, Google/Search, Twitter, LinkedIn, Facebook, Word of mouth, other)
  • What would you likely use as an alternative if this product were no longer available? (I probably wouldn’nt use an alternative, I would use an alternative: ___)
  • Would it be okay if we followed up by email to request a clarification of one or more of your responses? (Yes - enter email / No)

How does Product-Market Fit look like?

“If you have to ask whether you have Product/Market Fit, the answer is simple: you don’t.” – Eric Ries

You ultimately want to put yourself in a position where you can observe real user behavior that proves that you have indeed found Product-Market fit.

How many responses do I need?

Buffer conducted a study using the PMF survey and found that they only needed 40-50 respones for the results to carry significance. The critical factor is ensuring a diverse range of respondents and understanding their characteristics. It is essential to avoid surveying customers who are not invested in your product.

To prevent skewed results, it is imperative to collect feedback from users who are actively engaged with your product or company, possess a basic understanding of its core features, and have used your product within the two weeks preceding the survey. This approach guarantees fresh and relevant insight.

It is worth noting that the “Sean Ellis Product Market Fit Survey”, also referred to as the “40% Test,” calls for a 40% response rate after distributing the survey to the designated customer group.

Sean Ellis recommends to survey a crowds with the following characteristics:

  • People that have experienced the core of your product offering
  • People that have used your product at least twice
  • People that have used your product in the last two weeks

When should I send Product Market Fit surveys?

While identifying the appropriate individuals to survey is important, determining the best time to send out the survey is equally crucial. Should the survey be sent during the product development phase? Following a sales pitch? Or, perhaps, during the early hours of the morning?

Product Market Fit during product development

The answer to this question varies depending on the stage of your business and the specific insights you hope to gain. The following product deelopment phases can especially benefit from receiving feedback from Product Market Fit surveys.

  • Slow Business Growth. If your business is experiencing slow growth, conducting a survey can provide valuable insights into the interests and needs of your target market.
  • Scaling and Fundraising. Before acquiring funding to scale your business, it is essential to gather insights into your customers’ interests and needs.
  • Product or Feature Changes. If your product has undergone a significant change or a new feature has been introduced, surveying customers can provide insight into how they perceive and interact with the new offering.
  • A/B Testing and Ad-hoc Surveys. Conducting ad-hoc surveys during the A/B testing phase or for a momentary marketing change can provide maximum output and valuable insights.

Ongoing Product market fit surveys

As you progress on your journey toward creating a successful product, the Product Market fit survey result goes from being a key result you are chasing to being another health metric or KPI that you are continuously monitoring. Just like the Net Promoter Score (NPS). In this case, you want to contiously ask a sample of customers or users from various segments and in various stages of your product life-cycle. Try to avoid asking the same person twice in in the middle of the lifecycle, but insist on for instance including the survey question in your exit-survey.

This will allow you to monitor how customer sentiment changes over time as users progress from being just onboarded, to being active, to existing.

Product Market Fit Surveys grows in importance as your product grows

As a product grows, it can be challenging to determine whether new features or offerings are truly valuable to customers. It’s crucial to maintain a laser focus on key metrics, even after achieving product market fit.

To avoid getting carried away by positive buzz, strive to separate marketing and product launch as much as possible, keeping new features or products in beta until they are confident they provide value to customers.

This is where the Product Market Fit survey becomes especially valuable. By conducting surveys before the general public is aware of a new feature or product, you can gather valuable feedback and determine the feature or product’s effectiveness.

After the experiment

To make sure you move forward, it is a good idea to systematically record your the insights you learned and what actions or decisions follow. We created a sample Learning Sheet, that will help you capture insights in the process of turning your product ideas successful. Download the Learning Sheet.

Popular tools

The tools below will help you with the Product-Market Fit Survey play.

  • PMF Survey

    A free tool by Sean Ellis to measure Product-Market fit as defined by the man himself.

Examples

Survey on Slack

An open research project with the mission of discovering why Slack is so addictive found that it not only hard Product-Market fit, but also why it worked annd what could be improved.

Source: 731 Slack Users Reveal Why It’s So Addictive

Buffer

To test product-market fit, Buffer surveyed their most engaged users with Sean Ellis’s PMF survey. They focused on understanding user reliance by asking how disappointed they would be if unable to use the product. With responses from about 40-50 users, Buffer gained valuable insights into the significance and impact of their social media tools.

Source: A Simple Guide to Measuring the Product-Market Fit of Your Product or Feature

Superhuman

Superhuman’s PMF survey asked users about their potential disappointment if they couldn’t use the service anymore. This strategy pinpointed necessary improvements and customer segments, guiding product refinement.

Source: Building a Superhuman growth funnel to find product-market fit

This experiment is part of the Validation Patterns printed card deck

A collection of 60 product experiments that will validate your idea in a matter of days, not months. They are regularly used by product builders at companies like Google, Facebook, Dropbox, and Amazon.

Get your deck!

Want to learn more?

Receive a hand picked list of the best reads on building products that matter every week. Curated by Anders Toxboe. Published every Tuesday.

No spam! Unsubscribe with a single click at any time.

Ice Breakers

Relieve initial group awkwardness and establish a safe space

Educate

Broaden knowledge or insight regarding the behavior or situation to inform decisions.

Demonstrate

Show practical examples or models of the desired behavior for clear guidance.

Alert

Highlight current actions and their reasons, bringing unconscious habits to awareness.

Train

Develop necessary skills and competencies to enable effective action.

Community events
Product Loop

Product Loop provides an opportunity for Product professionals and their peers to exchange ideas and experiences about Product Design, Development and Management, Business Modelling, Metrics, User Experience and all the other things that get us excited.

Join our community

Made with in Copenhagen, Denmark

Want to learn more about about good product development, then browse our product playbooks.