Relevant metrics: Conversion
How: Write out your value proposition in 1-3 sentences. Show it to participants for just enough time to read it, then take it away. Finally, ask participants to explain the value proposition back to you in their own words. If the explanation is roughly comparable to your own, count it as a positive result. If 20 people participate, you want 80% to understand your message.
Why: Evaluate whether customers understand a marketing message explaining the value proposition to eliminate a possible false negative bias in later tests where customers indicate they do not want the value proposition when they actually do not understand it. As an example, you will want to run a comprehension test before a spoof landing page test or you won't understand why it doesn't work.
This experiment is part of the Validation Patterns printed card deck
A collection of 60 product experiments that will validate your idea in a matter of days, not months. They are regularly used by product builders at companies like Google, Facebook, Dropbox, and Amazon.Get your deck!
Before the experiment
The first thing to do when planning any kind of test or experiment, is to figure out what you want to test. To make critical assumptions explicit, fill out an experiment sheet as you prepare your test. We created a sample sheet for you to get started. Download the Experiment Sheet.
How to proceed
Before you run a smoke test (that tests commitment), you should run a comprehension test.
You can run the comprehension test manually, showing a piece of paper to the participant for them to read it – and then taking it away to explain what they just read back to you in their own words.
If the participant’s explanation roughly compares to your own, you can count it as a positive result. Otherwise, it’s negative. Aim for a sample size of at least 20 participants and a positive success rate of around 80%. Eighty percent intuitively seems very high - but it must be as regardless of what your value proposition is, your target customers should still understand it.
As a digital alternative to the manual Comprehension Test, the Five Second Test is commonly used.
Who to recruit
Depending on your target market, who you bring to a comprehension test is important.
If your target market has a high school level education, you just need someone with the same basic level of education and vocabulary. At this point, you are not testing to see if someone wants your value proposition, but if they can understand it.
If your value proposition uses industry jargon, you will want to be more critical of who you recruit as test participants. You want to make your value proposition understandable to those who aren’t already converted, so if the customers you want to sell to do not understand your industry specific words, you need to change your value proposition.
Once your target audience understands your value proposition (conversion of at least 80%), you can move on to more sophisticated and (maybe more) expensive tests to probe real customer behavior.
After the experiment
To make sure you move forward, it is a good idea to systematically record your the insights you learned and what actions or decisions follow. We created a sample Learning Sheet, that will help you capture insights in the process of turning your product ideas successful. Download the Learning Sheet.