Evidence strength
Relevant metrics: Conversion
Validates: Desirability
How: Make potential customers go extra lengths to get something done. The higher the hurdle, the more validity you can attribute to your results. Create a long survey, require an approval process, or create a similar hurdle that only users with a real interest will live through.
Why: Having users merely browse pages on a website does not validate actual interest in a product. Instead make users 'pay' by spending time and energy to go through a more cumbersome process to increase evidence strength and make the validation results more significant.
This experiment is part of the Validation Patterns printed card deck
A collection of 60 product experiments that will validate your idea in a matter of days, not months. They are regularly used by product builders at companies like Google, Facebook, Dropbox, and Amazon.
Get your deck!Before the experiment
The first thing to do when planning any kind of test or experiment, is to figure out what you want to test. To make critical assumptions explicit, fill out an experiment sheet as you prepare your test. We created a sample sheet for you to get started. Download the Experiment Sheet.
Make things hard to test gauge keen interest
Have the user go through a set of activities containing abnormal amounts of usability friction to gauge the customer’s desire for a particular solution. If they are willing to go through the extra high hurdle you put in front of them, chances are greater that they will also be willing to pay for it later.
If you are putting on an event, organize in a remote location with lots of travel time for attendees. Or gie them a lengthy online survey.
An added benefit of adding a lengthy survey to your sign up process, is that you can pre-qualify sign ups to ensure that users signing up are in fact exactly the right kind of customers for the product.
After the experiment
To make sure you move forward, it is a good idea to systematically record your the insights you learned and what actions or decisions follow. We created a sample Learning Sheet, that will help you capture insights in the process of turning your product ideas successful. Download the Learning Sheet.
Examples
Style profiles at Stitchfix
When users sign up for Stitchfix, a “clothing as a subscription”, they are asked to go through a lengthy sign up form that takes around 1 hour to fill out. Even though this seems as an uncomparable high hurdle to put in front of users, it serves another purpose: to reassure that the service provided is a serious one that needs elaborate data to work. This move plays on the Endowment Effect – the fact that we as humans have a tendency to place higher value on products we have invested- and feel ownership in. Ownership creates satisfaction.
Source: Stitchfix.com
Sign-up process
Design a sign-up process with multiple answers that are difficult to answer. This will increase friction to toward completing the sign up process and in turn constuct hurdle for customers to sign up that will ensure that only the most passionate customers are captured for your early testing. The purpose of this trick is alone to ensure higher evidence strength when conducting experiments. You will most likely not be interested in keeping the high hurdle in the product as you launch.
Financial Times
The FT conducted a high hurdle experiment by adding delays to their site for different user groups, to assess the impact on engagement. This approach helped gauge the true value users place on website speed, linking slower load times to decreased engagement and revenue.
Source: A faster FT.com
This experiment is part of the Validation Patterns printed card deck
A collection of 60 product experiments that will validate your idea in a matter of days, not months. They are regularly used by product builders at companies like Google, Facebook, Dropbox, and Amazon.
Get your deck!Related plays
- Customer Development Experiment: The High Hurdle by jminton
- High bar Smoke Test by Tristan Kromer
- The Real Startup book - High bar smoke test by Tristan Kromer, et. al.
- Experiments where you make your product worse – the most underrated product manager tool by Oleg Ya
