Also called: Public Beta, Private Beta
See also: Working Prototype
Strength of evidence
Relevant metrics: Acquisition, Activation, Cycle time, Customer feedback, Cost
Validates: Feasibility, Viability, Desirability
How: Offer your early-stage functioning product or service in an open (public) or closed (private) beta to interested users. Consider limiting the number of active beta users Ð either to make your product seem more scarce and interesting or to iteratively receive fresh feedback to onboarding flows.
Why: Launching your product before fixing bugs, performance issues, and scalability issues will put your product in the hands of customers earlier and will thus provide feedback on implementation and market demand faster.
This card is part of the Validation Patterns printed card deck
A collection of 60 product experiments that will validate your idea in a matter of days, not months. They are regularly used by product builders at companies like Google, Facebook, Dropbox, and Amazon.Get your deck!
Before the experiment
The first thing to do when planning any kind of test or experiment, is to figure out what you want to test. To make critical assumptions explicit, fill out an experiment sheet as you prepare your test. We created a sample sheet for you to get started. Download the Experiment Sheet.
More than bug hunting
Launching your product in beta, knowingly releasing an unfinished product with known bugs and unknown bugs to users, is a great way to identify and rid its most critical bugs, usability friction, and bad performance before releasing it in its finished version. However, releasing your product in beta will provide a number of other possible benefits.
- Validating problem/solution fit. Do people care about the pains and gains of your problem and does your product address each one in a way they understand? And does that match with the value proposition of your proposed product?
- Raising product awareness. Beta testing can be a marketing strategy – creating initial buzz and momentum among your target audience that can pave the way for better reception when the product is officially relased.
If you are aiming for receiving enough qualitative and constructive feedback to fix your product, a rule of thumb is that 200-300 users will do. However, if your goal is more around creating momentum, creating awareness, and perhaps building an email list of users committed to being interested, you want as mamny as possible.
Common sources for recruiting beta-testers are:
- Beta-testing as a service. Services like BetaList, BetaTesting.com provide Beta testing as a service. If you don’t already have an email list or social precense, these websites help with people willing to test.
- Social Media. Twitter, Reddit, LinkedIn, Facebook, and other social media platforms can be a great place to attract users of a specific target audience. Paying for reach can help you choose the exact demographic you are looking for.
- Startup directories. Early adopters looking to hunting for the next big thing can be found at directories such as BetaBound
- Your personal contacts. Ask friends and family or people in your professional network to test your product.
Depending on whether you are conducting a beta-test to test problem/solution fit or simply to raise awareness about your product, you will want to consider rewarding beta testers for their time and effort to analyze, critize, and make suggestions for changes. Gifting beta testers is also a great way to help build brand loyalty so that the next time you need to test a product, you have plenty of willing testers around.
If you do chose to provide incentives, you will want to only reward active and complete contributions. You might also consider keeping the gifting a surprise till the very end of the test, so that you don’t bias your findings.
Common ways to acknowledge the efforts users took to work hard to improve your product include:
- Gift cards or cash. Real monetary awards that is comparable to the effort put in. The amount will vary in size depending on the target audience.
- Early bird pricing. To keep beta testers as loyal users, several services offer beta testers or early adopters to lock in at a favorable low price for ever, leading users to accept an inferior product for a reasonable price for a longer time.
- Founding members. Create a special page where the name of the early beta-testers will be listed, forever, as an eternal thank you for your efforts.
There are several ways in which you can collect feedback from your beta testers:
- Reporting bugs. Let beta testers spot and eliminate bugs and other unnoticed issues when you’re still in development mode.
- Submitting feature requests. As beta testers use your product, they might encounter things they wished they could do. Allow users to let you know. The closer to the context you can let them tell you, the better (hint: use micro-surveys).
- Discussion forums. Consider opening a closed forum for beta testers to openly provide insights into what they are experiencing. Getting other beta testers to comment on each others findings, can be a great way to both nuance the findings and test whether an issue or request is a recurring phenomenon or not.
- Let the data speak. Make sure to define what beta test sucess and failure looks like before you endeavour into one. What numbers to you expect to see in your metrics, how far into your product do you expect users to go, and how do you expect users to use your features.
After the experiment
To make sure you move forward, it is a good idea to systematically record your the insights you learned and what actions or decisions follow. We created a sample Learning Sheet, that will help you capture insights in the process of turning your product ideas successful. Download the Learning Sheet.
The tools below will help you with the Beta Launch play.
Hotjar is an anlytics service that lets you track user activity, replay their actions, and add micro-surveys that ask about specific behavior.
Product Hunt lists and promotes exciting new products.
BetaList is a community of makers and early adopters showcasing their startups and exchanging feedback.
Beta test with real people in real environments. You get real-world engagement data & feedback to help scale your product.
A directory of beta testing opportunities
Through a 7 month private beta program, HotJar managed to entice 18,000 beta testers (7,000 active) that created over 6000 conversations with the HotJar team, 681 feature suggestions, 231 bugs addressed, and over 1000 paying customers in the first month.