Evidence strength
Relevant metrics: Acquisition, Activation, Customer feedback, Cost
Validates: Feasibility, Viability, Desirability
How: Provide the core features of your intended final product, but limit its scope (and scale) to support only a small subset of the final market.
Why: Costs associated with a product are often not in developing its basic functionality, but in scaling it to support and make it useful for a large number of users.
This experiment is part of the Validation Patterns printed card deck
A collection of 60 product experiments that will validate your idea in a matter of days, not months. They are regularly used by product builders at companies like Google, Facebook, Dropbox, and Amazon.
Get your deck!Before the experiment
The first thing to do when planning any kind of test or experiment, is to figure out what you want to test. To make critical assumptions explicit, fill out an experiment sheet as you prepare your test. We created a sample sheet for you to get started. Download the Experiment Sheet.
Test with a small and carefully selected segment
This approach can be combined with most other experiments. It is testing a product or feature with a small, carefully selected segment of the target market before committing to a global launch. The underlying principle of the Provincial experiment is to gather insightful data and feedback from this limited rollout, which can significantly inform and guide the subsequent broader market introduction.
By focusing on a smaller subset of the target audience, product teams can gain valuable insights into how the product is received, understand user behavior, and identify any potential issues or areas for improvement. This method allows for a more controlled environment where the impact of new features or products can be closely observed without the complexities and variables that come with a full-scale launch.
The chosen sample should be representative of the broader market to ensure that the findings are relevant and scalable. This could mean selecting a demographic slice, a particular geographic region, or a segment based on user behavior or other relevant criteria. The key is to ensure that this group’s response to the product can reasonably predict the larger market’s reaction.
This type of experiment acts as a preliminary gauge of market demand and user acceptance, providing a safer platform for testing hypotheses about how the product will perform in the wider market.
Once the product is launched within this test segment, the focus shifts to meticulous data collection and analysis. The team observes how the product is used, gathers feedback, and monitors any technical or user experience issues. This feedback loop is critical as it informs necessary adjustments and iterations to the product. Based on this iterative process, the product team can make a more informed decision about the full-scale rollout, adjust the product strategy, or even pivot in a new direction if required.
After the experiment
To make sure you move forward, it is a good idea to systematically record your the insights you learned and what actions or decisions follow. We created a sample Learning Sheet, that will help you capture insights in the process of turning your product ideas successful. Download the Learning Sheet.
Examples
Place to eat
If you want to build an app that helps users find the best place to eat, only build up its database for a specific city or country before scaling it to worldwide use. Valid learning can come from running marketing and testing efforts locally before scaling it globally.
McDonalds
McDonald’s tested an all-day breakfast menu in select locations before deciding to roll it out nationwide. The positive customer response from the test significantly boosted their sales and market share.
Samsung
Samsung’s One UI 5.0 Beta, based on Android 13, was initially released in seven countries including South Korea, the US, India, Germany, the UK, Poland, and China. This selective rollout allowed Samsung to gather feedback on new features and improvements before a broader release.
Source: ANDROID 13Not all regions eligible for Samsung One UI 5.0 Beta, find yours
Google’s testing framework employs four key stages: internal tests, crowdtesting, employee usage (‘dogfooding’), and beta testing with select users. This tiered approach ensures robust quality checks for core products.
Source: 5 effective and powerful ways to test like tech giants
This experiment is part of the Validation Patterns printed card deck
A collection of 60 product experiments that will validate your idea in a matter of days, not months. They are regularly used by product builders at companies like Google, Facebook, Dropbox, and Amazon.
Get your deck!Related plays
- Pretotype It by Alberto Savoia
- Pretotyping Techniques for Building the Right Product by Alberto Savoia
