See also: Closed-Ended Surveys, Conjoint Analysis, Trends and Keyword Analysis
Difficulty: Hard
Evidence strength
Relevant metrics: Jobs to be done ranking, Ranking needs, wants, desires, pains
Validates: Viability, Desirability
How: Discover common themes across data sources to confirm or invalidate assumptions. It's 'Garbage in, garbage out', so be careful what data you use. Get at least two data sources that are counter to each other (control group and experimental group).
Why: Uncover patterns of causality in observed results and suggest what results are more likely to be true. Ruling out statistically insignificance, confirmation biases, and false positive results, data mining can become one of your strongest allies.
This experiment is part of the Validation Patterns printed card deck
A collection of 60 product experiments that will validate your idea in a matter of days, not months. They are regularly used by product builders at companies like Google, Facebook, Dropbox, and Amazon.
Get your deck!Before the experiment
The first thing to do when planning any kind of test or experiment, is to figure out what you want to test. To make critical assumptions explicit, fill out an experiment sheet as you prepare your test. We created a sample sheet for you to get started. Download the Experiment Sheet.
Listening to customers
Where common lean startup advice is about getting out of the building to talk to real users, data-mining is an inhouse activity, listening to users and their activities.
Data mining uses statistics from large amounts of data to learn about customer behaviors and current and potential target markets. Typically, it involves using a data warehouse or a data management platform (DMP).
By discovering patterns in customer perceptions and behaviors, a fundamental understanding of the target audienc e and open opportunities can be revealed. By enriching your data set with mulitple data points such as customer satisfaction questionaires, NPS surveys, and the likes, links between reported satisfaction and actual usage can be spotted and in turn reveal important drivers for customer loyalty and churn.
Garbage in, garbage out
Data matters, but perspective matters more. As humans beings, we have a tendency to see what we want to see and draw conclusions based on our own biases (i.e. Confirmation bias, False positives, Ignorance of black Swans).
Getting outside help – or at least an outside perspective – to interpret the data will help ensure your objectivity. Also - checking on double data points counter to each other (control group and experimental group) will help you spot common patterns that shouldn’t be interpreted as more than regular variations.
After the experiment
To make sure you move forward, it is a good idea to systematically record your the insights you learned and what actions or decisions follow. We created a sample Learning Sheet, that will help you capture insights in the process of turning your product ideas successful. Download the Learning Sheet.
Examples
USDA
USDA regularly uses data mining to identify problems such as food deserts in the United States.
Source: USDA Defines Food Deserts
Related plays
- The Real Startup book - Data Mining by Tristan Kromer, et. al.