Difficulty: Hard
Evidence strength
Relevant metrics: Jobs to be done ranking, Ranking needs, wants, desires, pains
Validates: Viability, Desirability
How: Discover common themes across data sources to confirm or invalidate assumptions. It's 'Garbage in, garbage out', so be careful what data you use. Get at least two data sources that are counter to each other (control group and experimental group).
Why: Uncover patterns of causality in observed results and suggest what results are more likely to be true. Ruling out statistically insignificance, confirmation biases, and false positive results, data mining can become one of your strongest allies.
This experiment is part of the Validation Patterns printed card deck
A collection of 60 product experiments that will validate your idea in a matter of days, not months. They are regularly used by product builders at companies like Google, Facebook, Dropbox, and Amazon.
Get your deck!Before the experiment
The first thing to do when planning any kind of test or experiment, is to figure out what you want to test. To make critical assumptions explicit, fill out an experiment sheet as you prepare your test. We created a sample sheet for you to get started. Download the Experiment Sheet.
Listening to customers
Where common lean startup advice is about getting out of the building to talk to real users, data-mining is an inhouse activity, listening to users and their activities.
Data mining uses statistics from large amounts of data to learn about customer behaviors and current and potential target markets. Typically, it involves using a data warehouse or a data management platform (DMP).
By discovering patterns in customer perceptions and behaviors, a fundamental understanding of the target audienc e and open opportunities can be revealed. By enriching your data set with mulitple data points such as customer satisfaction questionaires, NPS surveys, and the likes, links between reported satisfaction and actual usage can be spotted and in turn reveal important drivers for customer loyalty and churn.
It enables the extraction of hidden predictive information from an array of databases, transforming raw data into valuable insights. By leveraging data mining, product managers can forecast trends, understand customer preferences, and make data-driven decisions that align with market needs.
Combine data sources
Consider combining questionnaires with behavioral data such as past purchases and sign-ups to determine behavioral links between reported satisfaction and actual usage.
By integrating data from questionnaires with behavioral data such as past purchases and sign-ups, you can paint a more comprehensive picture of their customers. This approach allows for the correlation of self-reported customer satisfaction with their actual usage patterns, providing valuable insights into how customers interact with the product and their true sentiments about it.
Questionnaires, often used to gauge customer opinions and satisfaction levels, offer direct feedback from customers. However, this self-reported data can sometimes be skewed by subjective biases or the inability of customers to accurately recall their experiences. To counteract this, pairing questionnaire responses with objective behavioral data becomes crucial. Behavioral data, such as purchase history, website interaction, and product usage statistics, offers tangible evidence of how customers are engaging with the product.
When these two data sources are combined, product managers can identify behavioral links between reported satisfaction and actual product usage. For example, a customer might report high satisfaction in a survey but rarely use the product, indicating a possible discrepancy between perceived and actual value. Conversely, a customer might report average satisfaction but frequently use the product, suggesting a different kind of engagement that might not be captured through surveys alone.
Garbage in, garbage out
Data matters, but perspective matters more. As humans beings, we have a tendency to see what we want to see and draw conclusions based on our own biases (i.e. Confirmation bias, False positives, Ignorance of black Swans).
Getting outside help – or at least an outside perspective – to interpret the data will help ensure your objectivity. Also - checking on double data points counter to each other (control group and experimental group) will help you spot common patterns that shouldn’t be interpreted as more than regular variations.
Before diving into data mining, it’s crucial to understand the data at your disposal. This includes knowing where it comes from, how reliable it is, and what limitations it might have. Ensuring data quality is critical for deriving accurate insights.
Be clear about what you want to achieve with data mining. This might include identifying customer preferences, predicting market trends, or understanding competitive landscapes. Having specific goals will guide the data mining process more effectively.
How Data Mining can aid product validation
- Market validation. Before launching a new product or feature, data mining helps validate the market need. It enables product managers to analyze customer behavior, preferences, and trends, ensuring that the new product aligns with what the market desires.
- Personalization. In an era where personalization is key to customer satisfaction, data mining allows product managers to tailor products and experiences to individual customer needs. By understanding specific customer segments, products can be customized to meet diverse expectations.
- Risk mitigation. By predicting trends and customer responses, data mining helps in mitigating risks associated with new product launches. It provides a safety net by validating hypotheses with actual data, reducing the chances of product failure.
Incorporate customer feedback
Data mining is not just about numbers; it’s also about understanding customer sentiments. Incorporate social media analysis and customer feedback into your data mining strategy to get a holistic view of the market
Social media, in particular, has become a goldmine for customer sentiment analysis. Platforms like Twitter, Facebook, and Instagram are not just social networking sites but rich sources of customer opinions and trends. Analyzing comments, posts, and interactions on these platforms can reveal a lot about customer preferences and pain points. This type of qualitative data, when merged with traditional data mining techniques, enriches the understanding of the market. It enables product managers to see beyond the surface-level data, capturing the nuances of customer behavior and preferences.
Moreover, customer feedback gathered through surveys, reviews, and direct interactions provides a direct line of communication between the consumer and the company. This feedback is often more specific and detailed, offering direct suggestions, critiques, and accolades. By analyzing this data, product managers can identify common themes and patterns, helping them to understand what aspects of a product are working well and what areas require improvement.
The integration of customer feedback into data mining also fosters a more customer-centric approach to product development. It aligns product strategies with actual customer needs, ensuring that the end product resonates with its intended audience. This alignment not only enhances customer satisfaction but also drives loyalty and advocacy, which are crucial for a product’s success in the market.
Why do data mining for produt research?
The focus is on leveraging analytical techniques to sift through large datasets, aiming to uncover patterns, correlations, and trends that inform product strategy and development. This approach can validate assumptions about market and user behavior or reveal new insights.
Data mining serves as a critical tool for extracting actionable knowledge from vast amounts of data. By identifying meaningful patterns and trends, businesses can make informed decisions that are grounded in empirical evidence. T
he process involves not only the technical extraction of data but also its careful analysis to ensure that the conclusions drawn are both valid and applicable to real-world challenges. The key to successful data mining lies in the quality of the data sources, the robustness of the analysis methods, and the strategic interpretation of the results.
After the experiment
To make sure you move forward, it is a good idea to systematically record your the insights you learned and what actions or decisions follow. We created a sample Learning Sheet, that will help you capture insights in the process of turning your product ideas successful. Download the Learning Sheet.
Examples
USDA
USDA regularly uses data mining to identify problems such as food deserts in the United States.
Source: USDA Defines Food Deserts
Instacart
Instacart aimed to optimize product recommendations. Using data mining, they analyzed shopping patterns to enhance strategic product placements and marketing approaches.
Source: Use Data Mining to Predict If Your Product Will Crash and Burn
Etsy
Etsy explored enhancing user experience through tailored recommendations. Their system analyzed past searches and purchases with natural-language-processing, improving customer engagement.
Source: Use Data Mining to Predict If Your Product Will Crash and Burn
Electronic Arts
EA focused on optimizing marketing spend. They used data mining to update daily CLV estimates based on gamer behavior, leading to more targeted and cost-effective marketing.
Source: Use Data Mining to Predict If Your Product Will Crash and Burn
Free People (Urban Outfitters)
Free People, a segment of Urban Outfitters, uses data mining to shape its future offerings. They analyze millions of customer records to understand trends such as what sold, what didn’t, and customer returns. This information fuels the brand’s product recommendations, website design, and promotional strategies.
Source: 20 companies do data mining and make their business better
Macy's
Macy’s employs sentiment analysis of big data to understand customer preferences and trends. For instance, they analyze social media data to discover brand associations with products like jackets, using this information to target future advertising campaigns. This method of data mining aids in validating market interest and preferences before launching targeted marketing efforts for new products
Source: 20 companies do data mining and make their business better
Netflix
Netflix uses data mining to determine the popularity of shows and movies. Their approach to data analysis informed the decision to invest heavily in the show “House of Cards,” validating the market’s interest before the official launch. This strategy of analyzing viewer preferences and behaviors is central to their product development process.
Source: Top 11 Companies who use Data Mining That You Must Know
Booking.com
As a major player in travel e-commerce, Booking.com leveraged SEO data analytics to discover valuable information and expand its market presence. They used data mining to identify imbalances in their market focus between Europe and the US and to validate new ideas, such as the creation of specific landing pages, based on search volume and keyword competitiveness.
Source: 7 Real-World Examples Of Data Mining In Business, Marketing, Retail
Starbucks
Starbucks used data mining for deciding store locations. They implemented a data-driven approach with mapping software that analyzes vast amounts of data about potential store openings, including location-based data and demographics. This helped them identify the best locations for new stores without negatively impacting existing ones.
Source: 7 Real-World Examples Of Data Mining In Business, Marketing, Retail
This experiment is part of the Validation Patterns printed card deck
A collection of 60 product experiments that will validate your idea in a matter of days, not months. They are regularly used by product builders at companies like Google, Facebook, Dropbox, and Amazon.
Get your deck!Related plays
- The Real Startup book - Data Mining by Tristan Kromer, et. al.
