Also called: Product Reviews
See also: Customer Service Logs, Industry Forums
Difficulty: Intermediate
Evidence strength
Relevant metrics: Jobs to be done ranking, Ranking needs, wants, desires, pains, Quotes & stories
Validates: Desirability
How: Read reviews of your own and competing products in app stores and on review sites and distill emotional outbursts, frustrations, and requests to tangible insights. Consider whether users are missing features, experiencing confusing UI, complaining about bugs or performance, or could need better onboarding.
Why: The frustrations and pain points of your own and competing products are shared without filter and in all honesty in app reviews. App reviews provide free access to feedback from existing and potential customers.
This experiment is part of the Validation Patterns printed card deck
A collection of 60 product experiments that will validate your idea in a matter of days, not months. They are regularly used by product builders at companies like Google, Facebook, Dropbox, and Amazon.
Get your deck!Before the experiment
The first thing to do when planning any kind of test or experiment, is to figure out what you want to test. To make critical assumptions explicit, fill out an experiment sheet as you prepare your test. We created a sample sheet for you to get started. Download the Experiment Sheet.
Don’t mind the rating
When looking for insights on your own product or competing products on the App Store or Google Play, you should focus on the substance of what is being said rather than the rating itself (typically one to five stars).
You could look for these queus:
- Are users annoyed?
- Are users asking for specific features to be added?
- Are users struggling with specific parts of the experience
- Are uses excited about the product or part of the experience?
- Are users complaining about bugs or performance issues?
- Are users loving features not indended as part of the main experience (that should consider promoting)?
- Do users understand how to get started, or would it seem appropriate to add a tutorial?
- Are there any parts of the user experience that doesn’t get mentioned?
- Are users just loyal to the brand above anything else?
Reviews tend to be emtionally driven outliers
Users are most likely to leave feedback as they are motivated by an emotional reaction to the product. Emotional triggers, positive or negative, tend to happen by outliers on the bell curve. This is why it’s important to spend time distilling your findings into tangible insights.
Research your competitors
Reading app reviews is a great chance to study competing products – not only your own. Can you spot a frustation or feature gap in a competitors app that you can close?
After the experiment
To make sure you move forward, it is a good idea to systematically record your the insights you learned and what actions or decisions follow. We created a sample Learning Sheet, that will help you capture insights in the process of turning your product ideas successful. Download the Learning Sheet.
Examples
Recurly
For Recurly, the customer support team is the fastest and easiest way to get feedback from customers. By talking with the team, listening in on calls, and working to identify beta customers, customer support is an integrated part of product development.
Source: The UX of Customer Support
Related plays
- User Research When You Can’t Talk to Your Users by Jon Peterson
- The UX of Customer Support by Recurly