Idea Validation: Product

Takeaway Test

Remove or disable a feature to see if it is valued

Illustration of Takeaway Test
Run a Takeaway Test play

Also called: Disable a Feature

Difficulty: Intermediate

Requires existing audience or product

Evidence strength
50

Relevant metrics: Customer feedback, Ranking needs, wants, desires, pains

Validates: Desirability

How: Purposely remove or disable a product feature or function without notifying your existing customers. If it is a valued feature, you will hear from your customers – if not, it was probably a good idea to remove it after all. Consider removing the feature for a specific segment of users.

Why: The primary purpose of the experiment is to eliminate unnecessary features that may distract the user or otherwise add complexity to product maintenance without adding value. If you are unsure whether an existing feature or function of your product is valuable to your customers, this is the most effective way to find out. While some users may be particularly vocal and complain about a change, that does not necessarily indicate that removing the feature is negative for the larger user base.

This experiment is part of the Validation Patterns printed card deck

A collection of 60 product experiments that will validate your idea in a matter of days, not months. They are regularly used by product builders at companies like Google, Facebook, Dropbox, and Amazon.

Get your deck!

Before the experiment

The first thing to do when planning any kind of test or experiment, is to figure out what you want to test. To make critical assumptions explicit, fill out an experiment sheet as you prepare your test. We created a sample sheet for you to get started. Download the Experiment Sheet.

An easy, but sometimes grim experiment to run

By purposely removing a product feature without notifying customer, you will soon find out whether somebody will be missing it. Removing a feature commonly used will generally cause an immediate negative reaction from users, why it is relatively easy to identify changes in user behavior.

Even though users are likely to react, it is important that you also keep an eye on key metrics and user behavior to make a final decision. Even though your users did not use the particular feature as a core offering and did not complain when you removed it, removing it can be the cause of immediate confusion as they browse through the rest of your product.

In the opposite end of the spectrum, some users can be particularly vocal about your change, even though it’s hard to spot any significant trends in your usage data.

When removing a feature from a product with a large user base, expect to let time pass for user behavior to stabilize and initial complaints to stop.

After the experiment

To make sure you move forward, it is a good idea to systematically record your the insights you learned and what actions or decisions follow. We created a sample Learning Sheet, that will help you capture insights in the process of turning your product ideas successful. Download the Learning Sheet.

Examples

Instagram's Removal of 'Likes' Count

Instagram tested hiding the ‘likes’ count on posts in several countries to see how it would affect user interaction and mental health. This significant change in a core feature of the platform was an example of testing a feature’s impact before a global rollout.

Sources

Want to learn more?

Receive a hand picked list of the best reads on building products that matter every week. Curated by Anders Toxboe. Published every Tuesday.

No spam! Unsubscribe with a single click at any time.

Ice Breakers

Relieve initial group awkwardness and establish a safe space

Community events
Product Loop

Product Loop provides an opportunity for Product professionals and their peers to exchange ideas and experiences about Product Design, Development and Management, Business Modelling, Metrics, User Experience and all the other things that get us excited.

Join our community

Made with in Copenhagen, Denmark

Want to learn more about about good product development, then browse our product playbooks.