Every product decision is a bet. Whether you’re adding a new feature, restructuring a core function, or refining your product, you’re betting that the decision will lead to a positive outcome. However, the success of these bets hinges on the level of uncertainty involved. That’s where experiments come in—they reduce uncertainty and help you make better, more informed decisions.
Many product teams fall into two traps: either chasing after the next big thing while neglecting their core offering, or playing it so safe that they stagnate in a fast-moving market. Neither extreme is sustainable, and both can jeopardize long-term success.
So, how can you consistently make better product decisions?
Nils Koppelmann suggests balance – just like a smart investor balances risk and reward in their portfolio, you must balance your product experiments to achieve both short-term wins and long-term innovation. But how exactly can you do that?
Every product decision is a bet
Every time you make a strategic decision about your product, you’re making a bet. You’re placing a wager that a specific change—whether it’s a feature addition, a redesign, or a new market entry—will lead to success. But, not all bets are created equal. Some are high-risk, high-reward, while others are safer with more predictable returns.
That’s why experiments are crucial. They help you reduce uncertainty and validate your bets. The big question isn’t whether you should make bold bets, but rather how you can reduce the uncertainty behind them to make those bets more informed. So how can you balance these product bets effectively?
The key is to think of your experiments like an investment portfolio.
Balance your bets
Think of your product strategy like a portfolio of investments. A savvy investor wouldn’t put all their money into volatile stocks or stick only to low-yield bonds. Instead, they balance their investments, maximizing the potential for return while managing risk. Similarly, you should balance your product experiments—some with high-risk and high-reward potential, and others with low-risk, steady gains.
But why is this portfolio mindset so crucial? Because it ensures that you’re not over-investing in risky innovations or playing it too safe with only incremental changes. When you have the right mix, you can continuously improve your product while still exploring transformative opportunities.
Let’s look at the different types of experiments that make up a balanced portfolio—and why they matter.
Types of product bets
To create a well-balanced experimentation portfolio, you need a mix of experiments that reduce uncertainty across different areas of your product. Each type of experiment addresses different aspects of your product’s risk and reward, and getting this balance right is crucial to sustainable growth. Here’s how to think about it:
- Incremental optimizations These low-risk, small-scale changes can have a big cumulative impact. For mature companies or well-established products, incremental optimizations should make up the bulk of your experiments. These are minor adjustments that refine and improve existing features without overhauling the core product. They’re like the bonds in your portfolio—steady and reliable.
- Example: Tweaking the layout of a product page for better readability.
- Example: Adjusting the prominence of a CTA button to increase conversions.
- Example: Simplifying a sign-up form to reduce friction for new users.
Over time, these changes add up, creating a more polished and user-friendly product. The real power of incremental optimizations is their ability to build momentum, gradually improving user experience while minimizing risk.
- Moderate Restructuring Sometimes, more significant changes are needed to address shifting market demands or core functionality issues. Moderate restructuring involves making targeted adjustments to how users interact with your product, without reinventing the wheel. These changes carry more risk but offer the potential for greater gains.
- Example: Overhauling the onboarding flow to start immediately after sign-up.
- Example: Targeting a new user segment with revised messaging or UX tweaks.
- Example: Restructuring navigation to prioritize underutilized features.
By making moderate but impactful changes, you can refine your product’s positioning and open new avenues for growth without alienating your existing users. Moderate restructuring can help ensure that your product evolves in alignment with market needs.
- Feature Additions To stay competitive, you need to continually add new capabilities that create additional value for users. Feature additions bring new functionality to your product without altering its core. While they carry moderate risk, the potential for user retention and growth is high.
- Example: Introducing a calendar integration to enhance user productivity.
- Example: Implementing a progress dashboard that helps users track their activity.
New features can keep your product fresh and relevant in a competitive market. By expanding functionality, you create opportunities for greater user engagement and retention—essential for long-term success.
- Radical Innovations Every product needs bold moves to stay ahead of the curve. Radical innovations are the high-risk, high-reward experiments that have the potential to transform your business. They involve making fundamental shifts in your product’s identity, market positioning, or even its business model.
- Example: Pivoting from a one-time purchase model to a subscription service.
- Example: Rebranding your product to target a new market.
- Example: Launching a new AI-driven feature that automates manual tasks.
These are the bets that, if successful, can create massive differentiation in the market. Radical innovations are riskier, but they offer the potential for breakthrough success, keeping your company ahead of competitors and helping you lead the market.
An experiment can serve very different purposes
Not all experiments serve the same function. Some are meant to explore new possibilities, others test specific hypotheses, and some are focused on refining what you already know works. Understanding the purpose of each experiment helps you decide when and how to deploy them effectively.
Here’s how experiments break down by purpose:
| Experiment Type | Objective | Timing | Scope | Approach | Outcome |
|---|---|---|---|---|---|
| Discovery | Explore unknowns | Early stages with high uncertainty | Broad, open-ended exploration | Rapid iteration and course correction | Identify potential directions |
| Validation | Confirm hypotheses | After gathering initial insights | Focused on specific assumptions | Targeted experiments with success criteria | Create evidence to support decisions |
| Optimization | Fine-tune validated ideas | Fine-tuning before scaling or launch | Detailed and targeted improvements | Incremental adjustments | Achieve optimized outcomes for scaling |
Using this framework, you ensure that your experiments are serving the right purpose at the right time, moving your product closer to success.
Let’s examine each purpose in greater detail.
- Discovery: Exploring the unknown Early in the product lifecycle, discovery experiments help you explore the unknown and test your riskiest assumptions. This is where you uncover new opportunities and identify the most promising directions for your product.
- Example: Conducting customer interviews to identify new user needs.
- Example: Reading App reviews to understand user frustrations and praise
- Example: Surveying potential users to gauge demand for a new feature.
Discovery experiments allow you to gather insights and identify opportunities before making any major commitments. They are critical to reducing the risk of big product bets early on.
- Validation: Testing hypotheses Once you’ve gathered initial insights, validation experiments confirm whether your assumptions are correct. They reduce uncertainty by testing specific hypotheses, allowing you to validate whether a feature or strategy will perform as expected.
- Example: Launching a prototype to a subset of users and measuring their engagement.
- Example: Conducting usability tests to ensure users can navigate new features effectively.
- Example: Testing different pricing models to find the one that resonates best with your audience.
Validation experiments give you confidence that your product changes are grounded in data. They provide concrete evidence that helps you make more confident, informed decisions about product direction.
- Optimization: Fine-tuning for maximum impact Once you’ve validated your ideas, optimization experiments help you fine-tune your product to maximize its efficiency and impact. These experiments are essential to refining your product and ensuring it performs at its best.
- Example: Improving page load times to increase conversions.
- Example: Tweaking the layout of a feature based on heatmap data.
- Example: Optimizing marketing messaging for specific user segments based on A/B tests.
Optimization experiments deliver small, incremental improvements that lead to a more polished, user-friendly product. Over time, these refinements can significantly boost user satisfaction and product performance.
Categorizing your experiments
To effectively manage your product experiments, you need a way to categorize them. By evaluating each experiment’s scope, impact, resources, risk, and strategy alignment, you can prioritize your efforts and ensure your experimentation portfolio is balanced.
Your experimentation strategy should evolve as your company grows.
- Startups. Focus on discovery and radical innovations to explore new market opportunities. These high-risk, high-reward bets are necessary to find product-market fit.
- Mature companies. Shift toward incremental optimizations and moderate restructuring. Your focus should be on refining and maintaining your product while exploring occasional innovations to stay competitive.
- Industry-specific. Balance your experiments based on your industry. In eCommerce, for instance, radical changes may involve a new checkout process, while in SaaS, it might be a pricing model shift.
Tailoring your approach based on your company stage ensures you’re making the right bets at the right time.
How keeping bets balanced helps your team and product
Balance isn’t just about managing risk; it’s also about maintaining team motivation and product momentum. A mix of quick wins from incremental optimizations and bold innovations keeps your team energized and your product evolving. Overemphasis on long-term bets can demotivate your team, while focusing solely on small tweaks can make your product stagnate.
A balanced experimentation approach creates a rhythm that drives product growth, keeps your team motivated, and ensures that your product stays relevant.
How Validation Patterns helps
Managing a balanced portfolio of experiments across discovery, validation, and optimization can feel overwhelming. Validation Patterns is a deck of lean product experiments designed to simplify this process, helping you diversify your experimentation strategy and make informed bets.
By using Validation Patterns, you can reduce uncertainty, prioritize experiments that align with your strategy, and drive consistent product growth.
Every product decision is a bet. By balancing different types of experiments and understanding their purpose, you can make informed, strategic bets that reduce risk and maximize reward. Whether you’re a startup exploring new markets or a mature company optimizing for growth, managing your experiments like a portfolio is key to long-term success.
Ready to balance your experimentation strategy?
Grab your Validation Patterns card deck and start managing your product experiments like a pro!
- Your 'innovative' product strategy might be killing your business by Nils Koppelmann
- Experiment Type Taxonomy, a framework by Nils Koppelmann
