From Experimentation Skeptic to Advocate

The ability to innovate and adapt is crucial for any product team. This is where product experimentation, product discovery, and user research come into play. By integrating these practices into their workflows, organizations can better navigate uncertainties and make informed decisions based on real user feedback. This blog post explores the importance of product experimentation, addresses common obstacles, and provides actionable strategies to foster a culture of experimentation within your organization.

Why Product Experimentation Matters

Product experimentation involves testing hypotheses about your product with real users to gather data and insights. Unlike large-scale A/B tests, the focus is on day-to-day experimentation and user research to manage uncertainty in new product and service development. Product experimentation is essential because it allows teams to:

1. Validate Assumptions: Confirm or refute assumptions about user needs and behaviors.
2. Reduce Risks: Identify potential issues early, reducing the risk of costly mistakes.
3. Enhance User Experience: Continuously improve the product based on user feedback.
4. Drive Innovation: Foster a culture of creativity and innovation by encouraging new ideas and approaches.

Common Obstacles to Embracing Product Experimentation

Despite its benefits, many organizations struggle to embrace product experimentation. The common obstacles include:

1. Lack of Understanding: Stakeholders often see experimentation as a distraction from delivering features.
2. Misalignment with Goals: Teams may struggle to see how experimentation fits into their delivery schedules.
3. Focus on Tools over Principles: Similar to agile rollouts, there’s a tendency to focus on mechanics rather than the underlying mindsets.
4. Fear of Failure: Experimentation can be perceived as risky or as a challenge to influential team members’ ideas.
5. Valuing Outputs over Outcomes: Emphasis on deliverables rather than learning can push experimentation to the sidelines.

To successfully integrate product experimentation, product discovery, and user research into your workflow, consider these strategies:

1. Help Others See the Uncertainties

Invite stakeholders and team members to identify uncertainties and assumptions early in the project. Use methods like assumptions mapping to highlight potential risks and uncertainties. This process involves:

• Conducting a pre-mortem to identify what could go wrong.
• Turning identified risks into testable assumptions.
• Inviting key individuals to participate in this process to foster a shared understanding.

2. Demonstrate the Benefits of Testing

Instead of just explaining the benefits of experimentation, involve stakeholders in the process. Use real-life examples and case studies to show how learning leads to better outcomes. For instance, use sentence stems like “I used to believe X, then I saw Y, and now I believe Z” to illustrate shifts in understanding based on user research.

3. Implement Learning Swim Lanes

Create a parallel “learning swim lane” in your development process. This involves:

• Setting aside a small portion of team resources for ongoing user research and experimentation.
• Conducting discovery activities alongside development work without distracting the main team.
• Using insights gained to inform and improve the product as it evolves.

4. Make Learning Human

Use tools like video ask from Typeform to capture and share short video responses from team members about their learnings. This approach helps:

• Elevate individual contributors’ voices in a way that traditional status meetings may not.
• Create a “sizzle reel” of learnings to share with stakeholders, showcasing the human side of user research.

5. Go Rogue (With Caution)

Sometimes, you may need to conduct small-scale experiments without explicit permission. This can involve:

• Conducting night and weekend tests to gather preliminary data.
• Sharing results in a way that doesn’t disrupt the main development work.
• Gradually demonstrating the value of experimentation to gain more formal support.

6. Establish a Cadence of Continuous Discovery

Follow principles from books like “Continuous Discovery Habits” by Teresa Torres. Set up a regular cadence for user research and experimentation, such as:

• Allocating specific times each week for user interviews or testing sessions.
• Keeping the stakes low and the frequency high to make experimentation a natural part of the workflow.
• Demonstrating quick, iterative learnings to show that testing can be easy, safe, and cheap.

Building a Culture of Experimentation

Fostering a culture of experimentation requires a strategic approach:

Focus on Small Wins

Instead of trying to overhaul the entire organization at once, focus on small wins:

• Identify and work with early adopters who are open to experimentation.
• Slowly build a coalition of believers who can advocate for experimentation within the organization.
• Use data and success stories to convert skeptics over time.

Use Customer Voices

Leverage the power of customer feedback to make a compelling case for experimentation:

• Share quotes and audio clips from user interviews that highlight critical insights.
• Use real user experiences to illustrate the importance of experimentation and discovery.
• Ensure these insights are shared at the right time and place to maximize impact.

Create Relevant Experiences

Design experiments that are highly relevant to stakeholders’ interests:

• Develop cohort-based, longitudinal programs that involve stakeholders in designing and running their own experiments.
• Use peer coaching, accountability, and project-based learning to enhance engagement.
• Measure and highlight changes in thinking and behavior to demonstrate the value of experimentation.

The key to successful experimentation is not just about having the right tools and techniques, but also about cultivating the right mindsets and behaviors within your team. Embrace the journey of experimentation, and you’ll unlock new opportunities for your product and organization.