Don't Outsource Your Product Thinking to AI
AI can tell us what customers are saying. But can it tell us what they really need?
Recently, my CEO expressed frustration with how heavily we were relying on Harvestr to inform our product decisions.
âI donât want to outsource thinking and strategy to an AI.â
My initial reaction was that we werenât doing that. We were collecting customer feedback, identifying recurring problems, and using AI to make sense of the information. Surely, making decisions based on more customer evidence is a good thing?
But thereâs something important about where his objection came from.
My CEO participates in hundreds of sales conversations every year. He hears firsthand which value propositions resonate, what motivates customers to buy, and why they choose us over alternatives.
He also handles critical customer escalations. Heâs directly exposed to the frustrations, expectations, and circumstances behind some of our most difficult customer problems.
Over time, these experiences build something that is difficult to capture in a feedback management system: intuition about what customers actually value, what theyâre trying to accomplish, and how our product should help them.
So when AI identifies a recurring feature request and recommends prioritizing it, his response might be:
âYes, thatâs what customers are saying. But thatâs not necessarily what they need. And itâs not how we want to solve that problem.â
Thatâs not a rejection of customer evidence. Itâs an interpretation of that evidence through accumulated experience and strategic intent.
When does using AI to help us understand customers become a way of outsourcing the thinking that makes us good at building products?
Customer feedback is evidence, not strategy
Weâve spent years encouraging product teams to become more customer-centric.
Talk to customers. Collect feedback. Identify patterns. Validate assumptions. Make evidence-based decisions.
These are important practices. But somewhere along the way, itâs possible to confuse listening to customers with letting customers determine the roadmap.
Imagine that your feedback system identifies 200 requests for better reporting, 120 for additional integrations, and 80 for more customization options.
AI can categorize these requests, identify recurring themes, and recommend which opportunities deserve attention.
But what do those numbers actually tell us?
They tell us what some customers are asking for. They donât necessarily tell us what we should build.
The customer asking for an export feature might actually be struggling to collaborate with colleagues. The customer requesting more configuration options might be compensating for a workflow that doesnât fit their needs.
Building what they ask for could solve the immediate frustration without addressing the underlying problem.
And an AI analysis of incoming requests might miss the broader context: why customers bought the product, what they expected it to accomplish, and which solutions are consistent with the productâs direction.
Being customer-centric doesnât mean letting customers collectively design your roadmap.
It means understanding their needs deeply enough to make good decisions about how to serve them.
The hidden value of doing the thinking yourself
In my previous article about AI-powered Wizard of Oz experiments, I explored how AI allows us to deliver personalized experiences at a scale that previously required substantial human effort.
But thereâs a paradox: When we automate the work, we may also eliminate the firsthand experiences through which we learn about customers.
The same thing happens when we automate analysis.
My CEOâs product intuition wasnât developed by reading a summary of hundreds of sales calls and customer escalations. It developed through the experience of participating in them.
Hearing how customers describe their problems. Recognizing when stated objections hide different concerns. Understanding which promises make customers buy and which expectations we struggle to fulfill.
Over time, these observations form mental models that help us interpret new situations.
This is part of what we call product sense or product taste: the accumulated ability to make good decisions when evidence is incomplete.
AI can analyze thousands of conversations, identify patterns, and produce useful recommendations.
But it only has access to the information we capture. And even when it has excellent information, reading its conclusions isnât the same as developing the understanding yourself.
You can outsource the production of an insight without acquiring the understanding that would have produced it.
That doesnât mean intuition is always right. Experience can introduce biases, and a founderâs understanding may overrepresent the customers they encounter.
But that is precisely why we need to combine firsthand judgment with broader customer evidence.
AI can challenge our intuition and help us see what our direct experiences might have missed. It shouldnât simply replace those experiences.
When product discovery becomes reactive
Most customer feedback concerns the product people already know.
They request improvements to existing features, solutions to current frustrations, and capabilities that fit their established workflows.
These requests are valuable. But theyâre also shaped by the present.
Customers rarely request fundamentally different ways of solving problems they havenât yet imagined.
AI can identify patterns across thousands of requests. But the most frequently expressed needs arenât necessarily the most strategically important opportunities.
A team relying too heavily on incoming feedback risks becoming increasingly reactive.
The roadmap becomes a sequence of responses to customer requests. Each decision might be reasonable. Each improvement might make existing customers happier. Yet the product can gradually lose its direction.
Most companies begin with a belief: a problem worth solving, an opportunity others havenât recognized, or a better way of doing something.
That belief becomes the foundation of the product vision.
Of course, founders can be wrong. Thatâs why we need discovery and experimentation.
But discovery should help us test and evolve our strategic beliefs, not automatically replace them with whatever customers request most frequently.
Vision without evidence becomes dogma. Evidence without vision becomes reaction.
Good product strategy requires both.
Sometimes that means rejecting a popular feature request because it would take the product in the wrong direction. Other times, it means pursuing an opportunity before customers can clearly articulate what they need.
Use AI to challenge your thinking, not replace it
I donât think we should stop using AI for product discovery. Quite the opposite.
AI allows us to analyze more evidence, explore alternative interpretations, and identify weaknesses in our assumptions.
But instead of asking AI, âWhat should we build next?â, we could ask:
- What underlying customer needs might explain these feature requests?
- What evidence contradicts our current interpretation?
- Which customer segments are overrepresented in the feedback?
- What do sales conversations and customer escalations reveal that feature requests donât?
- Which opportunities fit our strategic intent, and which would pull us away from it?
I particularly like the idea of developing our own interpretation before consulting AI-generated recommendations.
Form a hypothesis. Explain why you believe it. Connect it to customer evidence and strategic intent.
Then use AI to challenge the reasoning, identify missing information, and suggest alternatives.
Sometimes AI will identify something weâve overlooked. Sometimes its recommendations will be better than our initial judgment.
We should be willing to change our minds.
But thereâs a difference between changing your mind because youâve understood a stronger argument and accepting a recommendation because the system appears confident.
In one case, your judgment develops. In the other, you may simply become more dependent on the tool.
Outsource processing, own the thinking
Looking back at my CEOâs objection, I think he was pointing to something that customer feedback systems struggle to capture.
The accumulated understanding of why customers buy, what theyâre trying to accomplish, and what kind of product weâre trying to build for them.
That understanding isnât infallible. It needs to be challenged by evidence and informed by perspectives beyond the founderâs own experiences.
But it shouldnât be discarded simply because AI can process more feedback.
The purpose of product discovery isnât simply to produce insights. Itâs to develop the understanding necessary to make better product decisions.
AI can help us listen to more customers and recognize more patterns. But deciding what those patterns mean, which problems matter, and how we want to solve them remains our responsibility.
Outsource the processing. Own the thinking.