Closed-Ended Surveys

What should I use closed surveys for?

Closed-ended surveys are often used as part of market validation and general data collection, focusing on gathering structured quantitative data from predefined answer choices. These surveys are particularly effective in exploring the known unknowns, where some level of understanding already exists, but specific, detailed information is needed. They are used when you have a clear direction and aim to fill knowledge gaps.

For instance, a startup founder who has established a problem-solution fit might use closed-ended surveys to prioritize or identify additional issues. The key lies in the structured nature of the questions, which range from simple yes/no queries to more complex matrix questions, allowing for a detailed exploration of specific areas.

The methodology of conducting these surveys can vary significantly, from online forms to in-person interviews, and the time commitment can range from a quick online setup to weeks of in-depth surveys and data entry.

When designing a closed-ended survey, clarity of purpose is crucial. Questions should be directly linked to the actionable outcomes they’re intended to inform, and the type of questions can vary widely, including dichotomous, Likert-type scales, list of items, ordinal, and matrix questions.

After collection, the data should be organized for analysis, employing statistical tools for deeper insights. It’s vital to ensure a sufficient sample size for reliability, particularly within subgroups. Analysis involves looking at overall scores, comparing them to benchmarks, and examining the distribution of responses.

Potential biases such as leading questions, sensitive content, and respondent bias should be carefully managed. Surveys should be designed to be straightforward and allow for responses like “not appropriate” or “don’t know” to capture a more accurate picture of the respondent’s views.

Be wary of false positives: don’t ask for opinions

Asking for opinions, you get opinions. People have opinions on everything, and it doesn’t necessarily reflect what people are actually doing in the real world. Whenever you hear yourself asking question that begin with “Would you…” or “Do you think…”, you will want to stop youself.

It’s better to ask for facts. Questions like “When was the last time you googled…” shows whether potential customers thought about it before you triggered them. If you ask “Would you” or “Do you think”, those same people will be very likely to answer positively. “When was the last time…?” refers to a concrete instance. You want to go for the facts and the instances when you ask.

Don’t force your own perspective

You do not want to focus on the solution because you will be framing the conversation around what you think is important. Instead, you want to go for uncovering the pains and gains of the potential customer.

Instead, you want to first go for playbooks:

• Validate the problem.
• Validate market demand.
• Validate the product.
• Validate willingness to pay.

Go for situations - concrete instances in the past life of the potential customer you’re asking. “Tell me the last time you thought about…” is a great starter. After that, you can ask follow up questions like “Why did it bother you?”.