Also called: Hallway Usability Testing
See also: Remote User Testing, Try it Yourself
Difficulty: Easy
Evidence strength
Relevant metrics: Usability, Customer feedback
Validates: Feasibility
How: Conduct 3-5 brief usability or product tests no longer than 15 minutes with users merely looking like somebody from your target audience to reveal valuable insights from outside perspectives. Approach a stranger, ask if they'd like to participate, give them a few tasks to do, observe their interactions, and ask about their experience.
Why: Some would argue that testing on more than 3-5 users is a waste of time and resources – some the opposite. Regardless of what is considered correct, testing on zero users yields zero insights. Conducting just a few small user tests with random lookalike users can quickly reveal valuable insights in a matter of hours rather than days and will as such allow faster design iterations.
This experiment is part of the Validation Patterns printed card deck
A collection of 60 product experiments that will validate your idea in a matter of days, not months. They are regularly used by product builders at companies like Google, Facebook, Dropbox, and Amazon.
Get your deck!Before the experiment
The first thing to do when planning any kind of test or experiment, is to figure out what you want to test. To make critical assumptions explicit, fill out an experiment sheet as you prepare your test. We created a sample sheet for you to get started. Download the Experiment Sheet.
Elaborate usability tests are a waste of resources
As early as in year 2000, Jakob Nielsen claimed that “The best results come from testing no more than 5 users and running as many small tests as you can afford.” The reason being that the number of new usability problems found and learning obtained drastically decline the more tests you conduct to a point where it isn’t of any significance. He claims that after the fifth user, you are practically wasting your time by observing the same findings repeatedly, but not learning anything new.
There are no facts inside your building
Go outside. Go to locations where you will find target users and ask them to use your prototype for a quick minute or two. The guerilla approach to usability testing tries to remove barriers and challenges related to scheduling, papwerwork and setup. Instead the focus is to get out of the building to find real potential users in the wild.
Why go discount?
Guerilla User Testing stands out as a valuable method for product teams, particularly in today’s fast-paced and resource-constrained environment. One of the most significant benefits of Guerilla User Testing is its flexibility and speed. Unlike traditional testing methods, which often involve extensive planning, recruitment, and structured environments, guerrilla testing is far more spontaneous. Conducted in everyday settings like coffee shops or parks, it allows teams to gather feedback in real-world scenarios. This spontaneity not only speeds up the testing process but also ensures that the feedback is grounded in genuine user experiences, rather than controlled and artificial testing environments.
The selection of participants in Guerrilla User Testing is typically more casual. Individuals are approached randomly, which can lead to a diverse range of insights. These participants are usually representative of the general population or a segment of the target audience, providing immediate and authentic responses to the product being tested. This approach, however, does not always guarantee a representative sample of the entire target audience, which is a factor teams must consider.
The testing tasks in Guerrilla User Testing are designed to be simple and focused. Since the interaction with participants is brief, it’s crucial that these tasks are direct and can be quickly understood. This simplicity allows teams to zero in on specific features or aspects of a product, making it easier to gather relevant feedback swiftly.
Cost-effectiveness is another appealing aspect of Guerrilla User Testing. It eliminates the need for expensive testing facilities and compensations usually associated with traditional methods. Sometimes, a small incentive, such as a coffee voucher, is offered to participants, but overall, the costs are significantly lower.
The qualitative nature of the data gathered through Guerrilla User Testing offers deep insights into user behaviors and preferences. This method is less about gathering statistically significant data and more about understanding how users interact with and react to a product. It provides a window into the user’s mind, revealing how they perceive and use the product in a natural setting.
When Guerrilla User Testing is not a great fit
While Guerilla User Testing is a valuable tool for many product teams, there are specific situations where this technique may not be the most suitable choice. Understanding these scenarios helps ensure that product teams utilize the method effectively and choose alternative approaches when necessary.
- Need for Demographically Specific Feedback. If the product or feature being developed targets a very specific demographic, Guerilla User Testing may not be ideal. This method often involves approaching random individuals, which may not yield the targeted demographic insights required for certain products.
- Complex or Highly Technical Products. Products that require specialized knowledge or extensive context to use or understand are not well-suited for Guerilla User Testing. The spontaneous nature of participant recruitment means that people unfamiliar with the specific technical context might not provide meaningful feedback.
- In-depth User Research Required. When a product development process necessitates detailed and in-depth user research, Guerilla User Testing might fall short. This approach is more suited for high-level, surface feedback and may not delve deeply enough into user behaviors and attitudes.
- Statistically Significant Data Needed. If the goal is to gather statistically significant data to inform decision-making, Guerilla User Testing is not the right approach. The method typically involves a small, non-representative sample size, which cannot provide the level of statistical rigor required for certain types of research.
- Sensitive or Confidential Products. Testing products that involve sensitive information or confidentiality concerns in public settings is not advisable. Guerilla User Testing lacks the controlled environment necessary to ensure privacy and data protection.
- Detailed Usability Testing. For products that require detailed and structured usability testing, such as identifying specific usability issues in a complex interface, Guerilla User Testing may be too broad and unstructured to yield the necessary insights.
- Regulated or Medical Products. Products that fall under regulatory requirements, especially in the medical or healthcare sector, should not be tested using this method. Such products require more rigorous, controlled testing environments to ensure compliance with industry standards.
- Products Requiring Long Interaction Times. If the product or feature requires extended interaction to understand or evaluate, Guerilla User Testing may not be practical. Participants in this form of testing typically engage with the product for only a short period.
After the experiment
To make sure you move forward, it is a good idea to systematically record your the insights you learned and what actions or decisions follow. We created a sample Learning Sheet, that will help you capture insights in the process of turning your product ideas successful. Download the Learning Sheet.
Examples
Discount testing
In his seminal 1994 academic article, Jakob Nielsen demonstrated that using discount usability testing methods yielded much better cost-benefit ratios over more expensive lab-based versions and that the point of diminishing returns on how many tests to carry out was around 3-5.
Source: Guerrilla HCI: Using Discount Usability Engineering to Penetrate the Intimidation Barrier
Typeform
Typeform used guerrilla testing to enhance ‘Drop-off Analysis’ feature. By iteratively testig design prototypes with real users, the speed of the technique allowd them to refine their solution based on quick feedback and in turnn improved form completion rates and the overall user experience.
Source: 5 Real-life usability testing examples & approaches to apply
This experiment is part of the Validation Patterns printed card deck
A collection of 60 product experiments that will validate your idea in a matter of days, not months. They are regularly used by product builders at companies like Google, Facebook, Dropbox, and Amazon.
Get your deck!Related plays
- Why You Only Need to Test with 5 Users by Jakob Nielsen
- The Art of Guerrilla Usability Testing by David Peter Simon
