Assumption Validation

Every assumption made when building products is a risk. Validate assumptions to build valuable products that solve real problems. Validate assumptions to build premium products that target customers will pay for.

Gathering Evidence: is this problem real?

Gain a deep understanding of the problem your target customer experiences before building a solution.

Anecdotal Evidence: what target customers say

Anecdotal evidence signals a problem might exist. This evidence is helpful because it's used to surface real problems, but it's not the truth, you're still making assumptions. Use anecdotal evidence to guide your investigation, avoid using anecdotal evidence to build a product. For example, you can collect anecdotal evidence using the following methodologies:

  • - Customer Feedback: customer support tickets, customer reviews, online forums (customer communities), social media posts.
  • - Qualitative Surveys (Open-Ended): asking questions such as "what can we do to improve our product?", "what features do you expect from our product?" or "let us know about your experiences with our product."
  • - User Interviews: discuss in depth what a target customer thinks, likes, or dislikes.
  • - Focus Groups: discuss in depth what a group of target customers think, like, or dislike.
  • - Stakeholder Walkthroughs: discuss a project or show a prototype to key stakeholders to collect feedback.

Real Evidence: what target customers do

Real evidence shows a real problem exists. It's okay not having this at the start, set up a research plan and, if needed, get support from a designer, user researcher, or an engineer to gather it. For example, you can collect real evidence using the following methodologies:

  • - Analytics: adoption metrics, engagement metrics (session duration, bounce rate, conversion rate), retention metrics, rage metrics (rage clicks, typing in all caps), data mining (surfacing patterns), clickstreams, drop-off points (onboarding flow, form completion, cart abandonment), performance metrics (latency), click tracking, heatmaps, etc.
  • - Quantitative Surveys (Closed-Ended): ratings, purchase probability, stack ranking, trade offs, or asking questions such as "have you done X in the last week?".
  • - Screen Recordings: monitor how target customers walk through their task flow and where they fail.
  • - Field Studies, Contextual Inquiries, or Diary Studies: observe or record how target customers attempt to achieve their goals.
  • - Task Completion Testing (Moderated or Unmoderated): ask your target customer to complete a series of tasks, monitor where they fail.

Value Validation

Solving problems creates value. After surfacing a real problem, how do you know if your solution creates value? How do you know if target customers would pay for this problem to be solved? You can learn if solving a problem creates value using the following methodologies:

No Prototype:

  • - Concept Testing: show a world where the problem is solved for your target customers, explain the simplest version of what you're planning to build, and surface if it meets the needs of your target customers.
  • - Willingness to Pay: learn if target customers are "willing to pay" for a solution to their problem before building it, get a ballpark idea of an acceptable price, learn if this price scales to meet your goals.
  • - MaxDiff Analysis: learn the most preferred and least preferred benefits your product could offer to target customers.
  • - Painted Door Test: create a "doorway" (link, button, etc.) that, when opened, notifies target customers you are working on solving this problem, measure how many target customers "open that door" to validate interest, recruit target customers for interviews or testing prototypes.
  • - Paid Waitlist: ask target customers to pay upfront for a solution that isn't built yet to validate demand.

Design-Only Prototype:

  • - Usability+Value Test: after usability testing, learn if the product solve the target customer problem: do they ask to sign up or pay, do they ask if they can use the product now, would they partner with you to test the product and provide feedback, will the target customer share the product on their social media right now?

Hybrid Prototype:

  • - Wizard of Oz Test: show a target customer a design prototype and manually “pull the strings” behind the curtain so it appears to function as if a production-ready product.

Live-Data Prototype:

  • - A/B Test: test the existing product against a prototype using 1%-50% of target customers (you do not need production-ready code), monitor key metrics to learn if the test group outperforms the control.
  • - Invite-Only Test: if you lack the users to reach the statistical significance of an A/B test, then invite a limited set of users to test the prototype.

Usability Validation

Users need to know how to use your product. Show a target customer a design prototype, can they use it to solve their problem? Does friction appear while using the prototype? End this interview with a value test.

Feasibility Validation

Your team needs to be able to build the product and you have deadlines. Discuss the project with your engineers, can they build it with their skillset? How much effort do they estimate it will take?

Viability Validation

Your product needs to be viable for the business. Walk through the problem and solution with your stakeholders. Address viability concerns from sales, finance, legal, security, marketing, or other teams working with you.