Chapter 4: Quantitative Verification

From Stories to Numbers

Interviews tell you why. Surveys tell you how many. You need both.

Qualitative discovery gives you insight -- the stories, emotions, and context behind customer behavior. Quantitative verification gives you evidence -- proof that these patterns exist at scale. The relationship between qualitative and quantitative research is sequential and complementary: qualitative research tells you what to ask about, and quantitative research tells you how widespread it is.

Too many founders make one of two mistakes: they either skip qualitative research entirely and go straight to surveys (producing precisely wrong answers to the wrong questions), or they rely solely on interviews and never verify that their qualitative patterns hold up across a larger sample. The ideal validation process alternates between qualitative and quantitative phases: discover patterns through interviews, verify them through surveys and experiments, then go back to interviews to understand the surprising results.

The Sean Ellis Test (Product-Market Fit)

Sean Ellis, who coined "growth hacking," created the most widely-used measure of product-market fit. It's one question that has been validated across thousands of startups and consistently predicts which products will achieve sustainable growth.

The Sean Ellis Test works because it measures dependence rather than satisfaction. There's a critical difference: a customer can be "satisfied" with a product they'd never miss if it disappeared. The "very disappointed" response indicates that your product has become part of their workflow in a way that would be genuinely painful to undo. That's the hallmark of product-market fit.

How Superhuman Used the Sean Ellis Test

Superhuman, the premium email client, is perhaps the most famous practitioner of the Sean Ellis framework. When CEO Rahul Vohra first surveyed users, only 22% said they'd be "very disappointed" -- well below the 40% threshold. Instead of scaling, the team segmented the responses. They found that users who matched a specific persona (founders, executives, and professionals who lived in their inbox) had a much higher "very disappointed" rate. Superhuman then narrowed their focus to that segment, built features specifically for them, and gradually increased their score to above 58%. Only then did they begin scaling. The lesson: the Sean Ellis score isn't just a pass/fail -- it's a compass that tells you which segment to serve and which features to build.

Pricing Research: Test Before You Build

Pricing isn't an afterthought -- it's a product feature. You must test pricing early to understand if your business model is viable. Many founders leave pricing until after they've built the product, which means they're essentially gambling that customers will pay a price that makes the unit economics work. Test pricing early to avoid this trap. For deeper pricing strategy, see Playbook 05: Pricing Strategy.

Van Westendorp Price Sensitivity Meter

The Van Westendorp method, developed by Dutch economist Peter van Westendorp in 1976, uses four questions to find your acceptable price range. It's particularly effective for new products because it captures the psychological boundaries of pricing without requiring respondents to commit to a specific number.

When you plot the cumulative curves of these four data points, the intersections reveal key price points. The intersection of "too cheap" and "too expensive" gives you the optimal price point. The intersection of "bargain" and "expensive" gives you the indifference price point (where an equal number of respondents consider it cheap vs. expensive). Most importantly, the range between these two intersections is your acceptable price window.

The "Wallet Test": Real-World Pricing Validation

The most reliable pricing test isn't a survey -- it's a real transaction. Here are three ways to test pricing with real money:

Survey Design Principles

Bad surveys lead to bad decisions. Here's how to avoid common mistakes. The first principle of survey design is that every question should be there for a reason -- if you can't articulate what decision a question's answer will inform, delete the question. Short, focused surveys get higher response rates and better data than comprehensive questionnaires that try to answer everything at once.

Survey Distribution and Sample Quality

Where you distribute your survey matters as much as what you ask. A beautifully designed survey sent to the wrong audience produces misleading data. Here are key principles for survey distribution:

• Target the right segment: If your product is for freelance designers, don't survey "freelancers" broadly. The responses from freelance writers and freelance developers will dilute your signal.
• Use screening questions: Start with 1-2 questions that filter out respondents who don't match your target profile. Better to have 50 qualified responses than 500 from a mixed audience.
• Beware self-selection bias: People who respond to surveys are inherently different from those who don't. They tend to have stronger opinions, more free time, and more interest in your topic. Account for this in your interpretation.
• Consider incentives carefully: Incentives increase response rates but can attract people who are motivated by the reward rather than the topic. Keep incentives modest and relevant.

The LeanPivot Insight Survey Builder helps you design bias-free surveys with proper screening questions, randomized answer orders, and built-in statistical validity checks.

Statistical Reality Check

Don't be fooled by small sample sizes. Early-stage founders consistently overinterpret small datasets, finding patterns in what is actually random noise.

Quick Statistical Reference

Don't let the pursuit of statistical perfection become an excuse for inaction. At the validation stage, directional accuracy (30+ responses) is usually sufficient. You're trying to determine if you're in the right ballpark, not calculate the third decimal place. Save the precision for when you're optimizing, not exploring.

What You Walk Away With

• Sean Ellis Score: A number that tells you if you have product-market fit -- and guidance on how to improve it if you don't.
• Pricing Validation: Evidence of what customers will actually pay, tested through both survey instruments and real-world transactions.
• Survey Results: Quantitative proof that your qualitative insights scale across your target segment.
• Statistical Awareness: Understanding of your margin of error and how much confidence your sample size supports.