Chapter 5: Behavioral Economics & Smoke Testing

Actions Speak Louder Than Words

When someone says "I'd buy that," they're giving you an opinion. When they actually try to buy it, they're giving you evidence.

Economists call this the difference between Stated Preference (what people say) and Revealed Preference (what people do). For validation, only revealed preference counts. This distinction, first articulated by economist Paul Samuelson in 1938, remains one of the most important concepts in customer validation. Samuelson's insight was that we can learn more about someone's preferences from observing their behavior than from asking them questions -- because behavior requires trade-offs, while opinions are free.

The gap between stated and revealed preference is well-documented. In a classic study by the Journal of Marketing Research, 45% of consumers who stated they would purchase an organic product actually did so when given the opportunity -- less than half. For novel products (like those most startups are building), the gap is even wider. The implication for founders is clear: design your experiments to observe behavior, not collect opinions.

The Commitment Spectrum

Not all skin-in-the-game commitments are equal. Daniel Kahneman's work on loss aversion shows that people weight potential losses roughly twice as heavily as equivalent gains. This means financial commitments are especially powerful validation signals -- the customer is risking money they could lose. Understanding the commitment spectrum helps you choose the right level of friction for each experiment.

The most powerful experiments combine multiple commitment types. When a customer pre-orders your product AND recommends it to three colleagues, you're seeing financial commitment plus reputational commitment -- an exceptionally strong signal. The LeanPivot Assumption Mapper helps you identify which assumptions need which level of commitment evidence to be considered validated.

Smoke Tests: The "Fake Door" Method

A smoke test offers a product that doesn't fully exist yet to see if anyone tries to buy it. It's the ultimate test of desirability. The term comes from hardware engineering, where a "smoke test" is the first test of a new circuit: plug it in and see if smoke comes out. In software validation, a smoke test plugs your value proposition into the market and sees if demand comes out.

The beauty of smoke tests is their speed and cost. While building a minimum viable product might take weeks or months, a smoke test can be set up in a weekend for under $500. If the test fails, you've saved yourself months of wasted development. If it succeeds, you have real behavioral data to inform what you build next.

Smoke Test Case Studies

These aren't hypothetical -- they're real examples of founders who used smoke tests to validate (or invalidate) demand before investing in development:

Designing Your Experiment

Every experiment needs a clear hypothesis and success criteria -- defined before you run it. This is non-negotiable. If you define success after seeing the results, you'll unconsciously adjust the threshold to match what happened -- a form of confirmation bias called "moving the goalposts." Write your hypothesis and success criteria down, share them with a co-founder or advisor, and don't change them once the experiment starts.

Example: "We believe that freelance designers will pre-order our cash flow tool ($29/month) because they're frustrated with unpredictable income. We'll know we're right if 50 people sign up in 2 weeks from $200 in Facebook ads."

Notice the specificity. The hypothesis names the audience (freelance designers), the action (pre-order), the value proposition (frustrated with unpredictable income), the metric (signups), the threshold (50), and the timeframe (2 weeks). There's no ambiguity about what success looks like. If you get 49 signups, that's a miss -- not "close enough." If you get 51, that's a hit. This level of precision is what separates rigorous validation from wishful thinking.

Setting the Right Threshold

How do you decide what threshold counts as "success"? Here are some guidelines:

• Landing page conversion: A cold traffic conversion rate of 2-5% is typical for a compelling value proposition. Below 1% suggests weak messaging or wrong audience. Above 5% suggests strong product-market fit signal.
• Email signup to engagement: If 40%+ of email signups open your first email, your audience is genuinely interested. Below 20% suggests they signed up impulsively.
• Waitlist to pre-order conversion: If 10-15% of waitlist members convert to paid when you launch, you have strong demand. Below 5% suggests the "interest" was superficial.
• Ad click-through rate: A CTR above 2% on Facebook/Instagram suggests strong message-market fit. Below 0.5% suggests your messaging doesn't resonate with the audience you're targeting.

The Anatomy of a Well-Designed Smoke Test

David Bland and Alex Osterwalder, in Testing Business Ideas, outline 44 different experiment types. But regardless of which type you choose, every well-designed smoke test shares the same structural elements. Missing any one of these increases the risk of ambiguous or misleading results:

The CTA Strength Ladder

The strength of your evidence depends on the friction of your call-to-action. This is a principle from behavioral economics: the more effort a behavior requires, the more strongly it indicates genuine motivation. A frictionless action (clicking a link) reveals almost nothing about intent. A high-friction action (entering credit card details) reveals a great deal.

Running Multiple Experiments in Parallel

One of the most common mistakes is running experiments sequentially when they could run in parallel. If you have three assumptions to test and each experiment takes two weeks, sequential testing takes six weeks. Running them in parallel takes two weeks. At the validation stage, speed matters enormously because your runway is finite.

The key constraint is that parallel experiments should test independent assumptions. If Experiment B depends on the outcome of Experiment A, they must run sequentially. But if you're testing a desirability assumption, a pricing assumption, and a channel assumption simultaneously, all three can run in parallel because they don't affect each other's results.

What You Walk Away With

• Revealed Preference Data: Evidence of what people actually do, not just say -- the gold standard of validation evidence.
• Conversion Metrics: Real numbers from your smoke test that you can compare against industry benchmarks.
• Validated (or Invalidated) Demand: Proof that people will take action for your solution, backed by behavioral data rather than opinions.
• Experiment Artifacts: Landing pages, ad copy, and results for your Lean Vault -- reusable assets for future experiments and investor conversations.
• Experiment Design Skills: The ability to write clear hypotheses, set appropriate thresholds, and design experiments that produce unambiguous results -- a skill that pays dividends throughout your entire startup journey.