Chapter 3: Pre-MVP Validation Techniques
Wizard of Oz, Concierge MVP, and Fake Door testing.
The Most Efficient Code is Code You Never Write
Most founders skip straight to building. They assume the only way to test an idea is to create a working product. This is the most expensive mistake in entrepreneurship.
"Pretotyping" (a term coined by Alberto Savoia) involves simulating the core experience of a product to validate demand and usage patterns with near-zero engineering effort. The goal: answer the question "Should we build this?" before asking "Can we build this?"
The concept is deceptively simple, yet most founders resist it. They argue that their product is too complex to simulate, that their customers expect a polished experience, or that manual processes don't scale. All of these objections miss the point. Pretotyping isn't about building a business -- it's about answering a question. The question is: "If this product existed exactly as we envision it, would people use it and pay for it?" If the answer is no, then the sophistication of your technology is irrelevant. You're building the wrong thing.
Consider the cost asymmetry: a typical pretotyping experiment costs between $100 and $1,000 and takes 1-2 weeks. A typical MVP costs between $25,000 and $150,000 and takes 3-6 months. If the pretotype reveals that your core assumption is wrong, you've saved five figures and months of your life. If it validates your assumption, you now have the confidence and customer insights to build a much better MVP. There is no scenario in which pretotyping first makes you worse off.
The Core Insight
You can test whether customers will pay for your solution, use your solution, and love your solution -- all without writing a single line of code. This chapter shows you how.
The Three Pretotyping Techniques
Each technique has its place. The key is matching the technique to the assumption you're testing:
Wizard of Oz
Best for: Testing if your solution works
Users think it's automated; humans do the work behind the scenes
Concierge
Best for: Deep problem discovery
Explicit high-touch service; learn exactly what customers need
Fake Door
Best for: Testing demand at scale
Measure real intent without building anything
Technique #1: The Wizard of Oz
Create a highly polished, functional-looking interface where humans secretly do the work that would eventually be automated. The user believes they're interacting with technology.
The name comes from the 1939 film: behind the impressive facade of the "Great and Powerful Oz" was just a man pulling levers. Similarly, behind the impressive facade of your "AI-powered" or "automated" product is a team of people doing the work manually. The user sees the output and judges the value, blissfully unaware that there's no technology behind it.
The Bug
"We need to build the AI before we can test if customers want it."
Building complex technology (AI, ML, automation) takes months. If customers don't want what it produces, those months are wasted.
The Fix
Use Wizard of Oz to fake the automation.
Build only the interface. Have humans do the "AI" work behind the scenes. If customers love the output, then build the real tech.
Case Study: Aardvark -- Acquired by Google for $50M
Aardvark was a social search engine where users could ask questions and get answers from their network. The founders needed to validate that users would trust the service.
The Wizard of Oz approach: Instead of building complex routing algorithms, they employed interns to manually search for answers and route questions to the right experts. Users thought it was automated.
Result: They validated that users wanted the service before investing in the AI. Google acquired them for $50M.
Case Study: Cardmunch -- The Manual Image Recognition Engine
Cardmunch let users photograph business cards and have the contact info automatically digitized. The founders claimed to use "advanced OCR technology" to parse business card images.
The reality: Behind the scenes, photographs were routed to Amazon Mechanical Turk workers who manually transcribed the information. The "OCR engine" was thousands of humans typing in contact details.
Result: By the time they needed to build real technology, they had validated user demand, understood edge cases (international formats, handwritten cards, damaged cards), and built a training dataset for their eventual machine learning model. LinkedIn acquired the company.
When to Use Wizard of Oz
Perfect For
- AI/ML-powered features
- Recommendation engines
- Automated workflows
- Smart assistants
- Personalization systems
- Complex matching algorithms
- Natural language processing
- Predictive features
How to Run a Wizard of Oz Experiment
Step-by-Step Process
- Build the interface only. Create a functional-looking frontend with forms, buttons, and outputs -- but no backend logic. Use LeanPivot's Prototype Generator to quickly create realistic mockups.
- Set up the "curtain." Create a system for humans to receive user inputs (email, Slack, dashboard) and provide outputs.
- Recruit 5-20 pilot users. Enough to learn, not enough to overwhelm your manual process.
- Operate manually. When users submit requests, humans fulfill them and make it look automated.
- Measure and learn. Track usage patterns, satisfaction, and -- critically -- willingness to pay.
- Decide: If validated, build the automation. If not, pivot before wasting engineering time.
A practical tip for Wizard of Oz experiments: set response time expectations that match what the automated version would deliver, not what is convenient for your manual team. If the future AI will respond in 30 seconds, your human operators should respond in approximately 30 seconds. If the future system would deliver results overnight, deliver results overnight. Testing with unrealistically fast or slow response times compromises the validity of the experiment because response time itself affects the user's perception of value.
Ethical Considerations
Be thoughtful about deception. Some argue users should be informed it's "powered by our team" during beta. Others argue the test is invalid if they know. At minimum, ensure the output quality matches what the automated version would deliver. Never collect sensitive personal data under false pretenses, and always comply with relevant privacy regulations regardless of your experiment format.
Technique #2: The Concierge MVP
Similar to Wizard of Oz in its manual execution, but with a critical difference: the human involvement is explicit. You deliver the value as a high-touch service, openly acknowledging that it's not yet automated.
The Concierge MVP is particularly powerful for startups in the discovery phase because it generates the richest qualitative data of any pretotyping technique. When you manually deliver the value, you see every facial expression, hear every hesitation, and witness every workaround. This intimacy with the customer experience is impossible to replicate through surveys, analytics, or even formal user interviews.
The Bug
"We think we know what customers need. Let's build it."
You've never actually delivered the value you're planning to automate. You're guessing at the workflow, the pain points, the edge cases.
The Fix
Become the product yourself.
Deliver the value manually to 5-10 customers. Stand next to them. Watch them struggle. Learn exactly what they need before you automate anything.
Case Study: Food on the Table
Manuel Rosso founded a meal planning service. Instead of building an app, he started with one customer.
The Concierge approach: He visited the customer's home, reviewed her preferences, went to her local grocery store, and manually planned her meals for the week. He was the app.
What he learned: Exactly how customers make decisions, which constraints matter, and what features were essential. This informed every product decision. The company eventually raised $5M.
Case Study: Wealthfront's Concierge Origins
Before building automated investment software, Wealthfront's founders manually managed investment portfolios for a small group of clients. They met with each client, discussed risk tolerance, manually rebalanced portfolios, and hand-calculated tax-loss harvesting opportunities.
What they discovered: The manual process revealed that clients cared far more about tax efficiency than portfolio optimization -- a counterintuitive insight that shaped the entire product strategy. Their tax-loss harvesting feature became the primary differentiator, something they would never have prioritized without the concierge experience.
The lesson: The concierge phase doesn't just validate demand -- it reveals which aspects of the service deliver the most value, which informs what to automate first.
When to Use Concierge
Perfect For
- Problem discovery: You're not sure exactly what customers need
- Service businesses: The value is partly human expertise
- Complex workflows: You need to understand edge cases before automating
- B2B products: High-touch sales and delivery are expected
- Premium positioning: "White glove" service can command higher prices
The Concierge Learning Framework
Every concierge interaction should generate learning. Use this framework:
After Each Customer Interaction
| Friction Points: | Where did they get confused or frustrated? |
| Delighters: | What made them unexpectedly happy? |
| Edge Cases: | What scenarios didn't we anticipate? |
| Feature Requests: | What did they ask for that we don't offer? |
| Willingness to Pay: | Did they pay? Would they pay more? |
| Referral Intent: | Would they recommend this to a colleague? |
Keep a structured log of every concierge interaction. After 10-15 interactions, patterns will emerge. You'll notice that certain pain points come up in every session while others are unique to individual customers. The recurring patterns are your "must have" features. The unique ones are "nice to have" at best. This data is worth more than any feature prioritization framework because it comes directly from observing real behavior, not from asking hypothetical questions.
The Concierge-to-Product Transition
The goal of the concierge phase is not to run a services business forever -- it's to learn enough to build the right product. Here's how to know when to transition:
Transition Signals
- Pattern saturation: New interactions stop revealing new insights. You've seen every major edge case.
- Process stability: Your manual workflow has stabilized. You do roughly the same steps for each customer.
- Demand exceeds capacity: More people want the service than you can manually handle.
- Unit economics clarity: You understand the cost to serve each customer and what they'll pay.
Technique #3: The Fake Door (Smoke Test)
Create a marketing asset (landing page, ad, or button) for a feature that doesn't exist. When users click "Buy" or "Sign Up," show them a waitlist or "Coming Soon" message. This measures actual behavior -- not what people say they'd do.
The Fake Door technique exploits a fundamental truth about human psychology: what people say and what people do are often very different. Surveys and interviews measure stated preferences. Fake Door tests measure revealed preferences. A user who clicks "Buy Now" on a landing page and enters their credit card information is demonstrating genuine intent in a way that no survey response ever could.
The Bug
"Everyone we talked to said they'd buy it."
People are polite. They say yes to hypothetical purchases. But stated intent does not equal actual behavior. You need to measure what they DO, not what they SAY.
The Fix
Run a Fake Door test to measure real intent.
Create a landing page with a "Buy Now" button. Run ads. See how many people actually click -- and ideally, enter payment info before you reveal the waitlist.
How to Run a Fake Door Experiment
Step-by-Step Process
- Create a landing page that clearly describes your value proposition, pricing, and includes a prominent CTA button ("Buy Now," "Start Free Trial," etc.)
- Drive targeted traffic using paid ads ($200-500 is enough for initial signal) to your ideal customer segment
- On CTA click, show a waitlist page: "Thanks for your interest! We're currently in private beta. Enter your email to get early access."
- Measure conversion rates: Visitor to CTA Click, CTA Click to Email Submit
- For stronger signal: Take users to a checkout flow before revealing the waitlist (measures intent to pay, not just interest)
A crucial detail that many founders miss: the quality of your landing page directly affects the validity of your test. If your page looks amateur, low conversion rates might indicate poor marketing rather than weak demand. Conversely, if your page uses misleading or exaggerated claims, high conversion rates might not represent sustainable demand. The landing page should honestly represent your value proposition with professional-quality design. You're testing the offer, not your design skills.
LeanPivot's Market Signal Test tool can help you design and analyze your fake door experiment, including recommended ad spend, audience targeting, and statistical significance calculations based on your business model.
Fake Door Benchmark Thresholds
What "Good" Looks Like
| Metric | Weak Signal | Moderate Signal | Strong Signal |
|---|---|---|---|
| Landing to CTA Click | <2% | 2-5% | >5% |
| CTA to Email Submit | <20% | 20-40% | >40% |
| CTA to Credit Card Entry | <1% | 1-3% | >3% |
Note: Benchmarks vary by industry and price point. B2B typically has lower volume but higher intent.
Interpreting Fake Door results requires nuance. A 3% CTA click rate might seem low in absolute terms, but if you're selling a $500/month B2B product, that conversion rate could represent a viable business. Meanwhile, a 10% click rate for a $2.99 consumer app might be insufficient if the lifetime value is too low to justify customer acquisition costs. Always interpret conversion rates in the context of your unit economics.
Advanced Fake Door: Multi-Variant Testing
Once you've validated basic demand, you can use Fake Door tests to answer more specific questions:
Multi-Variant Fake Door Tests
| Question | Test Design | What to Measure |
|---|---|---|
| Optimal pricing | 3 landing pages with different price points ($19, $29, $49) | CTA click rate at each price |
| Best positioning | 3 landing pages with different value propositions | Which framing converts best |
| Target segment | Same page, different ad audiences | Which segment converts highest |
| Feature importance | Feature-specific pages vs. bundled page | Which features drive conversion |
The Limitation of Fake Door
Fake Door tests measure interest, not utility or retention. They tell you if people want to buy -- not if they'll keep using. Use Fake Door for demand validation, then follow up with Wizard of Oz or Concierge to validate the experience.
Choosing the Right Technique
Match your pretotyping technique to the assumption you need to test:
| What You're Testing | Best Technique | Why |
|---|---|---|
| "Is there demand for this?" | Fake Door | Measures actual purchase intent at scale |
| "What price will they pay?" | Fake Door + A/B | Test different price points on landing pages |
| "Will the solution work?" | Wizard of Oz | Test the full experience without building backend |
| "What do they actually need?" | Concierge | Deep discovery through high-touch service |
| "Will they keep using it?" | Wizard of Oz | Measure retention over multiple interactions |
| "What features matter?" | Concierge | Learn from manual delivery before automating |
The Pretotyping Decision Tree
Quick Decision Guide
- Start with Fake Door to validate basic demand exists
- If demand is validated -- Run Concierge with 5-10 customers to deeply understand needs
- Once you understand needs -- Run Wizard of Oz to test the specific solution at scale
- If all three pass -- NOW you build with confidence
This sequential approach -- Fake Door, then Concierge, then Wizard of Oz -- represents increasing investment at each stage. You start with the cheapest, fastest test (Fake Door) and only proceed to more expensive, slower tests (Concierge, then Wizard of Oz) after each previous stage validates. Think of it as a funnel of confidence: each stage narrows your uncertainty and increases your conviction before you invest more resources.
The entire pretotyping sequence -- from Fake Door to Concierge to Wizard of Oz -- can be completed in 4-8 weeks with a total budget of $1,000-$5,000. Compare that to the typical "build first, ask questions later" approach of 3-6 months and $50,000-$150,000. The pretotyping path delivers more learning at a fraction of the cost, and it produces the specific insights you need to build a product that customers actually want.
Key Takeaways
Remember These Truths
- The best code is code you never write. Pretotyping lets you validate before engineering.
- Fake Door tests interest. Use it to measure demand before building anything.
- Wizard of Oz tests the solution. Fake the automation to validate the experience.
- Concierge drives discovery. Become the product to learn what to build.
- Match technique to assumption. Different questions require different tests.
Now that you can validate demand without building, let's explore how to prioritize features and make build-vs-buy decisions when you do start developing.
Design Your Pretotype Experiment with AI
Plan and track your validation experiments -- whether Wizard of Oz, Concierge, or Fake Door -- with our AI-powered experiment design tools.
Save Your Progress
Create a free account to save your reading progress, bookmark chapters, and unlock Playbooks 04-08 (MVP, Launch, Growth & Funding).
Ready to Design Your MVP?
LeanPivot.ai provides 80+ AI-powered tools to design, build, and launch your MVP.
Start Free TodayRelated Guides
Lean Startup Guide
Master the build-measure-learn loop and the foundations of validated learning to build products people actually want.
From Layoff to Launch
A step-by-step guide to turning industry expertise into a thriving professional practice after a layoff.
Fintech Playbook
Master regulatory moats, ledger architecture, and BaaS partnerships to build successful fintech products.
Works Cited & Recommended Reading
RAT vs MVP Philosophy
- 1. Ries, E. (2011). The Lean Startup. Crown Business.
- 2. "Why RAT (Riskiest Assumption Test) beats MVP every time." LinkedIn
- 3. "Pretotyping: The Art of Innovation." Pretotyping.org
- 6. "Continuous Discovery: Product Trio." Product Talk
- 7. "MVP Fidelity Spectrum Guide." SVPG
Minimum Lovable Product
- 8. Olsen, D. (2015). The Lean Product Playbook. Wiley.
- 9. "From MVP to MLP: Why 'Viable' Is No Longer Enough." First Round Review
- 10. "Minimum Lovable Product framework." Amplitude Blog
Hypothesis-Driven Development
- 11. Gothelf, J. & Seiden, J. (2021). Lean UX. O'Reilly Media.
- 12. "Hypothesis-Driven Development in Practice." ThoughtWorks Insights
- 13. "Experiment Tracking Best Practices." Optimizely
- 14. "Build-Measure-Learn: The Scientific Method for Startups." Harvard Business Review
Assumption Mapping
- 15. Bland, D. & Osterwalder, A. (2019). Testing Business Ideas. Wiley.
- 16. "Risk vs. Knowledge Matrix." Miro Templates
- 17. "Identifying Riskiest Assumptions." Intercom Blog
User Story & Impact Mapping
- 20. Patton, J. (2014). User Story Mapping. O'Reilly Media.
- 21. Adzic, G. (2012). Impact Mapping. Provoking Thoughts.
- 22. "Jobs-to-Be-Done Story Framework." JTBD.info
- 23. "The INVEST Criteria for User Stories." Agile Alliance
- 24. "North Star Metric Framework." Amplitude
- 25. "Opportunity Solution Trees." Product Talk
- 26. Torres, T. (2021). Continuous Discovery Habits. Product Talk LLC.
Pretotyping Techniques
- 27. Savoia, A. (2019). The Right It. HarperOne.
- 28. "Fake Door Testing Guide." UserTesting
- 29. "Wizard of Oz Testing Method." Nielsen Norman Group
- 30. "Concierge MVP Explained." Grasshopper
Prioritization Frameworks
- 31. "ICE Scoring Model." ProductPlan
- 32. "RICE Prioritization Framework." Intercom
- 33. "Kano Model for Feature Analysis." Folding Burritos
- 34. "MoSCoW Method Guide." ProductPlan
Build vs Buy & No-Code
- 35. "No-Code MVP Tools Landscape." Makerpad
- 37. "Technical Debt in Early Startups." a16z
- 38. "Prototype Fidelity Selection." Interaction Design Foundation
- 39. "API-First Development Strategy." Swagger
- 40. "Rapid Prototyping with Bubble & Webflow." Bubble Blog
Metrics & Analytics
- 41. Croll, A. & Yoskovitz, B. (2013). Lean Analytics. O'Reilly.
- 42. "One Metric That Matters (OMTM)." Lean Analytics
- 43. McClure, D. "Pirate Metrics (AARRR)." 500 Startups
- 44. "Vanity Metrics vs. Actionable Metrics." Mixpanel
- 45. "Cohort Analysis Deep Dive." Amplitude
- 46. "A/B Testing Statistical Significance." Optimizely
- 47. "Product Analytics Instrumentation." Segment Academy
- 48. "Activation Metrics Framework." Reforge
- 49. "Leading vs Lagging Indicators." Productboard
- 50. "Retention Curve Analysis." Sequoia Capital
- 51. "Feature Adoption Tracking." Pendo
- 52. "Experimentation Velocity Metrics." ExP Platform
Launch Operations & Analysis
- 53. "Soft Launch Strategy." Mind the Product
- 54. "Feature Flag Best Practices." LaunchDarkly
- 55. "Beta Testing Program Design." BetaList
- 56. "Customer Feedback Loop Systems." Canny
- 57. "Rollback Strategy Planning." Atlassian
- 58. "Why Startups Fail: Post-Mortems." CB Insights
- 59. "Pivot vs Persevere Decisions." Steve Blank
- 60. "Learning from Failed Experiments." HBR Innovation
This playbook synthesizes methodologies from Lean Startup, Design Thinking, Jobs-to-Be-Done, Pretotyping, and modern product management practices. References are provided for deeper exploration of each topic.