Synthetic user vendors advertise 85 to 92 percent parity with real interviews. Stanford and Google DeepMind built a thousand AI agents from two-hour interviews and matched human survey answers at 85 percent. The numbers are real, and they are measured on the vendor's own panel, on thematic overlap, under conditions that flatter the method.
Run synthetic research against a known ICP for a live product decision and the honest number drops. Count only the insights that change what you build, and synthetic interviews reliably surface about half. Not 85 percent. Half.
That half is still worth having. It arrives in minutes, costs a few dollars, and tells you which questions are worth a human's time. The mistake is treating the first half as the whole picture, or skipping it because it isn't.
Below is the six-step checklist Product Map uses to bank the synthetic half on purpose: define the ICP, plan the research, build the guide, generate the personas, run the interviews with subagents, and turn the transcripts into a strategic plan. Every step runs on an agent you can open today in the AI Agents Catalog.
The insights half synthetic interviews reliably reach
A synthetic user is a language model answering as a persona built from your data. Ask it about a workflow and it returns something close to what an average person matching that description would say. On a specific band of questions, close is enough.
- Theme coverage: the recurring problems, the vocabulary customers use, the jobs they're hiring your product for. Synthetic interviews surface these fast and rank them in a sane order.
- Message and concept reactions: a first read on whether positioning lands, which value prop leads, where a pitch confuses.
- Guide rehearsal: running your interview script against a synthetic ICP before a real call, so you cut the dead questions and sharpen the probes.
- Coverage at zero access: a directional read on a segment you can't book this week, enough to form a hypothesis worth testing.
Each of these is pattern-matching against what's already known and written down somewhere. That's exactly what a well-grounded model does well. The forty-plus percent of researchers now using synthetic data for early-stage idea screening are using it for this half, and they're right to.
The half a model can't reach
The other half is where products get decided, and it's the half a synthetic user sands off.
Real customers contradict themselves. They say retention matters, then churn over a price change they swore they didn't care about. They describe a budget they don't control, a workaround they're embarrassed by, a competing tool they'll never admit to in a survey. A synthetic user has no last Tuesday, no boss, no invoice. It won't surprise you, and the surprise is the signal.
A synthetic interview can tell you whether your questions are clear. It cannot tell you whether your customer exists.
Synthetic research also produces zero behavioral data. A model can describe using your product; it can't fumble the onboarding, rage-click a broken state, or abandon a flow at step four. The 8 to 15 percent gap the vendors report is the emotional nuance and personal anecdote. The gap you feel in practice is bigger, because the missing pieces are the ones that overturn a roadmap rather than confirm it.
Before the checklist: grounding decides the number
Whether you get the full first half or a thin quarter comes down to one variable: what you ground the synthetic user on.
Personas built from demographics alone are guesswork with a headshot. Personas built bottom-up from real interview transcripts matched real survey responses at roughly 85 percent in the Stanford work, and collapsed toward noise when the transcripts were removed. So before step one, put your evidence where agents can read it: commit real transcripts, support tickets, and prior research as plain files in a version-controlled product context repository. Every agent in this checklist reads from and writes to that repo, so the work compounds instead of resetting between steps.
With the corpus in place, run the six steps in order. Each one produces a file the next step consumes.
Step 1: Define your ICP with the Product Map ICP agent
Open the Define ICP and segments agent and point it at your corpus. It reads the evidence and drafts segment definitions from what customers said, not from demographic guesses. The output is an ICP file you commit to the repo; every downstream agent builds on it.
For a B2C or PLG motion, the ICP file follows this template:
# ICP — B2C (individual users)
## Profile
[2 to 3 sentences: who they are, what they're trying to
accomplish, what a typical day looks like]
## Attributes
- Role / title: [Product Manager, Senior PM, founder acting as PM]
- Seniority / life stage: [mid-level, career switcher, student]
- Context: [startup employee, freelancer, side project]
- Goals: [what this person wants to achieve in daily work]
- Frustrations: [what slows them down or causes stress]
- Tools used: [Notion, ChatGPT, Slack, tools they already pay for]
- Where they discover: [Reddit, newsletters, word of mouth, SEO]
- Willingness to pay: [self-pay ceiling, monthly vs annual]
## Behavioral signals
- [signs up without a sales call and completes onboarding alone]
- [returns within the first 7 days and forms a habit loop]
- [already pays for adjacent prosumer tools]
## Jobs to be done
- When [situation], I want to [motivation], so I can [outcome].
## Disqualifiers
- [only evaluates via procurement and never self-serves]
- [needs enterprise SSO or multi-seat admin before trying]This step is done when the ICP file is filled in, committed to the repo, and every attribute traces back to something a real customer said or did.
Step 2: Plan the research before writing questions
Before drafting a single question, decide what the research needs to answer. The user research planner reads your ICP and your open product questions, then picks the method that fits: generative interviews when you're hunting for problems, evaluative when you're testing a solution, a PMF conversation when you need to know if the pull is real.
The plan should name three things: the decision this research feeds, the segment from your ICP you're studying, and the number of interviews you'll run. For a synthetic round, eight to twelve interviews per segment is enough; beyond that, the model starts repeating itself.
This step is done when the research plan names a decision, a segment, and an interview count, and it's committed next to the ICP.
Step 3: Build the interview guide around your goals
Now turn the plan into a script. The customer interview planner drafts the guide from your research goals: opening context questions, the core problem or solution block, and the probes to run when an answer is vague. Because it reads the ICP from step one, the questions use your customer's vocabulary instead of generic research-speak.
Rehearse the guide once before spending it on humans. Run it against the synthetic ICP and watch where the answers go flat; a question the model answers in one bland sentence will bore a real customer too. Cut the dead questions, sharpen the probes, and keep the guide under forty minutes of material.
This step is done when the guide is committed, rehearsed once, and trimmed of every question with an obvious answer.
Step 4: Generate synthetic users grounded in your ICP
One ICP is a definition; an interview needs a person. This step generates a persona card for each planned interview: a named character with a specific age, role, company context, tool stack, and the quirks real people have. One persona is skeptical of AI tools. Another is three weeks into a new job. A third tried a competitor and left. The variation matters, because ten interviews with the same average customer produce one insight ten times.
Each card stays inside the boundaries of the ICP but fills in the human detail the ICP leaves open. Ground the cards in your corpus: where a real transcript mentions a workflow or a complaint, borrow it, so the persona speaks with facts from your customers in view.
This step is done when there's one committed persona card per planned interview, each distinct, each traceable to the ICP.
Step 5: Spawn one subagent per interview transcript
Here's where the checklist stops being a document exercise. Open the repo in Claude Code or Cursor and spawn a subagent per interview. Each subagent gets two inputs, the interview guide and one persona card, and one job: role-play the interview and write a realistic Markdown transcript into the interviews folder, hesitations, tangents, and contradictions included.
Because the subagents run in parallel, ten interviews finish in minutes. Because each subagent holds only its own persona, the transcripts stay independent; interview seven doesn't drift toward what interview three said.
Two rules keep this honest. Name every file so it's unmistakably synthetic, a synthetic- prefix works, and never let a synthetic transcript sit in the same folder as real ones without the label. A synthetic quote laundered into a real evidence base is the failure mode of this entire method.
This step is done when every persona has a labeled transcript in the repo and none of them can be mistaken for a real interview.
Step 6: Cluster insights into an opportunity solution tree
Transcripts aren't insights. The last step is analysis: point the product discovery agent at the interviews folder and ask it to find the themes.
It reads the whole corpus, not the twelve interviews you had time to re-read. It connects a complaint from March to a churn reason from June. It notices the need three personas described in three different words. Then it clusters those into opportunity areas mapped to the structure Teresa Torres calls an opportunity solution tree, ranks them by frequency and severity, and pins the verbatim quote to every branch so you can check its work.
Because the corpus is a repository, the output is a change you can review. The agent proposes updates to your product context files: a new opportunity added, a persona refined, a synthesis doc corrected. You read the diff, keep what holds up, and commit it. What comes out the other end is a prioritized opportunity map, which is the raw material of a strategic plan: the top opportunities become the questions your real interviews must answer next.
This step is done when the opportunity tree is committed, every branch has a source quote, and the top three opportunities have a named next step.
The question nobody answers: who re-grounds the corpus
Here's what most synthetic-research writeups skip, and what decides whether this checklist works past the first month: who owns re-grounding.
Synthetic parity is not a fixed property. It decays. Your product ships new surface area, your market shifts, a competitor changes the reference point, and the transcripts you grounded on describe a world that no longer exists. The model keeps answering with total confidence off stale evidence, and the first-half insights quietly turn wrong. Nobody gets an error. You start making decisions for a customer who used to exist.
Two disciplines keep the number honest.
- Re-ground on a cadence. Treat new research like a dependency you update. Every real interview round lands in the same repo, and the next synthetic round reads the current corpus, not last quarter's. Assign the job to a person, not a vibe.
- Label synthetic as rehearsal. Anything a synthetic user produced is marked as first thought and support, never filed as evidence. A synthetic transcript that shapes a roadmap without a real interview behind it is the failure mode, and a version-controlled corpus makes it easy to see which claims trace back to a real human and which don't.
Run the checklist first, then book the real interviews
Synthetic research changes the economics of the early work, not the responsibility for the decision. Run the six steps and you'll surface the known half of your customer's world in an afternoon, for the price of a coffee, grounded on evidence you already own. That's a real edge, and skipping it to protect a purist stance on "only real research" wastes it.
The teams getting this right run the checklist in order. Ground the corpus, define the ICP from evidence, plan the research, rehearse the guide, generate distinct personas, run the interviews as parallel subagents, and cluster the output into an opportunity tree. Then spend every real interview on the contradictions, the budgets, and the behavior a model will never reach.
Browse the AI Agents Catalog to see the full set of discovery agents; the six above are enough to bank the first half before you book a single call.
