Technology

How a Research Agent Runs a Concept Test: From Brief to Go/No-Go

How a Research Agent Runs a Concept Test: From Brief to Go/No-Go

How a Research Agent Runs a Concept Test: From Brief to Go/No-Go

Concept testing with a research agent is when an AI agent runs a concept test end to end with real target respondents. The agent drafts the study plan, screener and discussion guide, recruits respondents, moderates the interviews, checks response quality, and drafts a go, refine or no-go recommendation. The research team approves the decision rule, the study design, the full launch and the final call.

4 best practices for agile usability testing
4 best practices for agile usability testing
4 best practices for agile usability testing

Tag

Technology

Date

Read Time

10 Min

Content

Senior Growth Marketer

Summary:

  • In concept testing with a research agent, an AI agent plans the study, recruits real respondents, runs interviews, checks quality and drafts the decision.

  • It matters because concept decisions are costly, and agents compress weeks of work while the team keeps control.

  • The process runs ten steps with four approval gates: the action standard, study materials, full fieldwork and the final call.

  • Set the decision rule first, audit the agent's work and keep the final call with your team.


Concept testing with a research agent is a way of running a concept test where an AI agent does most of the work and your team approves the decisions that matter. The agent drafts the study plan, screener and discussion guide, recruits real target respondents, moderates the interviews, checks response quality and drafts a go, refine or no-go recommendation. The team approves four gates: the decision rule, the study design, the move to full fieldwork and the final call.

Two things this is not. It is not a desk-research agent that searches the web and summarizes what it finds, the kind covered in our complete guide to AI research agents. And it is not a panel of synthetic respondents answering on behalf of real people. This is primary research with real category buyers.

The stakes are high. McKinsey research found that more than 50% of product launches fail to hit their business targets, even though new products generate more than 25% of total revenue and profits across industries. A better concept decision earlier in the process is one of the cheapest ways to improve those odds.

To make the process concrete, this guide follows one example from start to finish: a D2C beverage brand testing two positionings for the same new drink. All numbers in the example are illustrative.

What a Research Agent Does, and What the Team Approves

It helps to separate three approaches that are often confused. The difference comes down to who designs the study, who answers it and who decides.


Research agent

AI moderator

Synthetic respondents

Who designs the study

Agent drafts, team approves

Researcher designs

Researcher designs

Who answers

Real respondents

Real respondents

AI-generated personas

Who decides

Team, using the agent's draft memo

Researcher

Researcher

Best for

End-to-end studies with human sign-off

Running interviews at scale

Early idea screening

A research agent covers the whole study. An AI moderator covers the interviews inside it. For a deeper look at how agents work in this field, see our overview of AI agents in consumer research.

The Ten Steps and Four Gates

Step

What the agent does

What the team approves

1. Brief intake

Restates the decision, flags gaps

The brief

2. Action standard

Proposes go/refine/no-go criteria

Gate 1: the decision rule

3. Study plan

Proposes design, cells, quotas and sample size

Included in Gate 2

4. Screener

Drafts qualifying and terminate logic

Gate 2: screener, guide and stimulus

5. Guide and stimulus

Drafts questions and probe rules

Included in Gate 2

6. Recruiting

Sets quotas and tracks fill

Panel and incentive choices

7. Pilot

Runs 5-10 interviews, reports issues

Gate 3: move to full fieldwork

8. Fieldwork and QA

Moderates, flags and replaces bad responses

Spot checks

9. Synthesis

Scores cells, codes themes, links quotes

Review of evidence

10. Decision memo

Drafts the recommendation

Gate 4: the final call

Agentic AI is still early, which is why these gates matter. Gartner predicts that over 40% of agentic AI projects will be canceled by the end of 2027, and estimates that only about 130 of the thousands of agentic AI vendors are real. Human-in-the-loop research keeps the agent's speed while keeping accountability with people.

A Worked Example: One Concept, from Brief to Decision

The example brand is a D2C beverage company launching a lightly sparkling drink with natural botanicals. It wants to know which positioning to take to market:

  • Cell A, "Calm focus": a drink that helps you stay clear-headed through the afternoon.

  • Cell B, "Natural lift": a drink that gives you gentle energy without a crash.

Each respondent sees only one positioning. Every number below is illustrative, not from a live study.

Step 1: The Brief the Team Writes

The team writes five lines:

Decision: Choose one positioning for launch, or send both back for rework.
Concept: Same drink, two positionings (Calm focus, Natural lift).
Audience: US adults 25-45 who buy functional or better-for-you drinks at least monthly.
Deadline: Decision needed in 10 working days.
If the answer is "no": Rework the positioning and retest before the retailer pitch.

The agent restates the decision in one sentence and flags what is missing. In this case, it asked for the planned price point and whether any competitor products should be named. The team added a price of $3.49 per can and chose not to name competitors. A clear brief makes everything downstream easier, as our guide to an effective research brief for AI agents explains.

Step 2: The Action Standard, Agreed Before Fieldwork (Gate 1)

The action standard is the decision rule, agreed before anyone sees a result. The agent proposed, and the team approved:

Outcome

Criteria (all must hold for "Go")

Go

Top-2-box purchase intent of 45% or more (the brand's norm for launches), at least 70% correct playback of the core benefit, and no single barrier theme raised by more than 25% of respondents

Refine

Purchase intent meets the norm, but playback or barrier criteria fail

No-go

Purchase intent below the norm

Setting the rule first guards against hindsight bias. Once results arrive, it is tempting to move the bar to fit the winner. A rule agreed in advance stops that.

The standard also avoids relying on purchase intent alone. People often overstate how likely they are to buy, so comprehension and barriers act as checks on a high intent score. Our existing AI moderated concept testing guide covers why purchase intent should never be the only gate.

Step 3: The Study Plan the Agent Proposes

The agent proposed a monadic design: two cells, 150 respondents per cell, with quotas on age, gender and region. Monadic means each person sees only one concept, which avoids order effects and comparison bias when the two positionings are genuinely different directions.

Sample size drives confidence. Nielsen Norman Group recommends 40 participants for most quantitative studies, which gives a 15% margin of error at 95% confidence for a binary metric. A go/no-go call needs tighter numbers, so the agent proposed 150 per cell. At that size, the margin of error on a percentage is roughly plus or minus 8 points at 95% confidence, using the standard formula for a proportion near 50%.

The team approved the plan with one change: adding a quota so at least a third of each cell were current energy-drink buyers.

Step 4: The Screener (Gate 2)

The agent drafted six screener questions:

  1. Category use: Which of these drinks have you bought in the past month? (Must select functional or better-for-you drinks.)

  2. Buying role: Who usually buys drinks for your household? (Must be sole or shared buyer.)

  3. Recency: When did you last buy a functional drink? (Must be within 30 days.)

  4. Industry rejection: Do you or anyone in your household work in beverages, marketing or market research? (Terminate if yes.)

  5. Red herring: Which of these brands have you bought? (Includes one made-up brand. Terminate if selected.)

  6. Attention check: Please select "Sometimes" for this question. (Terminate if missed.)

Anyone who fails the category, role, recency, industry, red-herring or attention criteria is screened out immediately. The team tightened question 3 from 60 days to 30 days to focus on active buyers.

Step 5: The Discussion Guide and the Agent's Probe Rules

The agent drafted the guide in this order: first impression, playback ("In your own words, what is this drink offering?"), appeal, believability, deal-breakers, price and purchase intent. Purchase intent comes last so earlier questions do not anchor it.

It also set probe rules. Allowed probes ask for detail ("What makes you say that?", "Can you give an example?"). Banned probes lead the respondent, such as "Wouldn't you agree this sounds refreshing?" Leading prompts are one of the most common sources of bias in AI moderated research, so the rules are part of Gate 2.

A sample exchange shows the agent probing a vague answer:

Respondent: It's fine, I guess.
Agent: Thanks. What would need to be different for it to be more than fine for you?
Respondent: I'd want to know it's not just another caffeine drink. "Calm" sounds nice but I don't know what's in it.

The team approved the guide and stimulus, adding one probe on ingredients.

Step 6: Recruiting Real Respondents

There are two main recruiting routes:

  • Panel: Fast access to fresh category buyers across markets, with quotas filled by the panel provider.

  • Your own customer list: Useful for testing with existing buyers, but it skews toward people who already like the brand.

For a launch decision, the team chose a panel so the sample reflects the wider market, not just fans. The agent tracked quota fill daily and set incentives in line with interview length. Good participant recruitment for consumer research matters because the go/no-go call needs real category buyers, not people who rarely buy the product type.

Step 7: Pilot, Then Full Fieldwork (Gate 3)

The agent ran eight pilot interviews before launching the full study. It reported two issues:

  • Several respondents were unsure whether $3.49 was per can or per pack.

  • One question about "botanicals" confused people who did not know the term.

The team reworded the price question to show "$3.49 per 12 oz can" and replaced "botanicals" with "plant ingredients like ginger and lemon balm." With those fixes, the team approved full fieldwork.

Step 8: Quality Control During Fieldwork

Data quality is one of the biggest risks in online research. A Pew Research Center study found that in online opt-in samples, an average of 8% of adults answered "Yes" to at least 10 of 16 Yes/No questions, a sign of bogus respondents who answer without real effort. Checks during fieldwork are not optional.

The agent kept a QA log:

Flag reason

Responses flagged

Removed

Replaced

Speeders

9

9

9

Duplicate devices

4

4

4

Answers contradicting the screener

6

6

6

Pasted or AI-written answers

7

7

7

Off-topic replies

4

4

4

Total

30

30

30

Of 330 completes, 30 were removed and replaced, leaving 300 valid interviews (150 per cell). A researcher also spot-checked 20 transcripts while fieldwork was running to confirm the flags were fair.

Step 9: Synthesis: Scores, Themes and the Evidence Trail

The agent scored each cell against the action standard:

Criterion

Standard

Cell A: Calm focus

Cell B: Natural lift

Top-2-box purchase intent

45% or more

48%

41%

Correct playback of core benefit

70% or more

74%

81%

Top barrier theme

25% or less

Price, 22%

Caffeine worries, 31%

Result


Go

No-go on intent

It then coded the top three themes across both cells, with the number of respondents who raised each:

  • Ingredient clarity (96 of 300): People wanted to know what makes it calming or energizing.

  • Price versus a regular soda (71 of 300): Most accepted $3.49 if benefits were clear.

  • Caffeine concerns (63 of 300): Mostly in Cell B, where "lift" suggested caffeine.

Every quote in the synthesis links back to its transcript, so the team can check context.

The difference in purchase intent between cells (7 points) is within the margin of error, so the agent did not claim Cell A beat Cell B head to head. It judged each cell against the action standard instead.

Where a platform adds emotion measurement, it can expose gaps between what people say and how they react. In an illustrative case, respondents in Cell B rated appeal highly, but facial coding showed flat reactions when the "no crash" line appeared. That kind of behavioral research for concept testing helps explain why stated scores and real interest diverge.

Step 10: The Decision Memo (Gate 4)

The agent drafted a one-page memo:

Decision: Go with Cell A, "Calm focus."
Three reasons: It meets all three criteria in the action standard. Playback shows people understand the benefit. Price was the top barrier but stayed under the limit.
Risks: Ingredient clarity is a recurring question. The intent lead over Cell B is within the margin of error.
What to test next: Pack copy that names key ingredients, and a price check at $3.29.

The team reviewed the evidence, agreed with the recommendation and signed off. The agent drafted the call. The team made it.

How to Audit the Agent Before You Trust the Call

An agent's output is only as good as its checks. Before you act on a recommendation:

  • Re-code 10-15% of transcripts by hand and compare your themes with the agent's.

  • Trace every quote in the memo back to its transcript.

  • Review follow-up probes for leading questions.

  • Check results by segment, not only the overall score.

  • Compare stated intent with behavior or emotion signals. The say-do gap is one of the most common reasons concept tests mislead.

  • Confirm no respondents were removed simply for disagreeing. Quality removals should follow the QA rules, not the results.

Where Synthetic Respondents Fit

Synthetic respondents can help shortlist ideas before a real test. They are fast and cheap for early screening, such as narrowing ten territories to two. They should never make the go/no-go call.

Our guide to synthetic audiences in consumer research explains when they help and where real respondents are essential.

When Not to Hand a Concept Test to an Agent

Agents are not the right fit for every concept test. Keep a human researcher in charge when you are dealing with:

  • Sensory or physical products that people need to touch, smell or taste.

  • Regulated claims, such as pharma or financial services, where every word needs legal review.

  • Very small B2B audiences where each respondent is hard to replace.

  • Sensitive topics that need human rapport.

  • Co-creation sessions where the goal is to build ideas with participants, not evaluate them.

Skipping proper testing has its own cost, as this look at the hidden cost of skimping on concept testing shows.

Adoption of agents is also still early. The Stanford AI Index 2026 reports that 88% of surveyed organizations used AI in 2025, but AI agent deployment was in the single digits across nearly all business functions. Expect the right use cases to expand as tools mature.

Running Agentic Concept Tests in Decode

Decode by Entropik offers AI research agents for consumer and user research. Each agent drafts the study for you to review and edit before anything runs, and every finding links back to its evidence.

For concept tests, Decode's concept validation agent, Cole, compares concepts on preference, comprehension and emotional response. By default, Cole uses a sequential monadic design, where each person sees every concept in random order, with face emotion measurement on each concept. Teams that need a purely monadic design, like the example above, can review and edit the study before it runs.

For interviews, the AI Moderator, Mira, runs hundreds of adaptive AI-moderated interviews at once in 70+ languages.

For recruiting, Decode reaches 103M+ participants in 120 countries through Cint and Dynata, or you can invite your own customers with a link. Every participant is checked before, during and after fieldwork, including duplicate detection, attention checks and flags for AI-generated responses, as described in the panel and recruitment guide.

As a consumer research platform, Decode also measures what people feel and notice, with 90%+ facial coding accuracy across 62 facial expressions and 96% eye tracking accuracy. It is SOC 2 Type II and ISO 27001 compliant.

That combination turns stated answers into deeper consumer insights behind each concept.

For a broader comparison of options, see our guide to the best consumer research platforms.

For more on concept testing methods in general, read our concept testing guide.

Conclusion

A research agent can take a concept test from brief to go/no-go in a fraction of the usual time, while your team keeps control at every gate that matters. Agree the action standard first, approve the study materials, check the pilot, audit the evidence and make the final call yourselves.

If you want to run agent-led concept tests with real respondents and multimodal evidence, explore Decode's AI research agents or request a demo.

Frequently Asked Questions

1. Can an AI research agent make the go/no-go decision on its own?

No. The agent drafts a recommendation based on the agreed action standard, but the team reviews the evidence and makes the final call. Keeping that decision with people is the point of the approval gates.

2. How many respondents does an agent-run concept test need?

It depends on how precise the decision needs to be. Around 40 participants gives a 15% margin of error for simple metrics, while go/no-go studies often use 100-200 respondents per cell for tighter estimates.

3. How long does a concept test take with a research agent?

Timelines depend on audience and sample size, but agents shorten design, moderation and synthesis significantly. Many studies move from brief to decision memo in days rather than weeks, with recruiting and fieldwork usually the longest steps.

4. How is a research agent different from an AI-moderated interview?

An AI moderator runs the interviews. A research agent runs the whole study, from brief and action standard through screener, recruiting, fieldwork, quality control, synthesis and the draft decision memo.

5. Can synthetic respondents replace real ones in concept testing?

No. Synthetic respondents can help shortlist ideas early, but the go/no-go decision should rest on real category buyers.

6. How do you know the respondents in an agent-run study are real?

Look for layered quality checks: screener traps, duplicate and device checks, attention checks, flags for pasted or AI-written answers, and human spot checks of transcripts. Ask the provider to show its QA log for every study.

From Emotion to Action, With Insights That Speak Your Language.

Start turning customer signals into smarter decisions.

From Emotion to Action, With Insights That Speak Your Language.

Start turning customer signals into smarter decisions.

From Emotion to Action, With Insights That Speak Your Language.

Start turning customer signals into smarter decisions.

Site footer

Decode by Entropik is a user research and consumer research platform with AI research agents. It shows how people feel, where they look and what they do, from ad and concept tests to prototypes and live websites.

Ask AI about Entropik

Opens with a ready-made prompt that points to our
AI-readable company page.

Headquarters

Lower Ground Floor, Diamond District, Tower B,HAL Old Airport Rd, Kodihalli, Bengaluru, Karnataka 560008