AI-Moderated Interviews: A Practical Guide for Product Teams

An AI-moderated interview is a conversation in which an AI interviewer asks real participants questions and follows up on their answers. A researcher or product team chooses the learning goal, decides whom to invite, and reviews the results. The interview can happen through voice or text.

For a product team, the useful question is specific: can this method help us understand an experience we need to make a decision about? This guide offers a study workflow and a practical checklist for evaluating the answers.

What an AI interviewer does

You give the interviewer a goal, relevant context, and topics to explore. A participant opens a link, chooses to start, and describes an experience. The interviewer uses that answer to decide what to ask next. The platform records the exchange and may prepare transcripts, summaries, and configured measures for review.

The participant is a person. A tool generating responses from fictional customer personas is a different method. A meeting assistant that transcribes a call is also doing a different job: the human still conducts the interview.

Look for these distinctions when evaluating AI interview software. “AI research” can refer to recruiting, moderation, transcription, synthesis, or simulated participants. Establish which task the product performs before comparing it with your existing process.

Choose a question that fits the method

Start with one decision and an experience that participants can describe. Useful starting points include:

• Onboarding: What happened when new users tried to finish account setup?
• Customer discovery: How do people currently handle the workflow we want to improve?
• Churn: What changed between a customer’s first useful experience and their decision to leave?
• Feature feedback: What did users try to accomplish with a feature, and what happened next?

These questions call for accounts of experiences. Interviews do not directly observe behavior. Nielsen Norman Group makes that distinction in its guide to user interviews. If your question is whether people can complete a checkout, observe them trying it in a usability study.

Choose a human interviewer when the conversation calls for sensitive judgment, a complicated relationship, or deep exploration of an unexpected subject. You can use AI interviews to identify experiences worth following up on personally.

A study workflow you can adapt

1. Write the decision before the questions

“Understand our users” is too broad to evaluate. Try: “Decide whether our next onboarding experiment should address permissions or importing existing data.”

Record what evidence would change your mind. If you already believe imports are the problem, include questions that could reveal an entirely different obstacle. Keep that note beside the study so analysis does not drift toward confirming the original idea.

2. Define who should take part

Identify the experience participants need to have had. For an onboarding study, distinguish people who completed setup, started and stopped, and never started. For a cancellation study, distinguish a billing failure from a deliberate decision to leave.

Decide how you will invite each group and keep a record of who received the invitation. A convenient sample of enthusiastic customers can be useful, but it cannot represent every customer. An AI moderator does not fix a sampling problem.

3. Prepare an interview guide

Use a small set of topics, with a starting question and optional follow-ups for each. Ask about a recent event before asking for general opinions. A starting question might be: “Walk me through the last time you tried to invite your team.” A follow-up could ask what happened when the participant reached a particular step.

Give the interviewer clear boundaries. It should allow someone to skip a question, avoid requesting unnecessary sensitive details, and finish when the participant wants to stop. Explain that it is an AI interviewer and how the conversation will be used before participation.

The GOV.UK guide to in-depth interviews is useful background on discussion guides and open questions. Our customer discovery question bank gives an original guide you can adapt.

4. Test several kinds of answers

Rehearse a detailed answer, a vague answer, a contradiction, an off-topic answer, and an early exit. Test in each language you intend to use. Pay attention to whether the agent:

• asks one understandable question at a time;
• follows a useful detail without suggesting the desired answer;
• accepts uncertainty rather than inventing a missing fact;
• moves on when the participant has nothing else to add;
• closes the conversation clearly.

These are recommended checks, not a published performance benchmark. Keep the test conversations separate from customer findings.

5. Review a small pilot before expanding

Read the first completed conversations in full. Check whether the questions actually reached your research goal. Did participants describe a concrete experience? Did the moderator miss important details or steer the answer? Did anyone struggle with voice, text, or the invitation?

Revise the guide when necessary, and record the version change. If you materially change the questions, preserve that context when comparing answers from different groups.

6. Analyze with an evidence trail

A useful finding should identify the experience, the participants or sessions supporting it, and what remains uncertain. Preserve the difference between something someone explicitly said and an interpretation generated during analysis.

An inferred sentiment score is not a directly answered survey question. A theme appearing in several interviews is not automatically the most common problem across your customer base. Use interviews to develop and refine explanations, then choose additional research or measurement where needed. See our interview analysis workflow for a reusable evidence record.

What to check before buying a platform

Ask to test the full workflow with a research question you understand. Evaluate:

1. Moderation: Does the interviewer ask relevant, neutral follow-ups?
2. Participation: Can your audience use the available voice or text experience?
3. Distribution: Do links or website embeds fit your invitation process?
4. Evidence: Can you inspect the original conversation behind a summary?
5. Measures: Are direct answers and inferred scores distinguishable?
6. Control: Can you edit the goal, context, language, and questions?
7. Data handling: Do the documented access, retention, and export conditions fit your project?
8. Cost: How do session allowances, voice minutes, analysis, and exports map to your planned study?

Confirm whether recruiting participants is included. A moderation platform and a participant panel solve separate operational problems.

How this works in Stetos.co

Stetos.co lets teams configure an AI agent, share a link or embed a website widget, and collect voice or chat interviews. Sessions produce transcripts, summaries, and configured metrics within a project. Insights chat helps explore the project’s findings; your team reviews the underlying conversations and decides what to do.

You invite your participants. Agents can be configured in English or Spanish. Features and limits depend on the plan, so check current pricing against your study design. The live demo lets you experience an interview before setting up your own.

Common questions

Are AI-moderated interviews the same as unmoderated usability tests?

No. A conversational interview collects an account of an experience. A usability test asks someone to attempt tasks with a design. Choose the method that produces the kind of evidence your decision requires.

How many interviews should we run?

Set the scope around the decision, the variation in your audience, and what remains unanswered. Review a pilot, identify gaps, and recruit deliberately. There is no universal interview count that establishes product-market fit or makes a convenience sample representative.

Can AI findings go straight into the roadmap?

Review them first. Check supporting conversations, contradictory examples, the participant group, and the consequences of acting. A useful output is a proposed next step with evidence and uncertainty attached.