Let the agent do the slow parts, and keep a person on every claim.
- The problem
Research analysis took weeks.
One long AI prompt over all the transcripts hides what it skips.
- What I did
I built UXRA, a research agent with 5 stages.
Each stage does one checkable job, and a researcher approves every draft.
- What it showed
The speed came from splitting the work and checking each part.
Every insight traces to observations, and simulated answers never count.
- What changed
Analysis now takes hours, and UXRA spread beyond research.
Designers, PMs, a product director and product ops use it too. I also started AI in UXR, our research team's community of practice, ran its workshops to upskill the team and contributed to How to AI UXR.
- What's next
The researcher now designs the study and decides what ships.
As simulated answers get more convincing, a person's sign-off matters more.
- The research behind it
- Small models split into planner, caller and summarizer beat one doing all three. (Shen et al., EMNLP 2024)
- Memory organized by subgoal doubled agents' success on long tasks. (Hu et al., ACL 2025)
- 44% of multi-agent failures traced to system design, and 24% to weak checks. (Cemri et al., NeurIPS 2025)
- ChatGPT matched survey averages, yet its answers varied less than people's and shifted with wording. (Bisbee et al., Political Analysis, 2024)
- The full story
-
The question
Research analysis on our team took weeks. Reading transcripts, coding open text and building up to findings is slow, and it's also where the rigour of a study lives. My question was which parts of that work an AI agent could take on without the research becoming any less trustworthy.
Why separate stages
The obvious first move is one long prompt over every transcript. It hands back a tidy report and hides what it skipped. I split the work into 5 stages: plan the study, code the open text, synthesize, check with a synthetic panel and draft the report. Each stage does one job, which means each can be checked on its own.
Synthesis builds up from the bottom. A transcript is first tagged into discrete observations, and every insight links to the observations under it. Planning draws on what the team already knows, from our docs and org knowledge, and templates shape what the agent writes. The research behind this case points the same way. One 2025 study traced 44% of multi-agent failures to system design and 24% to weak checks.
Who took part
All 7 researchers on our team use UXRA. Beyond the team, several designers, a couple of PMs, a product director and a product ops person have used it, which makes 5 roles in all. The people whose words it reads are protected as well. Customer data stays in its study, and anything that cites it is anonymized.
My part and the team's
I built UXRA, and I did the work that made it reliable enough to use on real studies. The other 6 researchers are my peers. They run their studies through it, and a researcher approves every draft or sends it back.
I also started AI in UXR, our research team's community of practice, and ran its workshops. The 4 sessions were shared exploration, side-by-side model tests, a mini-agent workshop and a standing showcase. All 7 of us built a skill, app or agent of our own, and some are still in use. I contributed to How to AI UXR, published in The ResearchOps Review in 2026.
How I know the output is right
Nobody is asked to take UXRA's output on trust. It can be checked in 3 ways:
- A researcher checks each stage's output, and a person signs off every draft or sends it back.
- Every insight links to the observations under it, and every claim in the report traces to anonymized quotes.
- Every run is logged, so any finding can be retraced later.
I also keep context apart from evidence. Our docs, org knowledge and templates tell the agent how we work, and evidence comes only from the study's own data.
The synthetic panel has a rule of its own. Simulated answers are green for exploring, yellow until real people confirm them and red as evidence. They are flagged wherever they appear, and they never count as support for a finding. Published tests of AI-written survey answers explain the caution. The answers can match a survey's averages while varying less than people's do and shifting with the wording of a question.
What was hard
The hard part was the distance between an agent that produces a report and one a research team can rely on. Closing it took validation, traceability and auditing, and that work is what made UXRA ready for real use. The synthetic panel will keep getting harder to handle, because as simulated answers get more convincing, a person's sign-off matters more.
What changed
Analysis to report went from weeks to hours. The whole research team uses UXRA, and it has reached 4 other roles. It also settled where the line sits for us. The agent does the slow parts and a person stays on every claim. The researcher now designs the study and decides what ships.
What I'd do differently
I'd log the time saved, study by study, from the first week. The move from weeks to hours is real, and a running record would let me put a number on each study.
- Related