Call center quality assurance with AI: a practical guide

Call center quality assurance (QA) has worked the same way for decades. A QA analyst picks a few calls per agent each month, listens to them, scores them against a checklist and talks the results through with the agent. It works, within its limits. The limits are well known: the sample is small, scoring varies between analysts, and by the time a pattern shows up in the scores, it has been happening for weeks.

AI can now transcribe and read every call. That removes the time limit, but it does not remove the need for judgment, fairness or trust. This guide covers what changes when QA is done with AI, what should stay with people, and how to roll it out so the people being scored accept the result.

What call center QA is for

It helps to start with the purpose, because AI makes it easy to measure many things that do not matter. Call center QA usually serves four goals:

• Consistency. Customers get the same standard of service whichever agent picks up.
• Compliance. Required steps happen: identity checks, disclosures, recording notices, correct handling of cancellations and refunds.
• Coaching. Agents learn what good looks like from their own calls, not from a generic training deck.
• Insight. The business learns why customers call, what goes wrong and what customers ask for.

The traditional process serves the first three reasonably well and the fourth poorly. A few calls per agent per month is enough to coach an agent, but it says very little about why ten thousand customers called last month.

Where manual QA runs out

Manual QA is not wrong; it is just bounded by listening time. A nine-minute call takes at least nine minutes to review, usually more with scoring and notes. That creates three problems that get worse as the operation grows.

The sample is too small for rare events

The most expensive events in a contact center are rare: a compliance slip, a customer who says they are leaving, a product defect described for the first time. A small random sample finds common behavior and misses rare behavior, which is exactly the opposite of what a risk-focused team needs. We work through the numbers in From a small QA sample to every call.

Scores depend on who scores

Two analysts listening to the same call often disagree on subjective items such as “showed empathy” or “handled the objection well”. Calibration sessions help, but they take time, and the disagreement shows up in agent scores as noise. Agents notice, and trust in QA erodes.

Insight arrives late

When a new issue appears, it takes a while before enough sampled calls mention it for someone to notice. By then the issue has reached the social media team, the churn report or the client's account manager.

What AI changes, and what it doesn't

An AI-based QA system, sometimes called automated QA or auto QA, adds three capabilities:

1. Transcription of every call, with timestamps and, ideally, the speaker of each sentence.
2. Analysis of every call: why the customer called, how it ended, whether a follow-up is needed, signs of frustration or churn, and scorecard items.
3. Aggregation: trends by day, queue, topic and team, and lists of calls that need action.

What it does not change is that someone has to decide what good looks like, check that the machine reads it correctly, and handle the cases where it is unsure. A good AI QA setup moves analysts away from finding calls and toward the work that needs judgment: calibration, disputes, coaching and the calls the system flags for review.

The building blocks of AI QA

1. Recordings that make speakers clear

Many scorecard items depend on who said something. A required disclosure counts only if the agent said it. Two-channel (stereo) recording puts the agent and the customer on separate tracks and removes the guesswork. If you can change one setting before starting, change this one; see Two-channel recording: the cheapest upgrade for fair QA.

2. A scorecard written for machines and people

Items like “professional tone” work for a human reviewer who shares your context. For automated scoring, each item needs an objective definition: what counts as yes, what counts as no, and when the item does not apply. Some items can be checked by plain code, for example whether a greeting came in the first ten seconds. Others need a language model, but always with a quote as evidence. We cover this in detail in How to build a QA scorecard AI can score fairly.

3. Evidence for every finding

Every score, flag and list item should link to the sentence that supports it, with a timestamp and audio. Quotes should be checked against the transcript by code, because language models sometimes paraphrase or invent a plausible line. In VoiceVera's public showcase of 873 calls, on average 99% of the quotes cited in a call matched the transcript word for word; the check caught the rest. More on this in Why every AI finding needs a quote and a timestamp.

4. Rules for when a person reviews

Some calls will be unclear: bad audio, short replies, a decision that could go either way. Instead of relying on a confidence number the model reports about itself, use rules anyone can read. For example: send the call to review when two independent passes disagree, when the winning answer is only narrowly ahead, or when the evidence does not match the transcript.

5. Disputes and corrections that keep the history

Agents need a way to dispute a score, and reviewers need a way to correct it. The correction should be recorded next to the original, not overwrite it. That is how you can show later what the machine said, what a person decided and why.

6. Lists, not only dashboards

A dashboard says complaints rose. A list says which 52 calls were complaints, with the sentence from each. Teams act on lists: follow-ups with an owner and due date, churn risks, possible product defects, questions your help center does not answer.

Metrics that change when every call is read

One caution: when the measurement changes, the numbers change. A team moving from a sample to every call will usually see more issues, not because quality dropped but because more is being seen. Tell leadership this before the first report.

Keeping it fair for agents

AI QA can be the fairest QA an agent has had, or the least trusted. The difference is in a few choices:

• Score behavior, not personality. “Confirmed the order number before making changes” is fair. “Sounded tired” is not, and should never be scored.
• Show the evidence. Every point lost links to the moment in the call.
• Allow disputes and track how many succeed. A high success rate means the scorecard needs work, not the agents.
• Compare like with like. An agent who takes complaint calls should not be ranked against one who takes simple orders.
• Don't score what the recording can't show. Items that depend on speakers should not be applied to mono recordings where the speaker is uncertain.

A 90-day rollout plan

Days 1 to 30: prove it on your own calls

1. Pick a few hundred recent calls from one or two queues. Check whether your recordings are two-channel.
2. Agree on topics and a first scorecard of 10 to 15 items, with objective criteria.
3. Run the analysis. Have two analysts label a small set by hand and compare with the results.
4. Tune the review rules so reviewers see the calls that need them, not every call.

Days 31 to 60: put the lists to work

1. Choose two or three lists with a clear owner: follow-ups, churn risks, possible defects.
2. Share evidence-backed scores with team leads first, then with agents, with a dispute path from day one.
3. Replace part of the manual sample with reviews of flagged calls.

Days 61 to 90: make it routine

1. Connect the phone system so new calls flow in automatically.
2. Set alerts for spikes in topics that matter, and a weekly digest for leaders.
3. Review the scorecard: drop items nobody acts on, sharpen items that are often disputed.

Questions to ask any AI QA vendor

• Can I click any score or finding and hear the exact moment that supports it?
• How do you check that quotes are real and not paraphrased?
• Who decides which calls go to human review, and can I read the rules?
• Do numbers in reports come from the data or from the model's text?
• What happens to a result when a reviewer corrects it? Is the original kept?
• Do you score agents' emotions or personality? (The right answer is no.)
• What is sent to AI providers, and is personal data removed first?
• Can I try it on my own calls before committing?

Common questions

Does AI replace QA analysts?

No. It removes the time spent finding calls to listen to. Analysts move to the work that needs judgment: calibration, disputes, coaching and the calls the system sends to review.

How do you know the AI scored a call correctly?

Every score should link to the sentence and moment that support it, quotes should be checked against the transcript by code, and a small set of calls labelled by hand should be compared with the results before the scores are used.

How long does it take to start?

A first set of a few hundred recent calls can be analyzed within the first month. Connecting the phone system so new calls flow in automatically usually comes in the last phase of a 90-day rollout.

VoiceVera was built around these questions. The case study shows the whole process on 873 public calls, and the security page explains how data is handled.