How to Use Agent Quality

Agent Quality is your weekly review queue. It checks recent calls for each agent, scores their health, and surfaces the agents that need attention — so you fix problems before callers run into them.

What It Does

Where Call Quality is a deep dashboard for one agent's latency, costs, and evals, Agent Quality is the roll-up across all your agents: which ones are healthy and which ones to look at first.

  • Shows a health score for each evaluated agent
  • Flags findings such as silent failures or behavior that needs a human to review
  • Shows the latest evaluation run and recent run history
  • Lets you mark findings as seen, fixing, ready to verify, or ignored so a team can work the queue together

How Health Is Scored

The health score summarizes how an agent has been performing on its recent calls. Agents with no recent problems sit near the top; agents with repeated or severe findings drop. A finding is something worth a person's eyes — for example a call that ended without resolving, a silent failure, or behavior that drifts from your prompt.

Findings carry a severity, and the table sorts so the most urgent agents rise to the top. The goal is triage: you should not have to read every call to know where to spend your time.

Where the detail lives: Agent Quality tells you which agent and what kind of problem. To see the per-call latency, outcomes, and eval breakdown behind a low score, open that agent in Call Quality.

Review Agent Health

  1. Go to Dashboard > Agent Quality.
  2. Review the summary cards for healthy agents and agents needing attention.
  3. Open the agent table to compare health score, recent call volume, last call date, and the suggested next step for each agent.
  4. Use Refresh after recent call or prompt changes when you need the newest status.

Work Through Findings

  1. In the findings table, start with high-severity items and the lowest health scores.
  2. Open the suggested fix or the agent link when one is available.
  3. Update the finding status as your team works on it.
  4. After the agent has taken fresh calls, mark fixed items ready to verify so the next run confirms the fix held.
Best practice: Treat Agent Quality as the weekly review queue for prompt and tool cleanup. Fix the most repeated findings first — they affect the most calls.

FAQ & Troubleshooting

  • •An agent has no score. It probably has not taken enough recent calls to evaluate. Scores appear once there is activity to judge.
  • •I fixed the prompt but the score is the same. Health is based on real calls. Give the agent fresh traffic, then refresh — the score updates as new calls come in.
  • •A finding isn't a real problem. Mark it ignored so it stops cluttering the queue without losing the record.

Related

    How to Use Agent Quality | KaiCalls Help