How to evaluate an AI agent before production
Evaluate the whole workflow: the answer, the information it used, the actions it proposed and the checks that allowed execution. A plausible response is only one part of a successful task.
Start with an explicit task and a baseline
Write down what the user is trying to accomplish, what information the application can access and the actions it may take. “Answer a sales question” is too broad. “Find the current blocker on an accessible opportunity, cite the evidence and propose a follow-up task” defines something you can inspect.
Collect representative examples before changing the implementation. Include ordinary requests, missing information, conflicting sources and requests outside the user’s permissions. Freeze a version of that set so you can compare changes. Keep separate cases for development and later checks so repeated tuning does not become the only evidence of quality.
What the RaptorCI evaluation shows
In the published RaptorCI case study, CloudHero evaluated AI code review against 60 frozen, marked pull-request scenarios. Overall recall increased from 42% to 66%, while precision increased from 51% to 60%.
| Measure | Before | After |
|---|---|---|
| Overall recall | 42% | 66% |
| Precision | 51% | 60% |
| Business-logic recall | 60% | 82% |
Recall describes how many expected findings were detected. Precision describes how many reported findings were valid. Business-logic recall is a category result; it should not be presented as overall accuracy.
The review also identified approximately 19 valid bugs outside the expected ground truth. That matters because an evaluation set can be incomplete: a finding absent from the answer key needs investigation before it is labelled a false positive. Preserve the original baseline, document the adjudication and make any updated ground truth explicit.
These results describe one application and evaluation set. They are not a forecast for another agent. The case study’s approximately $0.12 per pull request is model spend, not the full infrastructure or operating cost.
Test boundaries as well as answers
- Retrieval: can the application find the relevant source, cite it and recognise when information is missing or stale?
- Permissions: do searches and tools restrict access to the requesting user’s records? Include attempts to retrieve another user’s restricted information.
- Actions: does the proposed change match the user’s request? Are input validation, approval and stale-record checks enforced at execution?
- Failure handling: what happens if a tool times out, an integration is unavailable or the user retries? Check whether an action was actually completed.
- Operation: can the team inspect traces, usage and errors without exposing unnecessary sensitive information?
In Octora’s Otto workflow, an approved email action saves a draft. A test should assert that exact outcome and verify that it does not send an email. A generic “tool succeeded” label would miss the distinction.
Set a release decision, then keep observing
Agree the acceptance criteria with the people responsible for the process. A low-impact suggestion and a consequential record update need different review boundaries. Document failures that block release, those that trigger human escalation and any limitations users must understand.
Track task quality separately from availability, latency and cost. A healthy API can return poor answers; a strong offline evaluation does not prove that production integrations behave reliably. Repeat relevant evaluations after changes to models, prompts, retrieval, tool contracts or source data.
When the evidence is insufficient, the next step can be a focused experiment or a data correction. Our AI readiness assessment and implementation planning helps define that decision. For an existing system, explore AI evaluation and engineering.