# How to test an AI phone agent before it takes live calls

By Samuel Oram-Jones

To test an AI phone agent before it takes live calls on Bland, write a judge, build a test case, run them, and make a passing run a condition of promotion. A judge asks a question about a conversation, and a test case is a simulated caller.

On Bland, a required evaluation sits in front of production: a publish that fails it is blocked from staging, and production keeps serving the version it already had. Each step of [Bland's evaluations setup](https://docs.bland.ai/agents/evaluations) happens inside your agent, with no code.

## What do you need before you start?

On Bland, you need an agent you have built, which [building your first AI phone agent](/no-code/how-to/build-your-first-ai-phone-agent) covers, and the Evals feature enabled for your organization. You need an owner, admin, operator, or prompter role to write judges, run evaluations, and make one a required check. Publishing a judge, publishing to staging, and promoting need an owner, admin, or operator role.

To try your agent by hand first, see [testing an AI phone agent in your browser](/no-code/how-to/test-an-ai-phone-agent-in-your-browser). Inside an agent, Evaluations sits under Validate, with tabs for Judges, Test cases, and Runs.

## How do you test an AI phone agent on Bland before it takes live calls?

On Bland, you write a judge, write a test case, run them, read the run, and make the evaluation a required check.

1. Write a judge. The last row of the Judges table is **Create judge**, and under the table is a **Create a new judge** panel with three tiles: **Blank** opens the judge editor empty, **Templates** starts you from a list, and **Generate** builds a draft from a **Prompt**, **Conversation comments**, or a **Triage issue**.
   - **Inputs** sets **Mode** and **Context**: **Transcript** reads what was said, **Audio** hears how, and choosing a mode swaps the list under **Context**, so pick the mode first.
   - Only **Task** is required, and it is the question you are asking about the conversation.
   - **Verdicts** holds the answers the judge may give and which one you are hoping for: **Pass** and **Fail**, with an optional **Inconclusive**.
   - The button reads **Create judge** the first time and **Update** afterwards, and **Save as draft** keeps your work without making the judge live.
   - To tune the judge before you go on, open **Test** in the judge editor, select **Create new test**, and pick **Recent production calls**, **Specific production calls**, or **Simulated**. Correct the verdicts it got wrong, select **Align with AI** for a diff of proposed prompt changes, then choose **Apply** or **Apply and retest**. Alignment lands in the draft, and nothing changes for live runs until you publish the judge.
2. Write a test case. **New test case** starts either from scratch or from your own traffic, and **Generate based on production calls** reads up to 50 real calls and writes one test case per call. Select calls in a picker with the same filters as Conversations, then select **Generate test cases**. Read each generated caller prompt and correct it before you use it. Set a **Title** and a **Prompt** describing who the simulated caller is and what they want, and optionally **Labels**, **Max turns**, a **Start node**, and **Request data**. **Save** finishes it.
3. Run your judges against your test cases. A run pairs the two, so pick both, choose how many conversations each test case should produce, and start it.
4. Read a run. A conversation passes only if every judge passed it, and a judge that could not tell counts against it, so a run can show failed conversations and still clear the gate, which scores each judge on its own across conversations.
5. Make it a required check. An evaluation becomes a gate from the agent's Environments tab, where the Evaluations card sits between Development and Staging. **Add evaluations** opens the picker: attach the judges you want and up to five test cases, and choose **1**, **3**, or **5** conversations per test case. Attaching a judge makes it required by default, and clearing **Req'd** on its row makes it advisory.

## How do you know the test worked?

A judge on Bland passes when half or more of the conversations it scored matched the verdict you were hoping for.

On Bland, an agent's first publish goes straight to production and skips staging, unless a required evaluation is configured, which puts the agent back on the staging path.

Publishing to staging saves your changes as a new version, runs the evaluations against that exact version, and pins it to staging only once every required evaluation passes. When one fails, a toast reads **Publish blocked** with the number that failed, and the version stays on development with nothing lost.

## What can go wrong with Bland's evaluations?

Five common evaluation failures on Bland, each with its cause:

- **A run you started is gone.** Manual runs are stored in the browser that started them. Runs attached to a publish are on the server and survive.
- **A judge you created is not scoring anything.** It is still a draft, or it is not attached, and attaching happens on the Environments tab.
- **An evaluation failed and nothing was blocked.** **Req'd** is cleared on that row, so the judge is advisory.
- **A version you chose in the run dialog is not the one that ran.** Every conversation runs the agent's current version, and the picker only records your choice on the run.
- **Promotion is blocked although the evaluations passed.** The configuration changed after that run, so publish to staging again.

**Force promote** in Bland's Publish menu sends development straight to production and skips the evaluation gate, and Bland's audit log records the bypass.

## What comes next after your agent passes?

Once a version clears staging on Bland, **Promote** in the [Environments](https://docs.bland.ai/agents/environments) page header points production at it. Norm saves a change as a new version without changing live traffic, and the version still has to be published and promoted, with any required evaluations on staging passing first, as [how Norm works](/norm/how-it-works) explains.

Every enterprise customer runs on its own isolated infrastructure, run by Bland: dedicated compute, containers, and resources, with releases the customer chooses when to adopt and canary deployments on a share of real calls.

When a live call still goes wrong, [fixing a bad call with Norm](/no-code/how-to/fix-a-bad-call-with-norm) is the next guide. The [no-code AI call center](/no-code-ai-call-center) guide and the [no-code AI phone agents](/no-code) hub cover the rest.

Idaho Housing and Finance Association saves $750,000 a year by replacing its IVR with Bland, which handles 4,000 inbound calls a day and cut average call time by 20%.

> "We save thousands of dollars a day."

Greg Blake, Chief Information Officer, Idaho Housing and Finance Association

## Frequently asked questions

### Does Bland have a no-code option?

Yes. Norm, Bland's AI assistant, builds voice agents using natural language descriptions, and Bland's dashboard includes a visual builder for non-technical users.

### Do I need code to test a Bland agent before launch?

No. On Bland, you write a judge, build a test case, and start a run from inside your agent without code, and the Evals API scripts the same work.

### What makes a Bland judge pass?

A Bland judge passes when half or more of the conversations it scored matched the verdict you were hoping for. You cannot change the half.

### Can a Bland agent go live without passing its evaluations?

Yes. On Bland, a version can reach production without passing evaluations when none are configured on staging, when the failing judge has **Req'd** cleared and only advises, when someone uses **Force promote** in Bland's Publish menu, or when an API caller promotes with `force: true`. Bland's audit log records both forced paths.

### What happens on Bland when a required evaluation fails?

Bland blocks the publish to staging with a **Publish blocked** toast and dims **Promote**, so production keeps serving the version it already had.
