## What Is an Evaluation?

Agent evaluation is the process of testing and measuring how effectively an AI Agent can answer questions similar to what your internal users and customers will be asking. Think of an evaluation as a unit test for large language models (LLMs). Because LLM behavior cannot be tested programmatically the way traditional code can, evaluations instead programmatically test the inputs and outputs of an LLM against a defined model and configuration.

Evaluations are made up of two core components: **Datasets** and **Experiments**.

### Dataset

A dataset is a series of questions and answers that you create. This data serves as the test suite your agent can run each time before it is deployed to production.

### Experiment

When you run an experiment, you are instructing your agent to answer each question in the dataset. A separate validator agent — an LLM-as-a-Judge — then compares the agent's answers to the correct answers you have pre-defined. The validator scores each answer as correct or incorrect and returns an overall score for that experiment. You can compare results across previous runs to determine whether your agent is improving or degrading over time.

## How to Run an Experiment

Experiments can be run ad hoc from the Evaluations screen, or in context from the Agent Deployment screen within the agent configuration.

### Running an Experiment from the Evaluations Screen

1. Create your dataset with your questions and answers.
2. Click into the dataset.
3. Click **Run Experiment**.
4. Choose the agent to test and the evaluators to test against.

### Running an Experiment from the Agent Screen

1. Open an agent from the **Agents** menu.
2. Navigate to the **Deployment** screen.
3. Click **Run Experiment**.
4. Choose your evaluations and click **Run Experiment**.
5. Your results will be displayed once the experiment is complete.