Curate test cases, bind them to an agent, and run them before you promote a version. You get a verdict, the cases that failed, and the delta against a baseline run.
Write cases by hand or curate them from generations the agent already ran, with an optional reference answer.
An eval binds one agent to one dataset and freezes the scorers that grade it, so two runs are compared on the same terms.
Each run is real generations against one agent version, with a pass/fail against your threshold and a per-scorer delta against a baseline run you name.
When a run fails, its record names the step and the tool call that broke it.
On every plan, Free included. Each item a run scores counts as a run. Compare plans
By the end you will have a repeatable suite that answers whether a change is good enough to ship. Every step is one call, shown for the CLI, the SDK and curl.
Start the tutorialSign in, connect your assistant or open the quickstart, and run your first agent.
Free plan · 2,000 runs a month · no card
