Process
Tested: we have run it ourselves. Planned: part of the design, not yet run.
-
Intake
You bring the job, with examples of good work and bad, or an eval you already have.
-
Success
We agree what the test must tell you.
-
Review
You approve the test cases before we use them to score your agent.
-
Build
TestedWe package the cases and their scoring as an environment for Prime Intellect’s Verifiers.
-
Scoring check
PlannedWe show that the scoring accepts known-good work and rejects plausible mistakes.
-
Baseline
TestedWe measure your current agent or model with Prime Intellect’s Verifiers, several times over, and explain where it fails.
-
Full run
PlannedFor a large test, Prime Intellect’s prime-rl runs the whole evaluation at scale.
-
Handover
You get the test suite, baseline scores, and a report of where your agent fails.
You receive
- Reviewed test cases.
- Scoring for every case.
- Scores for what you run today.
- An analysis of the failures.
- One command that reruns the whole test.
Tools
Open-source libraries for your project, each named with its maker.
Tested
- Prime Intellect’s Verifiers
- Packages the cases, runs them and scores them. Results: React Best Practices Dependabot Gitmoji
Planned
- Prime Intellect’s prime-rl
- Runs the whole evaluation at scale.
Terms
- Model and compute usage is billed separately, under a cap we agree first.
- If you already have an eval or an environment, we reuse it. We quote separately for any evaluation or training environment we need to build.
- You pay for the scoped work and its evidence. If the result shows no improvement, you still receive it, with the reason.
Notes
- A better eval catches the failures that matter and scores the same work the same way every time. It need not produce a higher pass rate: it gives you a test you can trust.
Start
Contact
Bring the workflow and a few examples. We’ll agree what success means before we write a test.