Process
Tested: we have run it ourselves. Planned: part of the design, not yet run.
-
Intake
You bring your Skill, the agent and model that use it, real examples of the work, and the failures you already know about.
-
Success
We agree what better means for you, such as more correct work, fewer serious mistakes or a lower cost, and set a budget cap before anything runs.
-
Quality check
TestedNVIDIA’s SkillEvaluator reviews the Skill as it is written.
-
Baseline
TestedWe measure your current Skill with your agent through Prime Intellect’s Verifiers. We repeat the baseline evaluation to measure how much results vary. The cases for the final test are set aside, and the improvement never sees them.
-
Improvement
TestedMicrosoft’s SkillOpt proposes revisions from scored attempts on the working cases, and a single plain rewrite is made beside it.
-
Selection
TestedThe original and both revisions are measured on a separate set of cases, and the one that solves the most is chosen, so you see what the automatic revision adds over a plain rewrite.
-
Final test
TestedA single run on the untouched cases, scored by Prime Intellect’s Verifiers, with the model, tools and limits held fixed. When the test does not establish a difference between the two versions, we report the result as inconclusive.
-
Handover
You get the revised Skill, its report, and one command that reruns it. If the Skill did not improve, you still get the result and the reason.
You receive
- The original Skill and the revised one.
- A report covering cost, new errors, and uncertainty in the results.
- The exact setup it ran on.
- One command that reruns it.
Tools
Open-source libraries for your project, each named with its maker.
Tested
- NVIDIA’s SkillEvaluator
- Reviews the Skill as it is written. Results: React Best Practices Dependabot Gitmoji
- Microsoft’s SkillOpt
- Proposes revisions to the Skill from scored attempts. Results: React Best Practices Dependabot Gitmoji
- Prime Intellect’s Verifiers
- Runs the final test and scores it. Results: React Best Practices Dependabot Gitmoji
Terms
- Model and compute usage is billed separately, under a cap we agree first.
- If you already have an eval or an environment, we reuse it. We quote separately for any evaluation or training environment we need to build.
- You pay for the scoped work and its evidence. If the result shows no improvement, you still receive it, with the reason.
Notes
- SkillOpt improves the Skill’s written instructions. Repairing the scripts that come with a Skill is separate engineering, which we scope on the call.
Start
Contact
Bring the Skill and a few examples of good and bad work. We’ll agree what better means before anything changes.