Operation Guide and Mechanism Docs

Install and check this machine

uv tool install --python 3.12 regents-cli==1.10.2
# It should print 1.10.2:
regents --version
regents techtree doctor --climb hello-world-climb@3

Doctor checks what this machine needs without calling a model. From there, every answer ends with its next steps: the exact command to run, or what your person has to do first.

Test your Skill

Techtree makes practice tasks from what your Skill teaches, then your agent works them without the Skill and with it. Every step that sends anything to a model prints a review first and waits for your yes. Each command prints the next one.

1. Make the tasks

regents techtree forge inspect-skill path/to/your-skill
regents techtree forge plan SOURCE_ID --provider PROVIDER --model MODEL
# Review, correct, build and accept, one printed step at a time.

Accepting the tasks that qualified gives you a collection. Some of its tasks are held out from anyone improving the Skill.

2. Run them both ways, then compare

regents techtree forge run --arm baseline --collection COLLECTION_ID --provider PROVIDER --model MODEL
regents techtree forge run --arm candidate --collection COLLECTION_ID --provider PROVIDER --model MODEL --skill path/to/your-skill
regents techtree forge compare BASELINE_RUN_ID CANDIDATE_RUN_ID

Both runs use the same provider and model. The comparison calls no model: it pairs every task across the two runs and says whether the Skill improved, regressed, was mixed or made no difference, overall and on the held-out tasks alone. It writes a report you can open in a browser. The tdd example shows a finished one.

3. Revise once, if you want to

regents techtree uplift context COMPARISON_ID gives what a revision may learn from, never the held-out tasks. Write the revised Skill into a new folder, then run regents techtree uplift prepare --from-run COMPARISON_ID --candidate-skill path/to/revised-skill and the regents techtree uplift start command it prints. The revision is judged on the held-out tasks alone.

Fix a task, or check where things stand

If a built task is wrong, fix a copy of it by hand and record that with regents techtree forge correct-task CONSTRUCTION_ID TASK_NAME DIR. regents techtree forge status ID shows any build, plan, run or comparison. regents techtree forge export COLLECTION_ID --to FOLDER writes a copy of the tasks for someone else to read.

Compare two versions of a Skill

To decide whether a change is worth keeping, give the baseline run the earlier version instead of no Skill. Keep the earlier version in its own folder.

regents techtree forge run --arm baseline --collection COLLECTION_ID --provider PROVIDER --model MODEL --skill path/to/earlier-skill

Then run the candidate with the new version and compare the two runs, exactly as above.

Take a quick look with the Hello World Climb

The Hello World Climb is a small, fixed comparison that ships with the release, run with a starter Skill. It shows how a run is approved and what a Result looks like. Its toy tasks say nothing about your own Skill. Preparation prints the exact one-time start command and the most the run may spend before anything runs.

regents techtree skill starter
# On your ChatGPT plan:
regents techtree climb prepare hello-world-climb@3 --skill path/to/skill --access chatgpt-plan
# Or on your own Prime key:
regents techtree climb prepare hello-world-climb@3 --skill path/to/skill --access prime-key

Approve and run only the exact regents techtree climb start command printed by preparation. Closing the terminal does not stop a started Run.

Improve a Skill on Frontier-CS problems

The Frontier-CS Open-Ended Climb has ten open-ended programming problems from Frontier-CS, created with FrontierSmith, where no perfect answer is known. For each problem the agent writes a C++17 program, and the problem's own checker scores it from 0 to 1. Every problem runs once without your Skill and once with it. Start from its starter Skill, or from your own.

regents techtree skill starter --climb frontier-cs-open-ended-climb@3
# On your ChatGPT plan:
regents techtree climb prepare frontier-cs-open-ended-climb@3 --skill path/to/skill --access chatgpt-plan
# Or on your own Prime key:
regents techtree climb prepare frontier-cs-open-ended-climb@3 --skill path/to/skill --access prime-key

Use Hermes

Hermes can run every step itself, like any coding agent. Give it the same instruction:

Hermes may prepare commands and explain output. Techtree still stops before every model call and waits for your own yes.

The Techtree plugin for Hermes adds commands for the Hello World Climb and for publishing its Results:

hermes plugins install regents-ai/techtree/plugin --ref a469830f059588174e71b0a7df424af0a4c5a242 --enable
hermes plugins doctor techtree --ci
# In a fresh Hermes session, enter:
/techtree setup

Verify a Result locally

Verification reads a Result bundle, recomputes its checks, and makes no model call. Give it the run’s ID, the bundle’s folder, or a bundle downloaded from a published Result’s page.

regents techtree proof verify path/to/result-bundle

Read exactly what verification establishes.

Publish a Result

Only Climb runs can be published today. A Skill comparison stays on your computer; regents techtree forge export writes a copy of its tasks to share.

Publishing is a separate action performed after a Run finishes. The CLI shows the publication terms and asks before it sends the Result bundle.

regents techtree publish RUN_ID

The server returns a signed publication receipt and repeated publication of the same bundle is idempotent. Published comparisons appear in Results.

regents techtree withdraw BUNDLE_DIGEST

Withdrawing marks the entry withdrawn and stops offering its bundle. The page keeps its address, and copies anybody already holds are still theirs to check.

Use the machine interface

Pass --json to CLI commands for one machine-readable envelope on stdout. Machine mode does not prompt; operational messages go to stderr.

Release bootstrap

GET /api/v1/bootstrap

Published Climb catalog

GET /api/v1/catalog

One Climb

GET /api/v1/climbs/:slug

Published Results

GET /api/v1/publications

One published Result

GET /api/v1/publications/:digest

Publication key

GET /api/v1/publication-keys/:key_id

Protocol payloads retain their schema field names even where the public site uses simpler words.

Let a browser's agent read Techtree

Agents acting for a person

Errors

{"error": {"code": "publication_missing", "message": "no run is published under that fingerprint", "hint": "Use a fingerprint listed at https://techtree.sh/api/v1/publications."}}

Rate limits

RateLimit-Policy: "default";q=120;w=60
RateLimit: "default";r=119;t=42

Versioning and deprecation

Know what leaves the machine

Local Runs, Episodes, and Traces stay local. Model calls go to the provider you chose, or the one the Climb names. Techtree receives a Result bundle only when you explicitly publish it.

No Techtree account or browser upload is required.

Troubleshoot from the boundary inward

1. Run Doctor for the exact Climb reference.
2. Read the next action and any missing prerequisite it reports.
3. Use regents techtree run status RUN_ID for a started Run.
4. Use regents techtree run logs RUN_ID for execution details.
5. Use regents techtree forge status ID for a Skill test's build, plan, run or comparison.
6. Re-run local verification before attempting publication again.

Source and issue tracking live at github.com/regents-ai/techtree.

Method

Most agent improvements are anecdotes.

Someone changes a prompt or a Skill, runs a few examples, and says the agent feels better. But many other things may have changed at the same time: the model, the harness, the tools, the task set, or the budget.

Techtree starts with a narrower question:

Did changing this one Skill make the agent better?

The second question is just as important:

How can someone else check the result without running the whole evaluation again?

Techtree’s first answer is a controlled comparison: the same agent works the same tasks twice, and only the Skill changes. Techtree calls this a Climb.

The proof bundle

A Techtree proof contains the signed comparison, the task-level results, the exact task membership, and the fingerprints of the files used to produce the claim.

Anyone holding the bundle can verify it offline:

regents techtree proof verify path/to/result-bundle

No model call is needed. The verifier does not need to contact Techtree.

It checks that:

• the files still match their recorded hashes;
• the signatures are valid;
• the Result names a published Climb;
• both branches used the same ordered tasks;
• only the permitted Skill changed;
• the reported wins, losses, ties, and totals recompute from the task results;
• the bundle satisfies its publication policy;
• no private episodes, transcripts, or machine-local paths were included.

This is useful because a copied bundle can be checked independently of the website that published it.

The proof is participant-attested

It proves that the stored evidence is signed, internally consistent, and describes the controlled comparison it claims to describe.

It does not prove that:

• Techtree witnessed the run;
• the participant’s machine behaved honestly;
• the Result generalizes beyond this Climb;
• another party reproduced it;
• the Skill will help every model, harness, or environment.

Those are different claims.

Independent reproduction requires another participant to run the same scientific contract and publish a separately attributable result. Stronger claims about the machine itself would require stronger execution attestation.

Techtree keeps these questions separate rather than calling every valid signature “independent verification.”

The introductory Hello World Climb

It is intentionally simple.

It contains a Skill that clearly helps relative to the baseline. This makes it easy to see whether the machinery is working:

No tested Skill
    ↓
Starter Skill
    ↓
Measured difference
    ↓
Signed receipt

The benchmark is not meant to establish broad agent capability. It is a mechanism test for the complete local path:

• install the tools;
• prepare a controlled comparison;
• approve the model spend;
• run both branches;
• collect the Verifiers results;
• sign the proof;
• verify it offline.

The point is to show that an ordinary user can run the whole experiment on their own machine without trusting a Techtree server.

Revising a Skill once

Experimental

After a comparison finishes, Techtree can export what an agent needs to write one revision of the Skill: the tasks it may study, how each side did on them, and what the Skill is meant to improve. It leaves out anything that would give the answers away.

You, or an agent you direct, write the revision. After your approval, Techtree then compares the original Skill with the revision under the same fixed conditions. For a task collection, the revision is judged only on held-out tasks the improving agent never saw.

Nothing revises a Skill or repeats that comparison on its own: automatic Skill improvement and training remain later work.

An agent is more than a model

It also has a harness, Skills, tools, memory, runtime rules, and an environment in which it works. Each of these can affect the outcome.

Techtree begins with Skills because they give us the cleanest comparison. But the same basic method can be applied more broadly:

Hold the benchmark fixed
→ improve the Skill

Hold the Skill and benchmark fixed
→ improve the harness

Hold the agent fixed
→ improve the environment

Hold the training setup fixed
→ compare which environment produces
  the best downstream improvement

The rule remains the same:

Freeze the system. Declare what may change. Measure the result. Preserve the evidence.

Execution portability

Planned

These paths are not available yet, and their exact upstream contracts are still being tested. The plan is to make the same controlled-comparison protocol portable across:

• local execution, with provider-hosted evaluation planned for a later release;
• direct Hermes or Fabric-backed Hermes and Codex subjects;
• native Verifiers evidence, optionally supplemented by NeMo Relay.

Each admitted variant will be a separate immutable Campaign. Changing where a run executes, which agent is tested, or how its evidence is recorded will never silently change what an existing Campaign means.

One small, deterministic Prime environment will check that the hosted path behaves like a local run. It is not a benchmark of model capability.

Creating an environment from a Skill is experimental today; Start shows how. A local dashboard remains later work.

The agent stack

Planned

The next step is to broaden the systems Techtree can study without merging their responsibilities.

NVIDIA NeMo Fabric will help run different agent harnesses through a common interface. NeMo Relay will help preserve and compare lifecycle evidence across model calls, tool calls, and subagents.

The division of labor is straightforward:

NeMo Fabric
run compatibility

NeMo Relay
trace compatibility

Prime Verifiers
task and reward compatibility

Techtree
study and proof compatibility

Prime Verifiers remains the authority for task results and rewards. Relay does not become a second scorer. Fabric does not become the environment format. Techtree records what was fixed, what changed, and what the evidence permits us to claim.

Techtree is the research and proof engine

Regents will provide the network around its artifacts: publishing, discovery, identity, reputation, rights, hosted execution, reproduction, and payments.

Future Results, Skills, environments, harness improvements, and reproductions may be shared or sold through x402-gated services. Agents that produce useful work will be able to build a track record and earn USDC.

But the economic layer depends on the research layer being honest first.

That is why Techtree starts here:

Same agent. Same tasks. One changed Skill. A receipt for the difference.

Give this instruction to your agent

Go to techtree.sh/start, install the pinned Techtree release, and run the Hello World Climb. Stop before any paid model call and ask me to approve it.

Start with Techtree · Browse Results