Hermes Routing v1
A Laya typed-decision model, fine-tuned with System One Studio on an Apple silicon Mac. It answers the questions below in a single forward pass, with calibrated probabilities and zero generated tokens.
Base model: convaiinnovations/laya · method: lora, proper objective · trained in
56 minutes on an Apple silicon Mac.
Measured on the held-out test split
| Metric | Base model | This model |
|---|---|---|
| Accuracy | 57.8% | 80.0% [66.2%–89.1%] |
| Calibration error (ECE) | 0.233 | 0.062 |
| Log loss | 1.462 | 0.483 |
| Brier score | 0.576 | 0.149 |
| Decisions scored | 45 | 45 |
Fine-tuning fixed 13 test decisions the base model got wrong and broke 3 it got right (exact McNemar p = 0.021).
Test rows were never trained on. Accuracy intervals are Wilson intervals; the paired test is an exact McNemar test between the base and the fine-tuned model on the same rows.
Use it
pip install systemonemodels
systemone pull vorthalis/hermes-routing-v1
import json
import laya_mlx as laya # pip install laya-mlx, on Apple silicon
from systemone import snapshot_download
path = snapshot_download("vorthalis/hermes-routing-v1")
agent = laya.load(str(path))
questions = json.loads((path / "questions.json").read_text())
print(agent.predict("your text here", questions)["answers"])
Ask it these questions: the instructions and option texts are part of the model's input, so changing them changes the task it was tuned for.
{
"platform": {
"type": "choice",
"instructions": "Select the best agent for this task",
"criteria": [
"coder",
"researcher",
"analyst",
"writer",
"architect"
]
},
"parallelizable": {
"type": "noul",
"instructions": "Can run in parallel with other tasks?"
},
"complexity": {
"type": "score",
"instructions": "Rate the task complexity",
"criteria": [
"trivial",
"moderate",
"complex",
"critical"
]
}
}