Give your AI agent a calibrated classifier as an MCP tool

The decision

The questions your agent keeps asking its LLM: what does this customer want, is this urgent, should a person see it. The MCP server offers one tool, decide. The agent sends a state and typed questions, each a choice, a score or a noul, and gets back a probability for every option as structured output instead of a sentence.

How to build it

1. Install the two tools and pull a small model: pip install systemonemodels "noulxp[export,onnx]", then systemone pull supersonic-labs/julia-1 --dest ./models.
2. Package it: noulxp export julia ./models/julia-1 ./julia-noulxp. The package holds weights, a template and a calibration; nothing in it is executed.
3. Add it to your client. In Claude Code: claude mcp add julia-1 -- noulxp mcp /absolute/path/to/julia-noulxp. Claude Desktop and Cursor run the same command from their MCP configuration.
4. Tell the agent what the probabilities mean: act above 0.9, act and mention the doubt between 0.6 and 0.9, and hand over to a person below 0.6.
5. Read a day of tool calls, and note which questions the agent asks most. Those are the ones worth fine-tuning for.

Make it better

A threshold only works if 0.9 means right nine times in ten. Label 50 requests like yours, fit the model's confidence to them with noulxp calibrate, and start the server with --calibration so the agent sees the fitted probabilities.