H2O-Lightning-4B
H2O.ai's open decision model. A Qwen3.5-4B fine-tune served on unmodified vLLM behind a small shim with a TypeSafe-compatible /v1/systemone API; each choice, score or noul question is one forward pass reading one token, over text or, from v1.2, images.
Not a chat model: it returns a probability for every option and no text. Every number is the maker's own run. On JevBench (a third-party benchmark), run with JevBench's own CLI on the 231 public items, it scores 205/231 (easy 48/48, standard 71/72, hard 86/111); a JevBench bench request (issue #181) is open with no third-party result yet. Calibration was measured only on the maker's 1,877-question image set (87.4%, ECE 0.017); text yes/no probabilities are very sharp (none of 74 between 0.2 and 0.8). Mostly evaluated in English; works best with 16 or fewer options (up to 255 accepted). No video input. The card says no benchmark data was used in training. Tags v1.0, v1.1, v1.2 and v1.2.1 were published 2 to 4 October.
What it decides
- choice — picks one option from a set
- score — places the input on an ordered scale
- noul — answers a yes/no question with one probability
At a glance
| Parameters | 4B |
| Base model | qwen/qwen3.5-4b |
| Maker | H2O.ai |
| Released | 2026-10-02 |
| License | apache-2.0 |
| Reported accuracy | 88.7% |
| Reported latency | 33 ms p50 / 35 ms p95 serial on the standard tier, one RTX PRO 4500 Blackwell 32 GB, vLLM 0.30.0 |
Get the weights
pip install systemonemodels
systemone pull h2o-ai/h2o-lightning
The files are served from the maker's Hugging Face repository, h2oai/h2o-lightning-4b, and verified against the checksums recorded here.
Read more
This page was opened by System One for H2O.ai, who can claim the organisation and take it over at any time.