The decision
For each form element, a choice between filling an entity, checking, clicking or skipping. cua-s1-forms was built for exactly this, and at 706K parameters it adds almost nothing to the loop.
How to build it
- Drive a browser with Playwright and extract the form's elements with their labels.
- Give the model each element and the data you want entered.
- Act on its decision, then move to the next element.
- Test on forms you control first: sign-ups, surveys, internal tools.
Make it better
Pair it with a larger model only for the steps it is unsure about, and log those steps. They show where the small model needs more training.