System OneSystem One

A project to build with System One models

Smart home voice commands, decided on a Raspberry Pi

Turn a spoken command into one of your home's actions, such as lights, heating, music or a timer, on a Raspberry Pi with no cloud service, and ask again when the model is unsure.

Robotics and edgeIntermediate5 stepsSuggested by @biplov

The decision

A choice over your actions, each written as a short phrase: "turn the lights on", "turn the lights off", "set the heating", "play music", "set a timer", "none of these". Then a choice over the rooms, for commands that name one.

How to build it

  1. Run a speech-to-text model on the Pi, such as whisper.cpp or Vosk, and pass the text it hears as the state.
  2. Package Julia 1 on your computer with noulxp export julia, copy the package to the Pi, install noulxp[onnx] there and start noulxp serve. Julia 1 has 144M parameters and runs on a CPU.
  3. Map each action to a call to Home Assistant's API, or to your own scripts.
  4. Act above 0.85. Below it, say what you understood and wait for a yes.
  5. Measure the time from the end of speech to the action on your Pi, and keep a log of commands and corrections.

Make it better

Some actions, like unlocking a door or turning off an alarm, should be confirmed every time, whatever the probability. Then fine-tune on your log: a household's own phrasing is what a general model gets wrong.

More projects to build

All projects