What Assist is and how a request flows
Assist is the voice assistant built into Home Assistant. It was developed during the 2023 “Year of the Voice” push, and it can run entirely on your own hardware. Every spoken request passes through a voice pipeline with four stages:
- Wake word: the device notices a phrase such as “Okay Nabu.”
- Speech-to-text (STT): the audio is turned into text.
- Conversation agent: the text is matched to a command. The built-in agent handles common requests, and an AI agent can take over for anything it does not recognize.
- Text-to-speech (TTS): the reply is spoken back to you.
You set up pipelines under Settings > Voice assistants. Each assistant has its own language, STT engine, conversation agent, TTS voice and wake word. That means you can run one assistant in English and another in German.
Wake words: where detection happens
Home Assistant has two wake word systems, and they work differently:
- openWakeWord runs on the Home Assistant server. The satellite streams audio to Home Assistant continuously, so the server is the part doing the listening.
- microWakeWord runs on the device itself, usually an ESP32-S3. Audio is sent to Home Assistant only after the wake word is detected.
The stock wake words include “Okay Nabu,” “Hey Jarvis” and “Hey Mycroft.” openWakeWord also includes “Alexa.” The list changes between releases, so check what your version offers.
The Home Assistant Companion app for Android can also listen for wake words such as “Hey Nabu,” so a phone can work as a satellite even when its screen is locked.
Custom wake words
You can train a custom openWakeWord model in a Google Colab notebook, using synthetic speech generated with Piper. Expect the process to take about an hour, and expect some false triggers while you tune the model. A phrase of three or four syllables is a common starting point, because longer, distinctive phrases tend to trigger less often by accident.
Custom wake words on the Voice Preview Edition are less straightforward. They are set through the device’s ESPHome configuration, and community reports on the process depend on your Home Assistant version.
Speech recognition: Whisper or Speech-to-Phrase
Two local speech-to-text options cover most setups:
- Whisper (faster-whisper) handles open-ended speech. On a Raspberry Pi 4 it is slow, often several seconds per command. An Intel N100-class mini PC or a machine with a GPU is much better.
- Speech-to-Phrase handles a fixed set of commands and responds in under a second on a Raspberry Pi 4 or 5. It learns your sentence-trigger phrases, entity names and area names automatically.
The choice is mostly flexibility against speed and hardware. Speech-to-Phrase is the low-power option, but it only understands the phrases it knows. Whisper understands more, but it needs more hardware.
Satellites: what you can listen with
A voice satellite is any device with a microphone and speaker that sends your speech to Assist. Common choices include:
- Home Assistant Voice Preview Edition: launched in December 2024 at $59. It uses an ESP32-S3 with an XMOS audio chip, two microphones, an LED ring, a rotary dial, a hardware mute switch and a 3.5 mm audio output. Check the current price, since it may have changed. Its speaker is small, so it is not a replacement for an Echo if music quality matters.
- M5Stack ATOM Echo: cheap, but the audio quality is poor.
- ESP32-S3-BOX-3: another ESP32-S3 option for a DIY satellite.
- Raspberry Pi with the Wyoming satellite software: a Pi with a microphone and speaker can act as a satellite.
- Android phone: through the Companion app, as described above.
Echo and Google Nest speakers cannot be reflashed as Assist satellites. If you want an Echo to control Home Assistant instead, see how to add Alexa to Home Assistant.
Setting up a local assistant
- Install the speech services. Run Whisper or Speech-to-Phrase for speech recognition, and Piper for speech output. Each runs as an add-on or a Docker container and connects over the Wyoming protocol.
- Create an assistant. Go to Settings > Voice assistants and add one. Choose the language, the built-in conversation agent (named Home Assistant), your speech-to-text and text-to-speech services, and a wake word.
- Expose entities. Voice can only control entities that are exposed under Settings > Voice assistants > Expose. Aliases help when a device has a name that is hard to recognize.
- Test by typing first. Open Assist from the button at the top right of your dashboard and type a command. If typing works but speaking does not, the problem is in the speech part of the pipeline, not in the command.
- Assign the assistant to a satellite. Select the assistant on your satellite device, and it will respond to its wake word.
Once voice works, you can change how Assist speaks or add your own phrases.
The cloud option
Home Assistant Cloud, run by Nabu Casa, provides speech-to-text and text-to-speech through Microsoft Azure, and both are included in the subscription. It needs less setup on your side, but it still needs an internet connection. At the time of writing it costs about $6.50 a month or $65 a year in the US, and the price varies by region. Google Assistant has its own route, covered in how to add Google Assistant to Home Assistant.
Privacy and safety
A voice pipeline is only as local as its least local part. If the text-to-speech voice or an AI agent runs in the cloud, those requests leave your network, even when wake word and speech recognition run at home.
Be careful what you expose. Exposing locks, garage doors or alarms to voice lets anyone within earshot use them. Think twice before adding them, and keep the rest of your setup in mind when you do.
If you want an AI agent to handle requests that the built-in commands do not cover, see AI in Home Assistant for the options and their trade-offs.
Common misconceptions
- Wake word detection is not always on the device. With openWakeWord, it runs on the server.
- Fully local voice does not need an LLM. The built-in agent handles commands without one.
- Whisper is not fast on every Raspberry Pi. On a Pi 4 it is slow, and Speech-to-Phrase is the low-power option.
- The Voice Preview Edition is not an Echo replacement for music. Its speaker is small.
- Echo and Nest speakers cannot run Assist. They cannot be reflashed as satellites.