esp32-thinks
A multi-model Muse: personal AI voice assistant on ESP32 - wake word or knob, any OpenAI-compatible brain (STT/chat/TTS mix-and-match), streaming replies, out o
What is esp32-thinks?
A multi-model Muse: personal AI voice assistant on ESP32 - wake word or knob, any OpenAI-compatible brain (STT/chat/TTS mix-and-match), streaming replies, out of the box. Built on muse-gadget-sdk.
Overview of esp32-thinks
What it is
A multi-model Muse: personal AI voice assistant on ESP32 - wake word or knob, any OpenAI-compatible brain (STT/chat/TTS mix-and-match), streaming replies, out of the box. Built on Meta's open-source muse-gadget-sdk.
Key features
- Wake word + button: say “Hi, ESP” (esp-sr WakeNet10, runs on-chip) or hold the knob; both start a turn.
- Bring your own brain: three independent OpenAI-compatible endpoints (STT / chat / TTS) in one config file. Mix providers freely; presets:
openai,quicksilver,openmuse,custom. - Optional agent layer: the in-repo
openmuse/adds location, clock, weather, persistent memory and optional web search, owns the system prompt, and proxies STT. - Streaming voice replies: chat stream parsed at sentence boundaries (ASCII + CJK); each sentence synthesized the moment it lands and appended to audio already playing.
- Barge-in: press the knob while Muse is speaking; reply stops mid-sentence, in-flight requests cancelled, fresh turn starts immediately.
- Silence guard: quiet recording (interrupted turn, pocket press) discarded instead of becoming a hallucinated answer.
- Automatic stall recovery: outstanding I2S or LCD work gets a 20 s deadline, followed by zero-allocation task dump and reboot; wedged I2C expander (v0.2.11+) reboots after a minute of solid failure.
- Language following: answer in user's language, defaulting to English when unclear. English, Chinese and Korean tested.
- Headless provisioning: put Wi-Fi in config; device self-provisions on boot. No app, no BLE dance, no vendor account needed.
Use cases
Personal AI voice assistant on ESP32 hardware. Suitable for users who want to avoid vendor lock-in by choosing their own STT, chat, and TTS providers. Can be used with a local agent (openmuse) for location, weather, time, and memory. Supports English, Chinese, and Korean.
Getting started
bash # 0) One-time: ESP-IDF v6.0.x for esp32s3 (docs/REFERENCE.md §Prerequisites) git clone https://github.com/facebookincubator/muse-gadget-sdk.git git -C muse-gadget-sdk checkout 812c46f git clone https://github.com/raullenchai/esp32-thinks.git cd esp32-thinks # 1) Back up the factory NVS once — your restore path if anything goes sideways ./scripts/backup-nvs.sh ../muse-gadget-sdk /dev/ttyACM1 # use your port: ACM0 or ACM1 # 2) Apply the patch + configure your brain (pick a preset) ./apply.sh ../muse-gadget-sdk --preset quicksilver # or openai / openmuse / custom # 3) Build + flash (CH342-safe pacing; allow several minutes) ./build-and-flash.sh ../muse-gadget-sdk /dev/ttyACM1
License and pricing
MIT license. Open source. Pricing not stated beyond open source.}
Based on the project's public README and repository metadata.