esp32-thinks

A multi-model Muse: personal AI voice assistant on ESP32 - wake word or knob, any OpenAI-compatible brain (STT/chat/TTS mix-and-match), streaming replies, out o

What is esp32-thinks?

A multi-model Muse: personal AI voice assistant on ESP32 - wake word or knob, any OpenAI-compatible brain (STT/chat/TTS mix-and-match), streaming replies, out of the box. Built on muse-gadget-sdk.

Overview of esp32-thinks

What it is

A multi-model Muse: personal AI voice assistant on ESP32 - wake word or knob, any OpenAI-compatible brain (STT/chat/TTS mix-and-match), streaming replies, out of the box. Built on Meta's open-source muse-gadget-sdk.

Key features

  • Wake word + button: say “Hi, ESP” (esp-sr WakeNet10, runs on-chip) or hold the knob; both start a turn.
  • Bring your own brain: three independent OpenAI-compatible endpoints (STT / chat / TTS) in one config file. Mix providers freely; presets: openai, quicksilver, openmuse, custom.
  • Optional agent layer: the in-repo openmuse/ adds location, clock, weather, persistent memory and optional web search, owns the system prompt, and proxies STT.
  • Streaming voice replies: chat stream parsed at sentence boundaries (ASCII + CJK); each sentence synthesized the moment it lands and appended to audio already playing.
  • Barge-in: press the knob while Muse is speaking; reply stops mid-sentence, in-flight requests cancelled, fresh turn starts immediately.
  • Silence guard: quiet recording (interrupted turn, pocket press) discarded instead of becoming a hallucinated answer.
  • Automatic stall recovery: outstanding I2S or LCD work gets a 20 s deadline, followed by zero-allocation task dump and reboot; wedged I2C expander (v0.2.11+) reboots after a minute of solid failure.
  • Language following: answer in user's language, defaulting to English when unclear. English, Chinese and Korean tested.
  • Headless provisioning: put Wi-Fi in config; device self-provisions on boot. No app, no BLE dance, no vendor account needed.

Use cases

Personal AI voice assistant on ESP32 hardware. Suitable for users who want to avoid vendor lock-in by choosing their own STT, chat, and TTS providers. Can be used with a local agent (openmuse) for location, weather, time, and memory. Supports English, Chinese, and Korean.

Getting started

bash # 0) One-time: ESP-IDF v6.0.x for esp32s3 (docs/REFERENCE.md §Prerequisites) git clone https://github.com/facebookincubator/muse-gadget-sdk.git git -C muse-gadget-sdk checkout 812c46f git clone https://github.com/raullenchai/esp32-thinks.git cd esp32-thinks # 1) Back up the factory NVS once — your restore path if anything goes sideways ./scripts/backup-nvs.sh ../muse-gadget-sdk /dev/ttyACM1 # use your port: ACM0 or ACM1 # 2) Apply the patch + configure your brain (pick a preset) ./apply.sh ../muse-gadget-sdk --preset quicksilver # or openai / openmuse / custom # 3) Build + flash (CH342-safe pacing; allow several minutes) ./build-and-flash.sh ../muse-gadget-sdk /dev/ttyACM1

License and pricing

MIT license. Open source. Pricing not stated beyond open source.}

Based on the project's public README and repository metadata.

Add this badge to your README

Featured on YunoDB Badge styles & help

Maintain this project? Show that it is featured on YunoDB.

Markdown[![Featured on YunoDB](https://yunodb.com/badge/raullenchaiesp32-thinks.svg)](https://yunodb.com/tools/raullenchaiesp32-thinks?utm_source=github&utm_medium=badge)
HTML<a href="https://yunodb.com/tools/raullenchaiesp32-thinks?utm_source=github&utm_medium=badge"><img src="https://yunodb.com/badge/raullenchaiesp32-thinks.svg" alt="Featured on YunoDB" height="20"></a>