speech-to-speech

Build voice agents with open-source models

What is speech-to-speech?

Build voice agents with open-source models

Overview of speech-to-speech

What it is

Speech-to-Speech is a low-latency voice agent built from open-source models that runs entirely on your computer. It requires no API key and no cloud service. The default pipeline consists of Silero VAD, Parakeet TDT, Qwen3.5-4B, and Qwen3-TTS, and every stage is swappable. The server speaks the OpenAI Realtime protocol over WebSocket and WebRTC, so applications built for OpenAI's Realtime API can point at your own machine instead. This pipeline runs in production as the conversation backend for thousands of Reachy Mini robots.

Key features

  • Runs entirely on your computer with a single command; no API key or cloud service needed.
  • Default pipeline: Silero VAD → Parakeet TDT → Qwen3.5-4B → Qwen3-TTS, with every stage swappable.
  • Supports multiple STT, LLM, and TTS backends (e.g., Whisper, Transformers, Kokoro, Qwen3-TTS, etc.).
  • Server implements the OpenAI Realtime protocol over WebSocket and WebRTC.
  • Commands: serve (runs pipeline server), talk (microphone/speaker client), local (combines both in-process).
  • Optional features: echo cancellation, speaker diarization, tool calling, interruption handling.

Use cases

  • Building voice agents that run locally without cloud dependencies.
  • Developing applications that use the OpenAI Realtime API but want to run the backend on a local machine.
  • Powering conversational interfaces for robots (e.g., Reachy Mini).
  • Experimenting with different speech-to-text, language model, and text-to-speech combinations.

Getting started

Use CPython 3.11–3.14 (3.11 recommended): bash python3 -m venv .venv source .venv/bin/activate pip install speech-to-speech speech-to-speech local Then talk: speak, pause, and the assistant answers through your speakers. Stop with Ctrl+C. The first run downloads the models, so give it a few minutes. On Ubuntu, first install audio libraries with sudo apt-get install libportaudio2 libsndfile1. On Windows, activate the environment with .venv/Scripts/Activate.ps1.

License and pricing

The license is Apache-2.0. The software is open source and free to use.

Based on the project's public README and repository metadata.

Add this badge to your README

Featured on YunoDB Badge styles & help

Maintain this project? Show that it is featured on YunoDB.

Markdown[![Featured on YunoDB](https://yunodb.com/badge/huggingfacespeech-to-speech.svg)](https://yunodb.com/tools/huggingfacespeech-to-speech?utm_source=github&utm_medium=badge)
HTML<a href="https://yunodb.com/tools/huggingfacespeech-to-speech?utm_source=github&utm_medium=badge"><img src="https://yunodb.com/badge/huggingfacespeech-to-speech.svg" alt="Featured on YunoDB" height="20"></a>