ai-avatar-system
π AI Avatar / digital human platform β upload a photo, clone a voice, talk to any face in real time with lip-sync video. Open-source, self-hosted. Claude Β· Whi
What is ai-avatar-system?
π AI Avatar / digital human platform β upload a photo, clone a voice, talk to any face in real time with lip-sync video. Open-source, self-hosted. Claude Β· Whisper Β· Chatterbox Β· MuseTalk.
Overview of ai-avatar-system
What it is
AvatarAI is an open-source, self-hosted platform for building photorealistic AI avatar conversations. Users upload a face photo, clone a voice from a short audio clip, and have real-time conversations with lip-sync video generated on every response. The pipeline runs from microphone input through Whisper speech-to-text, a streaming LLM (Claude, GPT-4o, or local models), Chatterbox text-to-speech, and MuseTalk lip-sync. It can run 100% locally or deploy to AWS GPU instances.
Key features
- Zero-shot voice cloning from about 10 seconds of audio, supporting 23 languages
- Token-streaming pipeline: the LLM streams live tokens while TTS and lip-sync run per sentence, so the first video chunk plays before the reply finishes
- Barge-in: speaking or pressing stop mid-reply cancels the in-flight turn
- Multi-LLM support: Claude with prompt caching, GPT-4o, or local models via Ollama / vLLM / LM Studio
- Lip-sync video with MuseTalk V1.5, optional GFPGAN restore, and FFmpeg CPU fallback
- Local-first storage mode with no AWS required for development
- Production features: JWT + httpOnly-cookie auth, per-user rate limiting, Postgres + Alembic, S3/CloudFront, Prometheus, CI, and a pytest suite
Use cases
- Building real-time AI avatar or digital human web services
- Creating multi-user avatar chat products with authentication and conversation history
- Running avatar conversations fully locally for privacy-sensitive use
- Deploying GPU-backed real-time avatar systems on AWS
Getting started
See the repository README for Quick Start instructions. Prerequisites include Docker and Docker Compose v2+.
License and pricing
Licensed under MIT. The project is open-source and self-hosted; no pricing is stated in the repository metadata.
Based on the project's public README and repository metadata.