ai-avatar-system

🎭 AI Avatar / digital human platform β€” upload a photo, clone a voice, talk to any face in real time with lip-sync video. Open-source, self-hosted. Claude Β· Whi

What is ai-avatar-system?

🎭 AI Avatar / digital human platform β€” upload a photo, clone a voice, talk to any face in real time with lip-sync video. Open-source, self-hosted. Claude Β· Whisper Β· Chatterbox Β· MuseTalk.

Overview of ai-avatar-system

What it is

AvatarAI is an open-source, self-hosted platform for building photorealistic AI avatar conversations. Users upload a face photo, clone a voice from a short audio clip, and have real-time conversations with lip-sync video generated on every response. The pipeline runs from microphone input through Whisper speech-to-text, a streaming LLM (Claude, GPT-4o, or local models), Chatterbox text-to-speech, and MuseTalk lip-sync. It can run 100% locally or deploy to AWS GPU instances.

Key features

  • Zero-shot voice cloning from about 10 seconds of audio, supporting 23 languages
  • Token-streaming pipeline: the LLM streams live tokens while TTS and lip-sync run per sentence, so the first video chunk plays before the reply finishes
  • Barge-in: speaking or pressing stop mid-reply cancels the in-flight turn
  • Multi-LLM support: Claude with prompt caching, GPT-4o, or local models via Ollama / vLLM / LM Studio
  • Lip-sync video with MuseTalk V1.5, optional GFPGAN restore, and FFmpeg CPU fallback
  • Local-first storage mode with no AWS required for development
  • Production features: JWT + httpOnly-cookie auth, per-user rate limiting, Postgres + Alembic, S3/CloudFront, Prometheus, CI, and a pytest suite

Use cases

  • Building real-time AI avatar or digital human web services
  • Creating multi-user avatar chat products with authentication and conversation history
  • Running avatar conversations fully locally for privacy-sensitive use
  • Deploying GPU-backed real-time avatar systems on AWS

Getting started

See the repository README for Quick Start instructions. Prerequisites include Docker and Docker Compose v2+.

License and pricing

Licensed under MIT. The project is open-source and self-hosted; no pricing is stated in the repository metadata.

Based on the project's public README and repository metadata.

Add this badge to your README

Featured on YunoDB Badge styles & help

Maintain this project? Show that it is featured on YunoDB.

Markdown[![Featured on YunoDB](https://yunodb.com/badge/punithvtai-avatar-system.svg)](https://yunodb.com/tools/punithvtai-avatar-system?utm_source=github&utm_medium=badge)
HTML<a href="https://yunodb.com/tools/punithvtai-avatar-system?utm_source=github&utm_medium=badge"><img src="https://yunodb.com/badge/punithvtai-avatar-system.svg" alt="Featured on YunoDB" height="20"></a>