flama
The production framework for Predictive and Generative AI. Serve any model as an API in one line, with OpenAI/Anthropic/Ollama-compatible endpoints, a built-in
What is flama?
The production framework for Predictive and Generative AI. Serve any model as an API in one line, with OpenAI/Anthropic/Ollama-compatible endpoints, a built-in chat UI, and native MCP.
Overview of flama
What it is
flama is the production framework for Predictive and Generative AI. It turns any model into a production API in a single line, using a Rust-powered core. Models from mainstream frameworks are packaged into a single portable .flm file, so every model looks the same to the API. The framework serves models over HTTP and supports OpenAI-, Anthropic-, and Ollama-compatible endpoints, a built-in chat UI, and native Model Context Protocol (MCP). flama is published on PyPI with native wheels for Linux, macOS, and Windows, requiring no Rust toolchain.
Key features
- Package models from scikit-learn, TensorFlow, PyTorch, or an LLM into a portable
.flmfile. - Download and package any model from the HuggingFace Hub with one command.
- Serve LLMs with OpenAI-, Anthropic-, and Ollama-compatible endpoints side by side.
- Every served model includes a streaming chat UI at
/chat/with Markdown, LaTeX, and Mermaid support. - Expose tools, resources, and prompts to AI agents via MCP with a single decorator and type hints.
- Rust-powered core for routing, JSON encoding, request parsing, and compression, shipped as plain wheels.
- Production-ready from the start: run packaged models via CLI, Python, a spec file, or inside a container.
Use cases
- Serving predictive models (e.g., scikit-learn, TensorFlow, PyTorch) as HTTP APIs with OpenAPI schema and Swagger UI.
- Serving generative LLMs as APIs with built-in streaming chat UI and multi-dialect endpoints (OpenAI, Anthropic, Ollama).
- Deploying multiple models from a single configuration file (
flama.json) for reviewable pull-request-based deployments. - Offline model inspection, testing, and inference scoring using the self-describing
.flmfile without a server. - Gating model promotion in CI pipelines using metrics embedded inside the
.flmfile.
Getting started
Install flama with Pydantic: commandline pip install flama[pydantic] Optional extras include flama[typesystem], flama[database], flama[llm], and flama[full]. To serve a predictive model, package it into a .flm file using flama.dump(), then run: commandline flama serve --model file=model.flm,url=/model,name=iris To serve a generative model: commandline pip install flama[llm,pydantic] flama get --family llm --source huggingface mlx-community/gemma-4-E2B-it-qat-4bit flama serve --model file=mlx-community_gemma-4-E2B-it-qat-4bit.flm,url=/,name=gemma For full details, see the repository README.
License and pricing
Licensed under Apache-2.0. The project is open-source with no stated pricing.
Based on the project's public README and repository metadata.