sie
Open-source inference server and production cluster for all the models your agent needs.
What is sie?
Open-source inference server and production cluster for all the models your agent needs.
Overview of sie
What it is
SIE (Superlinked Inference Engine) is an open-source inference server and production cluster that serves models for agent tasks through a single API. It replaces multiple separate model servers with one system that loads over 100 models on demand.
Key features
- OpenAI-compatible API endpoints:
/v1/embeddings,/v1/chat/completions,/v1/completions,/v1/responses - Pre-configured model catalog including Stella, SPLADE, Qwen3, GLiNER, SigLIP, and others; embedding and retrieval models benchmarked on MTEB - Serves multiple models simultaneously with on-demand loading and LRU eviction - Ships Kubernetes and Helm deployment configs for the load-balancing gateway, KEDA autoscaling, and Grafana dashboards - Integrates with LangChain, LlamaIndex, Haystack, DSPy, CrewAI, Chroma, Qdrant, Weaviate, and LanceDB
Use cases
SIE runs the inference behind a whole agent. Tasks include search (embed, match, rerank), document-to-markdown conversion, structured output extraction, content safety classification, the agent loop itself, translation, image captioning and object detection, and audio transcription.
Getting started
bash # macOS (Apple Silicon) or Linux, native (requires Python 3.12) pip install 'sie-server[local]' && sie-server serve # Linux, NVIDIA GPU docker run --gpus all -p 8080:8080 -v sie-hf-cache:/app/.cache/huggingface ghcr.io/superlinked/sie-server:latest-cuda12-default # Linux, CPU docker run -p 8080:8080 -v sie-hf-cache:/app/.cache/huggingface ghcr.io/superlinked/sie-server:latest-cpu-default See the repository README for additional Docker images and SDK installation instructions.
License and pricing
The license is Apache-2.0. The project is open source.
Based on the project's public README and repository metadata.