sglang

SGLang is a high-performance serving framework for large language models and multimodal models.

What is sglang?

SGLang is a high-performance serving framework for large language models and multimodal models.

Overview of sglang

What it is

SGLang is an open-source inference framework for large language, vision-language, and diffusion models. It is optimized for agentic workloads, large-scale serving, and RL rollouts. SGLang Diffusion, a built-in image and video generation engine, is included in the same repository and Python package.

Key features

  • Open-source inference framework supporting large language, vision-language, and diffusion models.
  • Optimized for agentic workloads, large-scale serving, and reinforcement learning rollouts.
  • Built-in image and video generation engine (SGLang Diffusion).
  • Supports a wide range of hardware: NVIDIA (B200, H100, A100, RTX series, etc.), AMD (Instinct MI300X, etc.), Google TPU (v6e, v7), Intel (Arc GPUs, Xeon CPUs), Apple Silicon (Macs via Metal/MLX), Huawei Ascend (A2, A3, etc.), and Moore Threads (MTT S5000).
  • TPU serving via SGLang-TorchTPU (PyTorch-based) and SGL-JAX (JAX-based).
  • Ecosystem includes audio model serving (SGLang Omni), speculative decoding (SpecForge), RL training frameworks, and deployment tools.

Use cases

  • Serving large language models, vision-language models, and diffusion models for production workloads.
  • Agentic workloads requiring fast inference and tool use.
  • Large-scale serving with routing, load balancing, and cluster orchestration.
  • Reinforcement learning rollout generation.
  • Image and video generation using diffusion models.
  • Audio model serving for text-to-speech (TTS) and automatic speech recognition (ASR) via SGLang Omni.
  • Speculative decoding with trained draft models via SpecForge.

Getting started

Pull the Docker image:bashdocker pull lmsysorg/sglang:latestAlternatively, install SGLang in an activated Python environment with uv:bashuv pip install --prerelease=allow sglangAfter installation, launch your model. See the repository README for quickstart and cookbook instructions.

License and pricing

License: Apache-2.0. Pricing: open source (free).

Based on the project's public README and repository metadata.

Add this badge to your README

Featured on YunoDB Badge styles & help

Maintain this project? Show that it is featured on YunoDB.

Markdown[![Featured on YunoDB](https://yunodb.com/badge/sgl-projectsglang.svg)](https://yunodb.com/tools/sgl-projectsglang?utm_source=github&utm_medium=badge)
HTML<a href="https://yunodb.com/tools/sgl-projectsglang?utm_source=github&utm_medium=badge"><img src="https://yunodb.com/badge/sgl-projectsglang.svg" alt="Featured on YunoDB" height="20"></a>