EAGLE

Official Implementation of EAGLE-1 (ICML'24), EAGLE-2 (EMNLP'24), and EAGLE-3 (NeurIPS'25).

What is EAGLE?

Official Implementation of EAGLE-1 (ICML'24), EAGLE-2 (EMNLP'24), and EAGLE-3 (NeurIPS'25).

Overview of EAGLE

What it is

EAGLE (Extrapolation Algorithm for Greater Language-model Efficiency) is a speculative decoding method for large language models (LLMs) that accelerates text generation while provably maintaining the output distribution of the base model. The repository contains the official implementations of three versions: EAGLE-1 (accepted at ICML'24), EAGLE-2 (accepted at EMNLP'24), and EAGLE-3 (accepted at NeurIPS'25). EAGLE works by extrapolating the second-top-layer contextual feature vectors of LLMs. EAGLE-2 dynamically adjusts the draft tree structure using confidence scores from the draft model to approximate acceptance rates. EAGLE-3 replaces top-layer features with a fusion of low-, mid-, and high-level semantic features, removing the feature prediction constraint used in earlier versions.

Key features

  • EAGLE-1 achieves 2x speedup on gpt-fast, 3x faster than vanilla decoding (13B), 2x faster than Lookahead (13B), and 1.6x faster than Medusa (13B).
  • EAGLE-2 is 4x faster than vanilla decoding (13B) and 1.4x faster than EAGLE-1 (13B).
  • EAGLE-3 is 5.6x faster than vanilla decoding (13B) and 1.8x faster than EAGLE-1 (13B).
  • The method is trainable within 1-2 days and testable on 8x RTX 3090 GPUs.
  • It is combinable with other parallel techniques such as vLLM, DeepSpeed, Mamba, FlashAttention, quantization, and hardware optimization.
  • EAGLE has been integrated into multiple mainstream LLM serving frameworks including AMD ROCm, AWS NeuronX, Intel Extension for Transformers, NVIDIA TensorRT-LLM, SGLang, and vLLM.

Use cases

EAGLE is designed for accelerating autoregressive decoding of large language models in production or research settings. It is suitable for scenarios where generation speed is critical while maintaining the exact output distribution of the base model. The method can be applied to any LLM that supports the EAGLE framework, including models from the Vicuna, LLaMA, Qwen, DeepSeek, and Mixtral families. It is particularly useful for deployment on GPU clusters with limited resources, as it can be trained and tested on 8x RTX 3090 GPUs.

Getting started

bashgit clone https://github.com/SafeAILab/EAGLE.gitcd EAGLEpython -m venv ~/venvs/ea_envsource ~/venvs/ea_env/bin/activatepip install -r requirements.txt

License and pricing

The license is not stated in the repository metadata. The project is listed as open source.

Based on the project's public README and repository metadata.

Add this badge to your README

Featured on YunoDB Badge styles & help

Maintain this project? Show that it is featured on YunoDB.

Markdown[![Featured on YunoDB](https://yunodb.com/badge/safeailabeagle.svg)](https://yunodb.com/tools/safeailabeagle?utm_source=github&utm_medium=badge)
HTML<a href="https://yunodb.com/tools/safeailabeagle?utm_source=github&utm_medium=badge"><img src="https://yunodb.com/badge/safeailabeagle.svg" alt="Featured on YunoDB" height="20"></a>