Awesome-LLM-Long-Context-Modeling

📰 Must-read papers and blogs on LLM based Long Context Modeling 🔥

What is Awesome-LLM-Long-Context-Modeling?

📰 Must-read papers and blogs on LLM based Long Context Modeling 🔥

Overview of Awesome-LLM-Long-Context-Modeling

What it is

This is a curated repository of papers and blogs on long-context language modeling for large language models (LLMs). It is maintained by the community and organized into categories covering efficient attention, KV-cache optimization, recurrent transformers, state-space models, position encoding, long-context training, long-term memory, retrieval-augmented generation, in-context learning, context and model compression, long reasoning, long video and image, long-horizon agents, long-text generation, inference acceleration, benchmarks, and technical reports. The repository also includes a survey paper titled “A Comprehensive Survey on Long Context Language Modeling” (arXiv:2503.17407).

Key features

  • Curated list of must-read papers and blogs on LLM-based long context modeling.
  • Covers 20+ categories including attention, KV cache, training, memory, RAG, compression, reasoning, multimodal, agents, generation, acceleration, and evaluation.
  • Includes weekly and monthly paper updates with recent publications.
  • Provides a taxonomy diagram showing the structure of long-context modeling research.
  • Links to a companion survey paper and a separate dedicated repository for long-context language modeling.
  • Open to contributions from the GitHub community.

Use cases

Researchers and practitioners can use this repository to stay updated on the latest advances in long-context LLMs. It serves as a reference for literature reviews, paper selection, and understanding the landscape of long-context modeling techniques. It is also useful for identifying key papers in sub-areas such as efficient attention, KV-cache compression, long-context training, and evaluation benchmarks.

Getting started

See the repository README for instructions on browsing the paper list, contributing, and citing the survey.

License and pricing

The repository is licensed under MIT. It is open source and free to use.

Based on the project's public README and repository metadata.

Add this badge to your README

Featured on YunoDB Badge styles & help

Maintain this project? Show that it is featured on YunoDB.

Markdown[![Featured on YunoDB](https://yunodb.com/badge/xnhyacinthawesome-llm-long-context-modeling.svg)](https://yunodb.com/tools/xnhyacinthawesome-llm-long-context-modeling?utm_source=github&utm_medium=badge)
HTML<a href="https://yunodb.com/tools/xnhyacinthawesome-llm-long-context-modeling?utm_source=github&utm_medium=badge"><img src="https://yunodb.com/badge/xnhyacinthawesome-llm-long-context-modeling.svg" alt="Featured on YunoDB" height="20"></a>