LoongForge

A high-performance framework for training LLMs, VLMs, diffusion, and embodied models on NVIDIA GPUs and Kunlun XPUs. Supports Pi0.5, GR00T-N1.7, FastWAM, Cosm

What is LoongForge?

A high-performance framework for training LLMs, VLMs, diffusion, and embodied models on NVIDIA GPUs and Kunlun XPUs. Supports Pi0.5, GR00T-N1.7, FastWAM, Cosmos3, DreamZero, DeepSeek-V4, GLM-5.3, Kimi-K3 and more.

Overview of LoongForge

What it is

LoongForge is an open-source training framework developed by the Baidu AI Cloud Baige team. It is built to deliver faster training for mainstream LLMs, VLMs, diffusion, and embodied models. The framework is built on Megatron-LM and torch-native backends, with deep optimizations for training speed while keeping training loss aligned with the baseline. It supports pre-training, mid-training, SFT, and LoRA. LoongForge has been open-sourced from AIAK-Training-LLM, which powers enterprise customers' proprietary models and several open-source projects, with production runs reaching 5,000+ XPUs.

Key features

  • High performance: deep optimizations for training speed that keep training loss aligned with the baseline.
  • Easy to use: ready-to-run configs and launch scripts for 40+ open-source models, plus checkpoint conversion and data preprocessing tools.
  • Production-proven: open-sourced from AIAK-Training-LLM, powering enterprise customers' proprietary models and several open-source projects.
  • Multi-backend architecture: Megatron backend for LLM, VLM, and diffusion models; torch-native backend for embodied models (VLA and WAM).
  • Supports MoE parallelism, per-component heterogeneous parallelism, long-sequence optimizations, and FP8 communication optimization.
  • Embodied model support: unified data loading, per-model deep optimization, multiple distributed strategies (DDP, ZeRO-1, FSDP, HSDP), and unified evaluation in simulators.

Use cases

Training large language models (LLMs), vision-language models (VLMs), diffusion models, and embodied models (VLA and WAM). The framework is suitable for pre-training, mid-training, supervised fine-tuning (SFT), and LoRA-based fine-tuning. It can be used for both NVIDIA GPUs and Kunlun XPUs.

Getting started

See the repository README for installation and quick-start instructions.

License and pricing

Licensed under Apache-2.0. The project is open source and free to use.

Based on the project's public README and repository metadata.

Add this badge to your README

Featured on YunoDB Badge styles & help

Maintain this project? Show that it is featured on YunoDB.

Markdown[![Featured on YunoDB](https://yunodb.com/badge/baidu-baigeloongforge.svg)](https://yunodb.com/tools/baidu-baigeloongforge?utm_source=github&utm_medium=badge)
HTML<a href="https://yunodb.com/tools/baidu-baigeloongforge?utm_source=github&utm_medium=badge"><img src="https://yunodb.com/badge/baidu-baigeloongforge.svg" alt="Featured on YunoDB" height="20"></a>