align-anything
Align Anything: Training All-modality Model with Feedback
What is align-anything?
Align Anything: Training All-modality Model with Feedback
Overview of align-anything
What it is
Align Anything is an open-source framework for aligning any-modality large models (any-to-any models) with human intentions and values. It is developed by the PKU-Alignment Team and hosted on GitHub at PKU-Alignment/align-anything. The project provides a highly modular framework, supports various modality model fine-tuning, and implements different alignment algorithms including SFT, DPO, PPO, and others. It also includes a multi-modal CLI for image, audio, and video modalities, O1-like training based on DollyTails, and rule-based RL encouraged by Deepseek-R1.
Key features
- Highly modular framework allowing users to easily modify and customize code for different tasks - Various modality model fine-tuning for diverse multi-modal (image/video/audio) models - Different alignment algorithms, including SFT, DPO, PPO, and others - Multi-modal CLI for image, audio, and video modalities - O1-like training based on DollyTails - Rule-based RL encouraged by Deepseek-R1
Use cases
Align Anything can be used for aligning any-modality large models with human intentions and values. It supports alignment fine-tuning of models such as Qwen3, Qwen3-MoE, MiniCPM-o, Emu3, Janus-Series, and DeepSeek’s unified multimodal understanding and generation models. It also supports alignment training for Vision-Language-Action Models in embodied intelligence, Safe RLHF-V for text+image-to-text modality models, and learning from language feedback. The framework can be used on both Nvidia GPU and Huawei Ascend NPU.
Getting started
bash git clone [email protected]:PKU-Alignment/align-anything.git cd align-anything conda create -n align-anything python==3.11 conda activate align-anything For installation on Nvidia GPU or Huawei Ascend NPU, see the repository README.
License and pricing
The project is licensed under Apache-2.0 and is open source.
Based on the project's public README and repository metadata.