MOSS
An open-source, tool-augmented conversational language model from Fudan University
What is MOSS?
An open-source, tool-augmented conversational language model from Fudan University
Overview of MOSS
What it is
MOSS is an open-source conversational language model developed by Fudan University that supports bilingual dialogue in Chinese and English and tool invocation. The moss-moon series models have 16 billion parameters and can run on a single A100/A800 or two 3090 GPUs under FP16 precision, or on a single 3090 GPU under INT4/8 quantization. The base language model was pre-trained on approximately 700 billion Chinese, English, and code tokens, followed by dialogue instruction fine-tuning, plugin-enhanced learning, and human preference training to acquire multi-turn dialogue capabilities and the ability to use multiple plugins.
Key features
- Supports bilingual Chinese and English dialogue and tool invocation.
- Includes multiple plugin capabilities: search engine, text-to-image, calculator, and equation solving.
- Offers quantized versions (INT4, INT8) for reduced memory usage (e.g., INT4 requires about 12GB VRAM for inference).
- Provides multi-turn dialogue data and plugin-enhanced dialogue data for fine-tuning.
- Includes a preference model (PM) for improved factuality, safety, and response quality.
- Open-source models, data, and engineering solutions (deployment, frontend, backend).
Use cases
- Simple math word problems
- Solving equations
- Generating images
- Chinese language contexts
- Code generation
- Harmlessness (avoiding harmful content)
Getting started
git clone https://github.com/OpenMOSS/MOSS.git
cd MOSS
conda create --name moss python=3.8
conda activate moss
pip install -r requirements.txt
License and pricing
Licensed under Apache-2.0. Open-source and free to use.
Based on the project's public README and repository metadata.