MOSS

An open-source, tool-augmented conversational language model from Fudan University

What is MOSS?

An open-source, tool-augmented conversational language model from Fudan University

Overview of MOSS

What it is

MOSS is an open-source conversational language model developed by Fudan University that supports bilingual dialogue in Chinese and English and tool invocation. The moss-moon series models have 16 billion parameters and can run on a single A100/A800 or two 3090 GPUs under FP16 precision, or on a single 3090 GPU under INT4/8 quantization. The base language model was pre-trained on approximately 700 billion Chinese, English, and code tokens, followed by dialogue instruction fine-tuning, plugin-enhanced learning, and human preference training to acquire multi-turn dialogue capabilities and the ability to use multiple plugins.

Key features

  • Supports bilingual Chinese and English dialogue and tool invocation.
  • Includes multiple plugin capabilities: search engine, text-to-image, calculator, and equation solving.
  • Offers quantized versions (INT4, INT8) for reduced memory usage (e.g., INT4 requires about 12GB VRAM for inference).
  • Provides multi-turn dialogue data and plugin-enhanced dialogue data for fine-tuning.
  • Includes a preference model (PM) for improved factuality, safety, and response quality.
  • Open-source models, data, and engineering solutions (deployment, frontend, backend).

Use cases

  • Simple math word problems
  • Solving equations
  • Generating images
  • Chinese language contexts
  • Code generation
  • Harmlessness (avoiding harmful content)

Getting started

git clone https://github.com/OpenMOSS/MOSS.git
cd MOSS
conda create --name moss python=3.8
conda activate moss
pip install -r requirements.txt

License and pricing

Licensed under Apache-2.0. Open-source and free to use.

Based on the project's public README and repository metadata.

Add this badge to your README

Featured on YunoDB Badge styles & help

Maintain this project? Show that it is featured on YunoDB.

Markdown[![Featured on YunoDB](https://yunodb.com/badge/openmossmoss.svg)](https://yunodb.com/tools/openmossmoss?utm_source=github&utm_medium=badge)
HTML<a href="https://yunodb.com/tools/openmossmoss?utm_source=github&utm_medium=badge"><img src="https://yunodb.com/badge/openmossmoss.svg" alt="Featured on YunoDB" height="20"></a>