Qwen
The official repo of Qwen (通义千问) chat & pretrained large language model proposed by Alibaba Cloud.
What is Qwen?
The official repo of Qwen (通义千问) chat & pretrained large language model proposed by Alibaba Cloud.
Overview of Qwen
What it is
Qwen (通义千问) is a series of large language models proposed by Alibaba Cloud. It includes base language models (Qwen-1.8B, Qwen-7B, Qwen-14B, Qwen-72B) and chat models (Qwen-Chat). The models are pretrained on up to 3 trillion tokens of multilingual data with a focus on Chinese and English. They achieve competitive performance on benchmark datasets such as MMLU, C-Eval, GSM8K, and HumanEval. The repository provides tools for inference, quantization, finetuning, deployment, and tool use. Note: this repo is no longer actively maintained due to the release of Qwen2.
Key features
- Base and chat models in four sizes: 1.8B, 7B, 14B, 72B.
- Pretrained on up to 3 trillion tokens; supports 32K context length for 1.8B, 7B, and 72B models.
- Quantized versions (Int4, Int8) for reduced GPU memory usage.
- Supports full-parameter finetuning, LoRA, and Q-LoRA.
- Supports tool use, agent, and code interpreter capabilities.
- Provides inference performance statistics and deployment examples with vLLM and FastChat.
Use cases
- Chat, content creation, information extraction, summarization, translation, coding, and math problem solving.
- Tool use, agent, and code interpreter.
- Long-context understanding evaluation.
Getting started
See the repository README for installation and quick start instructions.
License and pricing
License: Apache-2.0. Pricing: open source (free).
Based on the project's public README and repository metadata.