Chinese-LLaMA-Alpaca-2

中文LLaMA-2 & Alpaca-2大模型二期项目 + 64K超长上下文模型 (Chinese LLaMA-2 & Alpaca-2 LLMs with 64K long context models)

What is Chinese-LLaMA-Alpaca-2?

中文LLaMA-2 & Alpaca-2大模型二期项目 + 64K超长上下文模型 (Chinese LLaMA-2 & Alpaca-2 LLMs with 64K long context models)

Overview of Chinese-LLaMA-Alpaca-2

What it is

This project is the second phase of the Chinese LLaMA & Alpaca large model series, based on Meta's Llama-2. It releases Chinese LLaMA-2 base models and Chinese Alpaca-2 instruction-tuned chat models. The models expand and optimize the Chinese vocabulary of the original Llama-2, use large-scale Chinese data for incremental pre-training, and support FlashAttention-2 training. Standard models support 4K context length, while long-context versions support 16K and 64K context. RLHF series models are fine-tuned with human preference alignment.

Key features

  • Expanded and optimized Chinese vocabulary (size: 55,296) for better Chinese text encoding/decoding - FlashAttention-2 for faster and more memory-efficient attention - Long context extension using PI (Position Interpolation) for 16K and YaRN for 64K context - Simplified Chinese-English bilingual system prompt for Alpaca-2 models - RLHF (Reinforcement Learning from Human Feedback) for improved value alignment - Supports 🤗transformers, llama.cpp, text-generation-webui, LangChain, privateGPT, vLLM

Use cases

  • Chinese LLaMA-2 base models: text continuation, given preceding text, generate following text - Chinese Alpaca-2 chat models: instruction understanding, question answering, writing, chatting, interaction - Long-context models: tasks requiring long document understanding, such as privateGPT - RLHF models: scenarios requiring correct value expression

Getting started

See the repository README for detailed instructions on model download, quantization, and deployment.

License and pricing

Licensed under Apache-2.0. The project is open source.

Based on the project's public README and repository metadata.

Add this badge to your README

Featured on YunoDB Badge styles & help

Maintain this project? Show that it is featured on YunoDB.

Markdown[![Featured on YunoDB](https://yunodb.com/badge/ymcuichinese-llama-alpaca-2.svg)](https://yunodb.com/tools/ymcuichinese-llama-alpaca-2?utm_source=github&utm_medium=badge)
HTML<a href="https://yunodb.com/tools/ymcuichinese-llama-alpaca-2?utm_source=github&utm_medium=badge"><img src="https://yunodb.com/badge/ymcuichinese-llama-alpaca-2.svg" alt="Featured on YunoDB" height="20"></a>