Ask-Anything
[CVPR2024 Highlight][VideoChatGPT] ChatGPT with video understanding! And many more supported LMs such as miniGPT4, StableLM, and MOSS.
What is Ask-Anything?
[CVPR2024 Highlight][VideoChatGPT] ChatGPT with video understanding! And many more supported LMs such as miniGPT4, StableLM, and MOSS.
Overview of Ask-Anything
What it is
Ask-Anything is a project that provides a family of video chat models called VideoChat. It enables ChatGPT and other large language models (LLMs) to understand video content. The project includes multiple versions, such as VideoChat2, which was accepted at CVPR2024 as a Highlight. It supports various LLMs including ChatGPT, StableLM, MOSS, and MiniGPT-4. The repository is hosted at OpenGVLab on GitHub and is licensed under MIT.
Key features
- Supports multiple LLMs: ChatGPT, StableLM, MOSS, MiniGPT-4, and Vicuna.
- VideoChat2 is a robust baseline built on UMT and Vicuna-v0, achieving state-of-the-art results on several video benchmarks.
- Includes VideoChat2_HD, fine-tuned with high-resolution data for better detailed captioning and diverse task handling.
- Provides VideoChat2_phi3 and VideoChat2_mistral for faster or more capable models.
- Releases 2 million diverse instruction data for effective tuning.
- Includes MVBench, a comprehensive multi-modal video understanding benchmark accepted at CVPR2024.
Use cases
- Building end-to-end chatbots that can understand and discuss video content.
- Integrating video understanding into applications that use ChatGPT, StableLM, MOSS, or MiniGPT-4.
- Research and development in video-language models, including long-form video understanding and streaming video.
- Benchmarking video understanding capabilities using MVBench.
Getting started
See the repository README for installation and quick-start instructions.
License and pricing
The project is licensed under MIT and is open source.
Based on the project's public README and repository metadata.