UForm

Pocket-Sized Multimodal AI for content understanding and generation across multilingual texts, images, and πŸ”œ video, up to 5x faster than OpenAI CLIP and LLaVA

What is UForm?

Pocket-Sized Multimodal AI for content understanding and generation across multilingual texts, images, and πŸ”œ video, up to 5x faster than OpenAI CLIP and LLaVA πŸ–ΌοΈ & πŸ–‹οΈ

Overview of UForm

What it is

UForm is a multimodal AI library for content understanding and generation. It provides compact pre-trained transformer models that handle multilingual texts, images, and future video support. The library includes both embedding models (for search and retrieval) and generative models (for chat, image captioning, and visual question answering). It supports ONNX, CoreML, and PyTorch runtimes, and can be deployed from servers to smartphones.

Key features

  • Tiny embeddings: 64-dimensional Matryoshka-style embeddings for fast search.
  • Throughput: Inference speed is 2-4x faster than competitors due to small model size.
  • Portable: Native ONNX support for easy deployment across platforms.
  • Quantization aware: Embeddings can be down-cast from f32 to i8 with minimal recall loss.
  • Multilingual: Trained on a balanced dataset covering over 20 languages.

Use cases

  • Content understanding and search across images and text in multiple languages.
  • Image captioning and visual question answering (VQA) using generative models.
  • Chat and conversational AI with small generative models.
  • Edge and mobile deployment where model size and speed are critical.

Getting started

Install the library with pip:pip install uformFor embedding models, load and use as shown in the repository README. For generative models, use the Hugging Face transformers library with the model name unum-cloud/uform-gen2-dpo. See the repository README for full examples.

License and pricing

The project is licensed under Apache-2.0 and is open source.

Based on the project's public README and repository metadata.

Add this badge to your README

Featured on YunoDB Badge styles & help

Maintain this project? Show that it is featured on YunoDB.

Markdown[![Featured on YunoDB](https://yunodb.com/badge/unum-clouduform.svg)](https://yunodb.com/tools/unum-clouduform?utm_source=github&utm_medium=badge)
HTML<a href="https://yunodb.com/tools/unum-clouduform?utm_source=github&utm_medium=badge"><img src="https://yunodb.com/badge/unum-clouduform.svg" alt="Featured on YunoDB" height="20"></a>