kalosm

Instant, controllable, local pre-trained AI models in Rust

What is kalosm?

Instant, controllable, local pre-trained AI models in Rust

Overview of kalosm

What it is

Kalosm is an ecosystem of Rust crates designed to simplify the development of applications that use local or remote AI models. It provides a simple interface for interacting with pre-trained language, audio, and image models. The repository also includes Fusor, a compiler-backed runtime for quantized ML inference on CPUs and WebGPU accelerators.

Key features

  • Supports a variety of pre-trained models including Llama, Mistral, Phi, Whisper, Segment Anything, and Bert, with quantized and GPU-accelerated options.
  • Provides utilities for extracting, formatting, and retrieving context for LLMs from txt, html, docx, md, and pdf files, as well as chunking and vector database integration for semantic search.
  • Supports audio transcription from microphone or file, and web crawling/scraping.
  • Enables structured generation with arbitrary parsers, using a custom parser engine and structure-aware acceleration. Users can derive Parse and Schema on Rust types to constrain model output with regex or custom grammars.
  • Includes a compiler-backed runtime (Fusor) that uses an e-graph compiler to fuse operation chains into optimized kernels for quantized ML inference.

Use cases

Kalosm is suitable for building chatbots, semantic search applications, audio transcription tools, image segmentation systems, and any application that requires local or remote AI model inference with structured output constraints.

Getting started

  1. Install Rust.2. Create a new project:shcargo new kalosm-hello-worldcd ./kalosm-hello-world3. Add Kalosm as a dependency:shcargo add kalosm --features llamacargo add tokio --features full4. Add the example code to main.rs (see repository README for the full example).5. Run with:shcargo run --release

License and pricing

Kalosm is licensed under Apache-2.0 and is open source.

Based on the project's public README and repository metadata.

Add this badge to your README

Featured on YunoDB Badge styles & help

Maintain this project? Show that it is featured on YunoDB.

Markdown[![Featured on YunoDB](https://yunodb.com/badge/floneumkalosm.svg)](https://yunodb.com/tools/floneumkalosm?utm_source=github&utm_medium=badge)
HTML<a href="https://yunodb.com/tools/floneumkalosm?utm_source=github&utm_medium=badge"><img src="https://yunodb.com/badge/floneumkalosm.svg" alt="Featured on YunoDB" height="20"></a>