kalosm
Instant, controllable, local pre-trained AI models in Rust
What is kalosm?
Instant, controllable, local pre-trained AI models in Rust
Overview of kalosm
What it is
Kalosm is an ecosystem of Rust crates designed to simplify the development of applications that use local or remote AI models. It provides a simple interface for interacting with pre-trained language, audio, and image models. The repository also includes Fusor, a compiler-backed runtime for quantized ML inference on CPUs and WebGPU accelerators.
Key features
- Supports a variety of pre-trained models including Llama, Mistral, Phi, Whisper, Segment Anything, and Bert, with quantized and GPU-accelerated options.
- Provides utilities for extracting, formatting, and retrieving context for LLMs from txt, html, docx, md, and pdf files, as well as chunking and vector database integration for semantic search.
- Supports audio transcription from microphone or file, and web crawling/scraping.
- Enables structured generation with arbitrary parsers, using a custom parser engine and structure-aware acceleration. Users can derive
ParseandSchemaon Rust types to constrain model output with regex or custom grammars. - Includes a compiler-backed runtime (Fusor) that uses an e-graph compiler to fuse operation chains into optimized kernels for quantized ML inference.
Use cases
Kalosm is suitable for building chatbots, semantic search applications, audio transcription tools, image segmentation systems, and any application that requires local or remote AI model inference with structured output constraints.
Getting started
- Install Rust.2. Create a new project:
shcargo new kalosm-hello-worldcd ./kalosm-hello-world3. Add Kalosm as a dependency:shcargo add kalosm --features llamacargo add tokio --features full4. Add the example code tomain.rs(see repository README for the full example).5. Run with:shcargo run --release
License and pricing
Kalosm is licensed under Apache-2.0 and is open source.
Based on the project's public README and repository metadata.