laya-mlx

Native MLX runtime for Laya typed decision models — 7–14 ms short decisions on M3 Max. No text generation, PyTorch, or cloud API.

What is laya-mlx?

Native MLX runtime for Laya typed decision models — 7–14 ms short decisions on M3 Max. No text generation, PyTorch, or cloud API.

Overview of laya-mlx

What it is

Laya-MLX is a native MLX runtime for Laya typed decision models. It runs on Apple Silicon without PyTorch, Transformers, or a cloud API. It answers constrained questions (pick an option, score against a rubric, judge true or false) with calibrated probabilities from one bidirectional forward pass. It generates no tokens and returns no JSON to parse.

Key features

  • Runs Laya weights in MLX, without PyTorch, Transformers, or a cloud API.
  • Supports three question types: choice (probability per option), score (probability per level and expected level), and noul (P(true)).
  • Includes language routing, batch prediction, confidence gate, large choice set handling (shortlist and tournament), and throughput options (compile, pad_to_multiple, cache_prompts).
  • Provides CLI, HTTP server, MCP server, evaluation and calibration tools, and integrations with LangChain, LlamaIndex, CrewAI, and Agno (as extras).
  • Export functionality to write a standalone MLX checkpoint.
  • Performance: 13.4 ms median for a short English decision, 7.4 ms with the multilingual checkpoint on M3 Max (FP16).

Use cases

  • Routing customer requests to the correct department based on a description of options.
  • Determining whether a customer asks for a refund (yes/no).
  • Scoring responses against ordered rubric levels.
  • Any constrained decision task that benefits from calibrated probabilities without text generation.

Getting started

Requires Apple Silicon, macOS 14+ and Python 3.11+.

pip install laya-mlx

See the repository README for a full usage example.

License and pricing

Apache-2.0 license, open source.

Based on the project's public README and repository metadata.

Add this badge to your README

Featured on YunoDB Badge styles & help

Maintain this project? Show that it is featured on YunoDB.

Markdown[![Featured on YunoDB](https://yunodb.com/badge/mizorewwwlaya-mlx.svg)](https://yunodb.com/tools/mizorewwwlaya-mlx?utm_source=github&utm_medium=badge)
HTML<a href="https://yunodb.com/tools/mizorewwwlaya-mlx?utm_source=github&utm_medium=badge"><img src="https://yunodb.com/badge/mizorewwwlaya-mlx.svg" alt="Featured on YunoDB" height="20"></a>