LimiX
LimiX: Unleashing Structured-Data Modeling Capability for Generalist Intelligence https://arxiv.org/abs/2609.17488
What is LimiX?
LimiX: Unleashing Structured-Data Modeling Capability for Generalist Intelligence https://arxiv.org/abs/2609.17488
Overview of LimiX
What it is
LimiX is a family of large foundation models for structured data (LDMs). LimiX-2 is the new-generation model, developed through model and data scaling guided by scaling laws. It adopts the contextual mechanism network (CMN) paradigm and is pretrained with context-conditional masked modeling (CCMM). The model is designed to learn a context-dependent representation of the joint structure underlying data generation, rather than a target-centric prediction. Pretraining uses synthetic datasets generated by structural causal models (SCMs). LimiX-2 performs classification, regression, and missing-value imputation in a single forward pass without task-specific parameter updates. Available model sizes include LimiX-2 (400M parameters), LimiX-2M (2M), and LimiX-16M (16M).
Key features
- Single pretrained model handles classification, regression, and missing-value imputation in one forward pass.
- Contextual mechanism network (CMN) paradigm enables mechanism-oriented joint modeling.
- Causal awareness: feature attention encodes direct causal relationships, enabling recovery of the causal skeleton.
- Outperforms existing dataset-specific models and tabular foundation models on TabArena, TALENT, and BCCO benchmarks.
- Scaling laws show log-linear improvement in downstream performance with model size.
Use cases
- Structured data classification and regression tasks.
- Missing-value imputation in tabular datasets.
- Causal structure discovery from observational data.
- General-purpose tabular data modeling without task-specific fine-tuning.
Getting started
See the repository README for installation and usage instructions.
License and pricing
LimiX-2 usage is subject to the StableAI LimiX Non-Commercial License. LimiX V1.0 is open-sourced under the Apache 2.0 license. The project is open source.
Based on the project's public README and repository metadata.