magika
Fast and accurate AI powered file content types detection
What is magika?
Fast and accurate AI powered file content types detection
Overview of magika
What it is
Magika is an AI-powered file type detection tool that uses deep learning to identify file content types. It employs a custom, optimized model of a few MBs, enabling precise file identification within milliseconds on a single CPU. The model was trained and evaluated on a dataset of ~100M samples across 200+ content types, achieving an average ~99% accuracy on the test set. Magika is used at scale by Google to route files in Gmail, Drive, and Safe Browsing to proper security scanners, processing hundreds of billions samples weekly. It is also integrated with VirusTotal and abuse.ch.
Key features
- Available as a command line tool (Rust), Python API, and bindings for Rust, JavaScript/TypeScript (experimental npm package), and GoLang (WIP).
- Trained and evaluated on ~100M files across 200+ content types.
- Achieves ~99% average precision and recall on the test set, outperforming existing approaches especially on textual content types.
- Inference time about 5ms per file after model load (one-off overhead), even on a single CPU.
- Supports scanning thousands of files at once and recursive directory scanning with
-r. - Near-constant inference time independent of file size; uses only a limited subset of file content.
- Per-content-type threshold system to decide whether to trust the prediction or return a generic label (e.g., “Generic text document”).
- Prediction modes:
high-confidence,medium-confidence,best-guessto control error tolerance.
Use cases
- File type identification for security scanning (e.g., routing files in Gmail, Drive, Safe Browsing).
- Integration with VirusTotal and abuse.ch for malware analysis.
- General file type detection in development workflows, such as identifying source code, documents, or binary files.
- Use via web demo (runs locally in browser) for quick testing without installation.
Getting started
Install the command line tool via one of these methods:shellpipx install magika``````shellbrew install magika``````shellcurl -LsSf https://securityresearch.google/magika/install.sh | shOr install the Python package:shellpip install magikaQuick start example:shellmagika -r *For more examples and documentation, see the repository README.
License and pricing
Licensed under Apache 2.0. Free and open source.
Based on the project's public README and repository metadata.