ram-coffers

LLM infrastructure cost reduction via NUMA-aware weight banking: 147 t/s (8.8x stock llama.cpp) on refurbished enterprise POWER8. Self-hosted inference, no clou

What is ram-coffers?

LLM infrastructure cost reduction via NUMA-aware weight banking: 147 t/s (8.8x stock llama.cpp) on refurbished enterprise POWER8. Self-hosted inference, no cloud APIs. Part of the Proof of Physical AI stack.

Overview of ram-coffers

What it is

RAM Coffers is a NUMA-aware conditional memory architecture for LLM inference cost reduction. It routes model knowledge through physical memory banks on self-hosted enterprise hardware such as refurbished IBM POWER8 and Apple Silicon-class machines, enabling local inference without cloud API dependency. The system is part of the RustChain Proof of Physical AI stack, where verified machines earn tokens for performing inference work. Measured at 147 tokens per second on POWER8, achieving 8.8x the performance of stock llama.cpp.

Key features

  • NUMA-Distributed Weight Banking: Model weights partitioned across NUMA nodes by domain (core knowledge, science/tech, creative, history).
  • Resonance Routing: Query embeddings matched to coffer domain signatures via cosine similarity for intelligent weight activation.
  • Non-Bijunctive Pruning: Selective path collapse before full weight fetch, reducing memory bandwidth requirements.
  • DCBT Resident Prefetch: PowerPC data cache block touch hints for L2/L3 residency, achieving 147+ tokens/second on POWER8.
  • vcipher Hardware AES as Attention Collapse Primitive: Repurposes cryptographic AES instructions for non-linear attention operations, providing capabilities beyond vec_perm.
  • Zero-abstraction LLMOps: The routing layer is the hardware topology itself.

Use cases

  • Self-hosted LLM inference on refurbished enterprise hardware (e.g., IBM POWER8) to reduce infrastructure costs.
  • Local model serving without reliance on cloud APIs.
  • DePIN (Decentralized Physical Infrastructure Network) nodes that perform useful AI work while earning rewards via RustChain's Proof of Antiquity consensus.
  • Research into NUMA-aware memory architectures and hardware-software co-design for LLM inference.

Getting started

See the repository README for installation and quick-start instructions. The README references BENCHMARK.md for commands to reproduce the measured performance.

License and pricing

Licensed under AGPL-3.0. Open source, no pricing.

Based on the project's public README and repository metadata.

Add this badge to your README

Featured on YunoDB Badge styles & help

Maintain this project? Show that it is featured on YunoDB.

Markdown[![Featured on YunoDB](https://yunodb.com/badge/scottcjnram-coffers.svg)](https://yunodb.com/tools/scottcjnram-coffers?utm_source=github&utm_medium=badge)
HTML<a href="https://yunodb.com/tools/scottcjnram-coffers?utm_source=github&utm_medium=badge"><img src="https://yunodb.com/badge/scottcjnram-coffers.svg" alt="Featured on YunoDB" height="20"></a>