LLamaSharp

A C#/.NET library to run LLM (🦙LLaMA/LLaVA) on your local device efficiently.

What is LLamaSharp?

A C#/.NET library to run LLM (🦙LLaMA/LLaVA) on your local device efficiently.

Overview of LLamaSharp

What it is

LLamaSharp is a cross-platform C#/.NET library for running LLaMA and other large language models on local devices. It is based on llama.cpp and provides efficient inference on both CPU and GPU. The library offers higher-level APIs and RAG (Retrieval-Augmented Generation) support to simplify deployment of LLMs in applications.

Key features

  • Cross-platform support for Windows, Linux, and Mac.
  • Efficient CPU and GPU inference via llama.cpp backends.
  • Multiple backend options: CPU, CUDA 12, CUDA 13, Vulkan, and Metal (Mac).
  • Higher-level APIs and RAG support for easier integration.
  • Integrations with BotSharp, Langchain, and MaIN.NET.
  • Supports GGUF model format, downloadable from Huggingface or converted from other formats.
  • Streaming generation and chat session capabilities.

Use cases

  • Building local AI chat assistants and conversational agents.
  • Deploying LLMs in desktop, web, and mobile applications.
  • Integrating LLMs into game engines like Unity.
  • Creating RAG-based applications for document querying.
  • Developing AI-powered tools for code generation, writing, and analysis.

Getting started

Install the LLamaSharp NuGet package: PM> Install-Package LLamaSharp Then install one or more backend packages (e.g., LLamaSharp.Backend.Cpu for CPU, LLamaSharp.Backend.Cuda12 for CUDA). Prepare a GGUF model file by downloading from Huggingface or converting from PyTorch/Huggingface format. For a complete example of a chat session, see the repository README.

License and pricing

LLamaSharp is open source under the MIT license.

Based on the project's public README and repository metadata.

Add this badge to your README

Featured on YunoDB Badge styles & help

Maintain this project? Show that it is featured on YunoDB.

Markdown[![Featured on YunoDB](https://yunodb.com/badge/scisharpllamasharp.svg)](https://yunodb.com/tools/scisharpllamasharp?utm_source=github&utm_medium=badge)
HTML<a href="https://yunodb.com/tools/scisharpllamasharp?utm_source=github&utm_medium=badge"><img src="https://yunodb.com/badge/scisharpllamasharp.svg" alt="Featured on YunoDB" height="20"></a>