A self-hosted, offline, ChatGPT-like chatbot. Powered by Llama 2. 100% private, with no data leaving your device. New: Code Llama support!

A C#/.NET library to run LLM (🦙LLaMA/LLaVA) on your local device efficiently.

Maid is a cross-platform Flutter app for interfacing with GGUF / llama.cpp models locally, and with Ollama and OpenAI models remotely.

Run AI models locally on your machine with node.js bindings for llama.cpp. Enforce a JSON schema on the model output on the generation level

llama.go is like llama.cpp in pure Golang!

Create characters in Unity with LLMs!

React Native binding of llama.cpp

Build and run AI agents using Docker Compose. A collection of ready-to-use examples for orchestrating open-source LLMs, tools, and agent runtimes.

Self-evaluating interview for AI coders

Run AI ✨ assistant locally! with simple API for Node.js 🚀

LLama.cpp rust bindings

Run larger LLMs with longer contexts on Apple Silicon by using differentiated precision for KV cache quantization. KVSplit enables 8-bit keys & 4-bit values, reducing memory by 59% with <1% quality loss. Includes benchmarking, visualization, and one-command setup. Optimized for M1/M2/M3 Macs with Metal support.

This repo is to showcase how you can run a model locally and offline, free of OpenAI dependencies.

Review/Check GGUF files and estimate the memory usage and maximum tokens per second.

A Pure Rust based LLM (Any LLM based MLLM such as Spark-TTS) Inference Engine, powering by Candle framework.

Local ML voice chat using high-end models.

Run LLMs locally. A clojure wrapper for llama.cpp.

Booster - open accelerator for LLM models. Better inference and debugging for AI hackers

Making offline AI models accessible to all types of edge devices.

LLaMA Server combines the power of LLaMA C++ with the beauty of Chatbot UI.