Skip to content

Super lightweight Ollama alternative to run Llama 3.3, DeepSeek-R1, Phi-4, Gemma 3, Mistral Small 3.1 and other large language models.

License

Notifications You must be signed in to change notification settings

KolosalAI/kolosal-cli

Repository files navigation

Kolosal CLI

A cross-platform command-line interface for discovering, downloading, and running language models from Hugging Face.

Kolosal CLI Demo

Key Features

  • Model Discovery: Fetch and browse Kolosal models directly from Hugging Face.
  • Interactive TUI: Navigate, search, and select models with an easy-to-use terminal interface.
  • Download & Run: Download GGUF models and run them locally with a built-in inference server.
  • Chat Interface: Interact directly with loaded models through a command-line chat.
  • Smart Caching: Reduces API calls and improves performance with intelligent caching.
  • Cross-Platform: Fully supported on Windows, macOS, and Linux.

Getting Started

The easiest way to get started is to download a pre-built binary for your operating system from the Releases page.

Quick Start

Once installed, you can run the application from your terminal:

# Launch the interactive model browser
kolosal-cli

# Stop the background inference server
kolosal-cli --stop-server

Building from Source

If you prefer to build from source, ensure you have CMake (3.14+) and a C++17 compiler installed.

1. Clone the Repository

git clone https://github.com/KolosalAI/kolosal-cli.git
cd kolosal-cli

2. Build the Project

  • Linux / macOS:

    mkdir build && cd build
    cmake .. -DCMAKE_BUILD_TYPE=Release
    make -j$(nproc) # Linux
    # make -j$(sysctl -n hw.ncpu) # macOS
  • Windows:

    mkdir build; cd build
    cmake ..
    cmake --build . --config Release

3. Run the Application

  • Linux / macOS:

    ./kolosal-cli
  • Windows:

    .\Release\kolosal-cli.exe

About

Super lightweight Ollama alternative to run Llama 3.3, DeepSeek-R1, Phi-4, Gemma 3, Mistral Small 3.1 and other large language models.

Topics

Resources

License

Stars

Watchers

Forks

Packages

No packages published