llama.cpp
by ggml-org
llama.cpp is a program that runs large language models (and models that read images) on your own computer or server, without needing many extra components. It is for people who want to run AI chat and text tools locally, with an optional built-in web page and an API that matches OpenAI's.
- Stars
- 132.5k
- Licence
- MIT
- Runs on
- Your computer
- Setup
- Moderate
- Language
- C++
- Age
- 1326 days
What you can do with it
- Download an open model from Hugging Face and chat with it from the command line
- Run a local OpenAI-compatible API server for your own apps and tools
- Use the built-in web interface to chat with text or image-capable models
Make money with it
Set up private, locally hosted AI assistants for small businesses that do not want to send their data to cloud providers.
An idea, not financial advice. Check the licence before charging for anything built on it.
Ways a business could use it
- Run a private in-house AI assistant for drafting and summarising, so customer data stays on your own machines
- Swap paid AI API calls in your own products for a local model to cut running costs
Can you use it commercially?
Free to use, change and sell, including in paid products. Keep the copyright notice.
Not legal advice. Read the project's licence file before building a product on it.
Getting it running
Install with the one-line script from llama.app, use Docker, download pre-built binaries from the releases page, or build from source.
Follow the official instructions in the project README.
Things to check
- Larger models need plenty of memory, and a GPU helps for speed
- Model quality and licences depend on the model you choose, not on llama.cpp itself
Who's talking about it
- Rost (rosgluk): Trending Open Source AI Projects on GitHub, October 2026 Featured in a roundup
- Rost Glukhov: Trending Open Source AI Projects on GitHub, October 2026 Featured in a roundup
Checked 27 Oct 2026. Summary drafted by AI from the project's README.
