vllm
by vllm-project
vLLM is a Python library and server for running large language models quickly and with less memory. It lets you host open models behind an OpenAI-compatible API so your own apps can use them.
- Stars
- 95.6k
- Licence
- Apache-2.0
- Runs on
- A gpu machine, with a GPU
- Setup
- Technical
- Language
- Python
- Age
- 1355 days
What you can do with it
- Serve open models such as Llama, Qwen or DeepSeek through an OpenAI-compatible API
- Handle many users at once with batching, streaming and quantised models to cut hardware costs
- Run embedding, multi-modal or fine-tuned (LoRA) models on your own NVIDIA, AMD, Intel or CPU hardware
Make money with it
Offer setup and hosting of private, self-hosted AI model endpoints for small businesses that want to avoid sending data to third-party APIs.
An idea, not financial advice. Check the licence before charging for anything built on it.
Ways a business could use it
- Replace per-token fees from hosted AI APIs by running an open model on your own or rented hardware
- Keep customer data in-house by powering a private chatbot or document tool with a self-hosted model
Can you use it commercially?
Free to use, change and sell, including in paid products. Keep the copyright notice.
Not legal advice. Read the project's licence file before building a product on it.
Getting it running
Install with "uv pip install vllm" (or pip), then run "vllm download-kernels"; it needs a suitable GPU and driver setup.
Follow the official instructions in the project README.
Things to check
- Needs a capable GPU, which can be costly to buy or rent
- Complex setup and tuning; aimed at technical users
- Model licences vary and must be checked for commercial use
Checked 26 Oct 2026. Summary drafted by AI from the project's README.
