vllm-omni
by vllm-project
vLLM-Omni is a framework for running AI models that handle text, images, audio, video and actions, not just text. It lets you serve these models through an OpenAI-compatible API, with support for spreading the work across several machines.
- Stars
- 7.5k
- Licence
- Apache-2.0
- Runs on
- A gpu machine, with a GPU
- Setup
- Technical
- Language
- Python
- Age
- 417 days
What you can do with it
- Serve image, video or audio generation models behind an OpenAI-compatible API
- Run text-to-speech and realtime voice models with streaming audio input and output
- Spread large multimodal models across several GPUs or machines for faster responses
Make money with it
A freelancer could set up and manage private multimodal AI endpoints for clients who want to keep their data off third-party services.
An idea, not financial advice. Check the licence before charging for anything built on it.
Ways a business could use it
- Host your own image, speech or video generation service instead of paying per-request fees to a hosted AI provider
- Build a voice assistant or text-to-speech feature for customers on your own hardware, keeping data in-house
Can you use it commercially?
Free to use, change and sell, including in paid products. Keep the copyright notice.
Not legal advice. Read the project's licence file before building a product on it.
Getting it running
Install it with pip or use the nightly CUDA Docker images, then load a supported Hugging Face model, as described in the documentation.
Follow the official instructions in the project README.
Things to check
- Needs powerful GPUs, which can be costly to buy or rent
- Aimed at engineers; setup and tuning need technical skill
- Fast-moving project with frequent releases and compatibility changes
Who's talking about it
- findarepo: 30 Best AI Video Generator GitHub Repos (October 2026) — Open-Source Tools Featured in a roundup
Checked 02 Nov 2026. Summary drafted by AI from the project's README.
