spark-local-inference
by NVIDIA
Flare is a local inference server that runs AI language models on NVIDIA SM120/SM121 GPUs, such as those in the DGX Spark. It is built for long conversations, coding agents and repeated tool use, with a Rust runtime serving models that have been prepared beforehand.
New and promisingUnder two months old. Expect rough edges and fast changes.
- Stars
- 696 +69 a day, est.
- Licence
- Apache-2.0
- Runs on
- A gpu machine, with a GPU
- Setup
- Technical
- Language
- Rust
- Age
- 10 days
What you can do with it
- Serve selected AI models locally on a DGX Spark or similar NVIDIA workstation
- Run coding agents against a local model with reusable conversation state
- Prepare a model into a runtime package, then serve it from Rust without Python
Ways a business could use it
- Run a private coding assistant on in-house hardware so code and data stay off third-party services
- Cut per-token API costs by serving a supported model locally for internal tools and agents
Can you use it commercially?
Free to use, change and sell, including in paid products. Keep the copyright notice.
Not legal advice. Read the project's licence file before building a product on it.
Getting it running
The README points to separate guides for model preparation and serving, which involves building a runtime package with PyTorch tooling and then running the Rust server.
Follow the official instructions in the project README.
Things to check
- Research prototype, so expect rough edges and changes
- Only works on NVIDIA SM120/SM121 GPUs
- Supports a limited set of models
- The authors say it is not a replacement for production engines such as vLLM
Who's talking about it
- raiscui (gh-trends): GitHub trends — 2026-10-28 (daily release) Featured in a roundup
Checked 30 Oct 2026. Summary drafted by AI from the project's README.
