Qwen3.8-Flash-Next-NVFP4-DGX-Spark
by tonyd2wild
A set of patches, launch scripts and test tools for running NVIDIA's Qwen 3.8 Flash Next language model on a single DGX Spark computer using vLLM. It lets you host a large 125-billion-parameter model locally and serve it through an OpenAI-style web endpoint.
- Stars
- 131
- Licence
- Apache-2.0
- Runs on
- A gpu machine, with a GPU
- Setup
- Technical
- Language
- Python
- Age
- 61 days
What you can do with it
- Serve NVIDIA's Qwen 3.8 Flash Next model from one DGX Spark through an OpenAI-compatible endpoint
- Run the included 40-prompt benchmark to measure speed, first-token delay and load handling on your own machine
- Choose between launch setups, such as the single-machine default or the two-machine SGLang setup, depending on speed and context needs
Make money with it
Someone with a DGX Spark could set up private, on-premises AI assistants for clients who cannot send data to cloud services, and charge for the setup and support.
An idea, not financial advice. Check the licence before charging for anything built on it.
Ways a business could use it
- Run a private, in-house language model for drafting, summarising and coding help, so client data stays on your own hardware and you avoid per-token API fees
- Offer an internal or client-facing AI endpoint for tasks such as JSON extraction or document summaries without paying a hosted provider
Can you use it commercially?
Free to use, change and sell, including in paid products. Keep the copyright notice.
Not legal advice. Read the project's licence file before building a product on it.
Getting it running
You run a bash launch script against a vLLM nightly build with the supplied patch, on a DGX Spark that already holds the 124 GB checkpoint on local NVMe storage.
Follow the official instructions in the project README.
Things to check
- Needs specific, costly hardware (DGX Spark with 128 GB unified memory)
- Depends on a vLLM nightly build pinned to one commit, so it may break with other versions
- Model loading takes around 10 minutes and needs the 124 GB checkpoint on fast local storage
- Very narrow focus on one model and one machine
- The README appears to be cut off, so some details are missing
Who's talking about it
Checked 27 Oct 2026. Summary drafted by AI from the project's README.
