Switchyard
by NVIDIA-NeMo
Switchyard is a Rust library, with Python bindings, that decides which AI model should handle each request from an AI agent. It mixes cheaper models with more capable ones so you can balance cost, speed and task success.
- Stars
- 3.5k
- Licence
- Apache-2.0
- Runs on
- Your computer
- Setup
- Technical
- Language
- Rust
- Age
- 163 days
What you can do with it
- Route each agent request to a cheaper or stronger model depending on the task
- Use it through OpenRouter by setting the model to nvidia/switchyard
- Compare routing set-ups against a single-model baseline using a local demo server
Make money with it
A freelance developer could offer cost-optimisation work for clients' AI agents, using routing to cut their model spend.
An idea, not financial advice. Check the licence before charging for anything built on it.
Ways a business could use it
- Lower the model bills of an AI product by sending easy requests to cheaper models
- Test whether a cheaper model mix completes your agent's tasks as well as a single premium model
Can you use it commercially?
Free to use, change and sell, including in paid products. Keep the copyright notice.
Not legal advice. Read the project's licence file before building a product on it.
Getting it running
You embed the Rust library (libsy) in your own gateway or agent code, or build a plugin from source for LiteLLM or NeMo Relay; Python bindings are also available.
Follow the official instructions in the project README.
Things to check
- You must supply your own model clients and credentials, which usually means paid API keys
- The standalone server is for demos and local evaluation only, not production
- The LiteLLM integration is experimental and checkout-only
- Development API that may change
Who's talking about it
Checked 30 Oct 2026. Summary drafted by AI from the project's README.
