Saturday, Nov 21 | --:--
Back to home

Nemotron Lightning Is a Cheap Worker Node — Switchyard Is the Real Product

NVIDIA shipped Nemotron 3.5 Lightning, a 30B MoE with ~3B active parameters for high-volume agent execution, plus NeMo Switchyard, an open router that steers each step to the cheapest capable model. NVIDIA’s thesis is that most agent tokens are execution, not reasoning.

5 min read Santa Clara, CA View as Markdown
Engraved NVIDIA Nemotron lightning bolt through a multi-model Switchyard router switchboard with MoE expert tiles and agent workflow rails in security-intaglio green-black linework.

Corrected October 14, 2026: Removed undated Artificial Analysis scores from the summary, SEO description and checks section; corrected the LangChain partner figures (74% lower cost on 145 tasks, 7% of calls routed to a frontier model, 6% accuracy tradeoff); turned an internal reference into a link.

If agents spend most tokens on boring steps, the winning stack is frontier planner + fast open worker + router. Switchyard is strategically heavier than another small open model: it puts NVIDIA inside the decision of which model runs.

NVIDIA expanded the Nemotron 3 open family with Nemotron 3.5 Lightning — a 30-billion-parameter mixture-of-experts model with roughly 3 billion active parameters for always-on agent execution — and NeMo Switchyard, an open-source model routing library that steers each request to the right model in a developer-defined pool. NVIDIA claims ~4× throughput versus comparable in-class models at similar intelligence and up to ~30% faster completion of agentic benchmark workloads. Weights land on Hugging Face / ModelScope / OpenRouter and as NIM microservices; Switchyard is on GitHub with gateway partners including OpenRouter, LiteLLM, and Kong.

What shipped#

LinkEmbed

Piece Detail (NVIDIA blog / analyst notes)
Nemotron 3.5 Lightning 30B MoE, ~3B active; hybrid Mamba-Transformer; multi-token prediction; latent MoE; speculative decoding
Role High-volume specialized calls in long-running agents — not frontier planning
Open stack Open weights + post-training datasets + training recipes (Nemotron Coalition)
Hardware targets Jetson, GeForce RTX, DGX Spark, DGX Station; single-GPU laptop/desktop class claims
NeMo Switchyard Open router: custom algorithms/policies across a model pool; mid-task reshuffle
Partner claim LangChain: 74% lower cost on 145 multi-turn Deep Agents tasks by routing only 7% of calls to a frontier model, at a 6% accuracy tradeoff (partner-measured)
Availability HF, ModelScope, OpenRouter, build.nvidia.com NIM; Switchyard on GitHub

Claims vs checks#

Lab / partner: Throughput and agent-workload latency claims are NVIDIA’s; they are vendor benches with disclosed architecture advantages (MoE + MTP + speculative decoding).

Positioning:

  • Lightning is not positioned as a Sol/Opus peer; the product is cheap reliable worker nodes behind a router.

Distinct from the $500B Wall Street financing MOUs (Aug 10) and Firebird factory open (Aug 8). Lightning + Switchyard is open model + agent infrastructure software — NVIDIA selling the orchestration layer, not only GPUs.

Limits#

  • LangChain cost-reduction path is partner-measured, not an independent composite.
  • Whether Switchyard becomes default in LiteLLM/Kong gateways is still open.

Sources#

Prior Coverage

Earlier Times of AI reporting on this thread.

Scroll to continue reading