Fireworks Pushes AI Inference to 1,500 Tokens per Second and Halves the Price of Work That Can Wait, Betting Agents Need Both Speed and Thrift

AI agents have a speed problem and a cost problem, and they usually show up together. An agent that books a meeting, answers a support ticket or fixes a bug rarely makes one model call. It makes dozens, and every call adds waiting time for the person on the other end and another line on the cloud bill. At its first annual Forge conference on November 3, 2026, inference provider Fireworks AI went after both at once, with an ultra-fast serving tier for work that can’t wait and a half-price tier for work that can.

The launches are a useful snapshot of where machine learning infrastructure is heading: away from one-size-fits-all model access and toward pricing and performance tiers that match how agents actually behave.

What Fireworks announced

According to Fireworks’ own announcement, the headline product is Ultra-fast, now in public preview and running on NVIDIA Blackwell GPUs. The company says it reaches speeds of up to 1,500 tokens per second, as demonstrated on the open-weight DeepSeek V4.1 Flash model. In a benchmarked 30-turn agent task, Fireworks says completion time fell from about one minute to eight seconds.

At the other end of the spectrum sits Background Mode, a runtime option priced at half the normal rate for eligible work that doesn’t need an immediate answer. Fireworks gives examples such as generating data for model evaluations and running overnight research jobs. Alongside it, the company added multi-day prompt caching, which cuts the cost of agents that pause and later resume with the same context, plus higher serverless rate limits and a feature called Elastic Burst, both now generally available. Elastic Burst lets customers go past their normal usage ceiling when spare capacity exists, with no code or pricing changes.

The rest of the slate, as summarized by The Agentic Weekly and Unite.AI, includes:

• Nexus Desktop, which joins the existing Nexus command-line tool and routes between open and closed models inside the Codex app, Claude Desktop and OpenCode. Pro plans start at $8 per month and Teams plans at $10 per seat per month, with Enterprise adding single sign-on, governance and shared budgets.
• Specialized Model Inference (public preview), which serves fine-tuned LoRA adapters through the serverless API with per-token pricing and no charge while idle, starting with GLM 5.3 Flash.
• No Eval (private preview), which turns production traces into task-specific evaluations.
• Fireworks Sovereign (private preview), which runs the company’s inference engine inside a customer’s own cloud or data center.
• Generally available model governance, spending controls and exportable audit logs, with regional data controls and customer-managed encryption in preview.

Why speed matters more for agents than for chatbots

For a simple chatbot, a model that answers in two seconds instead of one is a minor annoyance. For an agent, delays compound. If a task takes 30 sequential model calls, every bit of latency is multiplied by 30. That is why Fireworks framed its benchmark around a 30-turn task rather than a single prompt: the difference between a minute and eight seconds is the difference between a customer waiting on hold and a customer getting an answer while they are still talking.

Business readers should treat those numbers carefully. Both the 1,500 tokens-per-second figure and the one-minute-to-eight-seconds result are Fireworks’ own benchmarks, and The Agentic Weekly noted that the announcement doesn’t publish per-token prices for Ultra-fast or list which models Background Mode covers. Independent testing will tell whether the gains hold on other models and real workloads.

The quiet story is the cheap tier

Speed grabs the headlines, but the half-price option may matter more to most companies’ budgets. A large share of machine learning work has no human waiting on it: evaluating new models, generating synthetic training data, summarizing a day’s documents or running research jobs overnight. Paying peak prices for that work has been one of the hidden taxes of running AI at scale.

Splitting workloads into “now” and “later” lanes is an idea borrowed from older cloud computing, where batch jobs and spot capacity have long been cheaper than on-demand servers. Seeing it arrive in AI inference suggests the market is maturing. Buyers are no longer just asking which model is smartest; they are asking how to pay the right price for each job.

Routing, governance and the open-model bet

Nexus reflects another trend: companies don’t want to bet everything on one model vendor. Routing each request to the model best suited to it, whether open or closed, can lower costs and reduce lock-in. Fireworks says Nexus tracks daily spend by model and compares it against an estimated baseline of sending everything to a single premium model, so finance teams can see what routing saved. That kind of visibility is exactly what CFOs have been asking for as AI bills climb.

The Sovereign preview and the new governance controls target a different worry. Regulated businesses in finance, healthcare and government often can’t send sensitive data to a shared cloud. Running the inference engine inside a customer’s own environment, with audit logs and customer-managed encryption, is aimed squarely at those buyers.

What this means for business leaders

• Sort your AI work by urgency. Customer-facing tasks such as live chat, voice calls and checkout help need low latency. Reporting, data cleanup and evaluation can usually wait, and that is where discounted tiers pay off.
• Measure the whole task, not one response. If your agents chain many calls, end-to-end completion time is the metric that shapes customer experience.
• Ask for spend visibility up front. Model routing only saves money if you can see where each dollar went.
• Verify vendor benchmarks. Run a pilot on your own workload before assuming the published speedups or savings apply to you.

The bottom line

Fireworks’ Forge launches show the AI infrastructure race shifting from raw model quality to the economics of running agents all day. Faster serving for time-sensitive work, half-price serving for work that can wait, and tools to route and govern spending are what companies need as AI moves from pilots to production. The figures are the vendor’s own for now, but the direction is clear: the businesses that win with AI will be the ones that match each task to the right speed and the right price.

Edited by Coron @QUE.COM
Website: https://QUE.COM Intelligence
Sponsored by: https://MAJ.COM AI Autonomous