Sarvam AI Opens Token Factory To Run Global AI Models On Indian Servers

AI & Compute

Sarvam Says Token Factory Serves More Than 450 Billion Tokens a Day

Sarvam AI’s Token Factory serves more than 450 billion tokens daily on Indian GPUs, hosting Sarvam 105B and global open-weight models through one API with local data-residency options.

By
Tom Whitfield
Filed
Channel
AI & Compute
Read
4 min read

By Tom Whitfield

AI unicorn Sarvam AI says it serves more than 450 billion tokens per day on DeepSeek V4.1 Flash through its newly opened Token Factory, running that workload on GPU clusters in India. The service gives developers and enterprises access to Sarvam’s own model and several global open-weight models through one API.

What does Token Factory offer?

Token Factory hosts Sarvam 105B alongside Google Gemma 4 31B, DeepSeek V4.1 Flash, GLM 5.3 and Kimi K3.1 Pro. Developers can switch among the supported models through an OpenAI-compatible API, without building their own computing infrastructure.

Sarvam first outlined its plan to host its own and global open-weight models on Indian infrastructure at the Sarvam Epoch event earlier this year. The company announced general availability for Token Factory earlier this month, turning that plan into a commercial service.

For buyers, the immediate benefit is fewer model-specific integrations and no need to operate GPU clusters for each supported model.

How does Token Factory price and perform?

Sarvam charges for the tokens sent to a model and the tokens generated in response. Its catalogue lists Sarvam 105B at ₹15 per million input tokens and ₹60 per million output tokens. Sarvam charges ₹5 per million for cached input tokens.

Output therefore costs four times the uncached input rate in the published price list. That ratio makes generation volume, rather than prompt volume alone, the larger billing driver for customers using Sarvam 105B.

Sarvam claimed Token Factory “matches or beats leading global inference providers on speed and cost for most of the models it hosts.” In the company’s tests, Sarvam 105B produced its first token in 200 milliseconds and generated 330 tokens per second for a single user.

Sarvam is selling one API connection with claimed speed advantages, rather than making customers assemble their own inference stack.

How does caching support the throughput?

Sarvam said 95.7% of prompt tokens on DeepSeek V4.1 Flash came from cache, helping the service handle more than 450 billion tokens per day. Its three-tier cache reduces computation when parts of a prompt repeat, lowering the work required to serve repeated content.

Cache hits carry the ₹5 per-million-token rate, a third of the standard input price for Sarvam 105B. The lower charge gives developers a financial reason to reuse prompts, while the cache design handles the repeated computation behind the scenes.

The economic takeaway is simple: cached prompt content costs less and requires less computation to serve.

Why does running inference in India matter?

Sarvam’s enterprise pitch combines model choice with data location. Customers can move between Sarvam’s own model and several global open-weight models through one API while keeping prompt processing in India, the company said.

Sarvam said it “runs those models on its own GPUs,” so requests remain on its infrastructure instead of being routed to model developers’ overseas APIs.

For banks and other financial institutions that need to process certain workloads in India, Sarvam said local hosting could help address data-residency requirements. Token Factory also offers dedicated GPU capacity, and Sarvam deploys its inference software on premises for critical institutions.

The commercial takeaway is simple: for enterprise buyers, where data is processed may matter as much as which model answers.

Who else is competing for inference workloads?

India-based competition has been building since Krutrim began hosting DeepSeek models on Indian servers in January 2025. OpenAI introduced India-based API inference for eligible GPT-6 Luna workloads on November 12, while Anthropic offers in-country inference for select Claude models through Amazon Bedrock.

Sarvam also competes with Together AI and Fireworks AI, which manage infrastructure for running open-weight models, and OpenRouter, which gives developers access to hundreds of models through one API.

Amazon Web Services, Microsoft Azure and Google Cloud also provide model hosting and inference services. Sarvam competes with these providers on access, price and the location of the compute serving each request.

The competitive takeaway is that rivals already offer hosted models and one-API access, so Sarvam must stand out on Indian execution, price and service quality.

The next test is whether Sarvam can preserve its published token rates and latency figures as local and global providers add alternatives. Dedicated capacity and on-premises deployments give it ways to protect that position, but both still depend on delivering the claimed speed and cost advantages.

Source: inc42.com

Share this article:

More from Tom Whitfield

Tom Whitfield

Show full bio

Staff writer covering companies, deals and strategy in Semiconductors, chip manufacturing, hardware, AI compute at Chip Dispatch.

1,013 articles

Related articles

« Previous article