While large language models (LLMs) like GPT-4 and Gemini capture global headlines and attract major institutional investment — including from the government, which recently tapped four startups to develop them — a quieter but powerful shift is unfolding in the background. Small language models (SLMs) are rapidly gaining traction in India, driven by their affordability, efficiency, and real-world relevance in resource-constrained environments.
There’s a reason for this. Unlike their larger counterparts, SLMs are compact, energy-efficient, and designed to operate directly on local devices, without needing continuous cloud access. Startup founders say this makes them well-suited for Indian use cases, especially in regions with limited Internet connectivity or variable computing infrastructure.
“SLMs are in demand because they are ideal for building affordable, energy-efficient, and privacy-friendly AI solutions,” said Ankush Sabharwal, founder and CEO of CoRover, a conversational AI platform. “They can run directly on devices without needing cloud servers, protect sensitive data, and lower both cost and energy usage.”
To meet this growing demand, CoRover launched BharatGPT Mini, a compact SLM, in December 2024. In less than five months, most of the platform’s 18,000 users — spanning enterprises, researchers, and developers — have adopted the model. The company now expects a 60–70% increase in project volume over the next year, with a target of crossing one million implementations.
“BharatGPT Mini integrates seamlessly across native apps, voice interfaces, IoT devices, and manufacturing systems,” Sabharwal noted. “It ensures reliable performance even in offline mode and offers a practical, cost-effective alternative to cloud-dependent LLMs.”
LLMs like GPT-4 and Gemini are highly capable but come with significant infrastructure demands. This is precisely where SLMs find their edge. “SLMs are gaining traction because they bring precision, privacy, and performance to environments that LLMs can’t easily serve — especially in a country as diverse and dynamic as India,” said Krishna Rangasayee, CEO and founder of SiMa.ai.
SiMa.ai recently launched Modalix, a machine learning system-on-chip (MLSoC) platform built to support compact, high-performance models like SLMs. Modalix enables AI models like LLaMA, LLaVA, Gemma, and DeepSeek to run directly on devices such as drones, surveillance systems, industrial robots, and medical equipment — no cloud connection required.
Another reason startups are doubling down on SLMs is their strength in task-specific applications. Unlike broad LLMs that aim to be generalists, SLMs excel when narrowly scoped and deeply optimised.
“Small models are very well-suited to domain-specific needs,” said Suraj Amonkar, Chief AI Research & Platforms Officer at Fractal. “We’re also seeing these models become more sophisticated and moving from ‘language models’ to ‘reasoning models’ that can do much deeper thinking on complex queries.”
Fractal has rolled out a range of domain-specific models such as Kalaido.ai for image generation and Vaidya.ai for medical and health queries. It also recently open-sourced Fathom-R1-14b, a math reasoning model tested on this year’s IIT JEE Advanced mathematics exam. “It was able to solve all the questions,” Amonkar said. Within days of release, the model saw thousands of downloads — 50% more than competing models in India during the same period.
India’s linguistic diversity further boosts the relevance of SLMs. Their smaller size and lower training costs make them easier to fine-tune for low-resource Indian languages.
Gnani.ai, one of the four startups selected by the government last week to develop foundational AI models, has built voice-first SLMs trained on billions of Indic language conversations and millions of hours of audio. “Our SLMs have helped a leading bank recover over $1 billion in overdue EMIs, showing real-world impact,” said Ganesh Gopalan, co-founder and CEO of Gnani.ai.
Gnani’s enterprise AI business grew over 130% year-on-year. Its real-time voice agent solution, Assist365, now handles more than 50 million conversations per month. Its automation platform, Automate365, tripled in usage in the last year, while voice biometrics tool Armour365 saw 90% growth in adoption by financial institutions. The company is now preparing to launch Inya.ai, a no-code platform allowing businesses to deploy intelligent voice agents in minutes. “Businesses are no longer experimenting with AI — they’re operationalising it,” Gopalan added.
Similarly, conversational AI unicorn Gupshup has also entered the SLM race, launching a suite of compact models tailored to different cost and performance needs. Integrated into its core platform, these models are being deployed across sectors such as retail, BFSI, logistics, and vernacular content. Gupshup expects SLM adoption to rise 60–80% over the coming year.
With growing concerns around cloud infrastructure costs, data privacy, and the need for locally embedded intelligence, industry experts forecast that SLM adoption by small and mid-sized enterprises will grow at least fivefold by FY26.
While LLMs may continue to capture headlines, it is the quiet utility and adaptability of SLMs that is turning heads on the ground, shaping the future of AI in India.