The cost of using big language models on cloud services is sparking interest in smaller models to harness generative AI for business.
The big language models found on AWS, Google Cloud, and Microsoft Azure can handle a wide array of tasks like writing code, predicting protein structures, and answering a multitude of questions. The capabilities are impressive, but handling these massive models with hundreds of billions of parameters can be pricey. Enterprises are wondering if training a smaller language model (SLM) to power a customer service chatbot, for example, is a more cost-effective approach.
Our favorite customer quote is that generalized intelligence might be great, but I don’t need my point-of-sale system to recite French poetry. This comment was made by Devvret Rishi, Chief Product Officer at Predibase, which provides tools for training SLMs.
There’s been a recent rise in enterprise clients exploring SLMs to lower the cost of inference, which is the process of training a GenAI model to produce valuable responses to natural language queries.
Enterprises are now looking into models with 500 million to 20 billion parameters, as they are more price efficient. While SLMs may not be as versatile as their larger counterparts, they show potential as a more affordable alternative when focused on specific tasks.
Small models with concentrated capacity on a specific target task are likely to show improved performance, according to researchers at the University of Edinburgh and the Allen Institute for AI.
SLMs have piqued the interest of mainstream enterprise vendors like Microsoft, which recently introduced Phi-2, a 2.7-billion-parameter SLM that outperforms larger models.
Open source SLMs also offer access to the models’ inner workings, allowing users to access weights that reveal how the models craft their responses. This access is critical for many organizations concerned about discriminatory biases and data governance.
However, there are also security concerns with open-source technology, prompting enterprises to assess options carefully. But as the open-source SLMs mature, we’re likely to see more deployment of GenAI in the coming year.