Learn & Discuss about how we can design and scale SLMs
The definition of SLM is Small Language Model which can manage specialized tasks with lower hallucination problem.We do think SLM is a next big thing in AI. Training LLMs is getting more expensive (e.g. the training costs of ChatGPT 4o is over $100M) and also causes serious global warming effect (e.g. training a single AI model can emit as much carbon as five cars in their lifetimes)This research papery by MIT Technology Review will help you capture the  environmental problems of LLMs.MIT Technology ReviewTraining a single AI model can emit as much carbon as five cars in their...Deep learning has a terrible carbon footprint.Another problem of LLM is privacy concerns, especially in healthcare.Based on the above, SLM with decentralized architecture model is on critical demand in the near future. Wifi 7, next generation Wifi standard, which can process 46 Gbit/s at max, will also enable us to run multiple SLMs on edge computing network like our home devices and wearable computers.Here is some amazing work done by a research team at UC Berkley and ICSIhttps://arxiv.org/pdf/2409.00608The purpose for this forum is to build community in which we can proactively learn and discuss together and meet our potential collaborator to build and scale SLMs.If you are SLM enthusiasts, come and join us!
2024-10-07T09:59:06Z
This article mentions about the privacy concerns of LLM.Firstpost – 20 May 24AI companies are finally looking at Small Language Models, and expect to make...Most tech companies have spent a fortune on training their LLMs or Large Language Models. While LLMs take up exorbitant costs and are difficult to monetise, SMLs, or small language models are easy to make, scale well and can be monetised in a better...
2024-10-07T10:04:13Z
I am thinking Query Categorization will be one of the key technologies to scale SLMs because users prefer a single user interface to use multiple SLMs with lower friction costs. To achieve this goal, the system need to categorize user’s query to call the accurate SLMs to complete the task. Any thought?MarkTechPost – 27 Sep 24Microsoft Researchers Introduce Advanced Query Categorization System to...Large language models (LLMs) have revolutionized the field of AI with their ability to generate human-like text and perform complex reasoning. However, despite their capabilities, LLMs need help with tasks requiring domain-specific knowledge,...Est. reading time: 5 minutes
2024-10-07T10:11:39Z
Here is another topic for minimizing the hallucination risk of SLMs.  I am thinking that LLM works a teacher for SLM, this  hybrid approach will be realistic solution to scale SLMs in the initial stage. Any thought?Here is the related research paper.MarkTechPost – 31 Aug 24Microsoft Researchers Combine Small and Large Language Models for Faster,...Large Language Models (LLMs) have demonstrated remarkable capabilities in various natural language processing tasks. However, they face a significant challenge: hallucinations, where the models generate responses that are not grounded in the source...Est. reading time: 6 minutes