Several language models have recently come up in the range of a few hundreds of millions up to 7B parameters. In this talk, I will present a summary of architectures of various small language models (SLMs) like llama 2, Vicuna, Mistral, Mixtral, Falcon, Phi-2 and zcode++. I will also talk about initial thoughts on how to make use of these small language models.
In this video, I will talk about the following: Models based on llama and Falcon (llama, RefinedWeb, Alpaca, Vicuna, Baize, Falcon Instruct, Orca), Models based on llama2 (llama 2, llama 2 chat, orca 2), Models based on Mistral (Mistral, Mixtral, SOLAR), The Phi series (Phi-1, Phi-1.5, Phi-2) and Indian SLMs.
