The UAE government is supported Institute of Technology Innovation (TII) has announced the launch of Falcon 3, a family of open source small language models (SLMs) designed to operate efficiently on lightweight, single GPU-based infrastructures.
Falcon 3 has 4 model sizes – 1B, 3B, 7B and 10B – with Base and Instruct variants and guarantees to democratize access to advanced AI capabilities for developers, researchers and corporations. According to Hugging Face's leaderboard, the models already outperform or come near popular open-source counterparts of their size class, including Metas Llama and category leader Qwen-2.5.
The development comes at a time when demand for SLMs with fewer parameters and simpler designs than LLMs is growing rapidly as a result of their efficiency, affordability and talent to be deployed on resource-limited devices. They are suitable for a variety of applications in various industries, comparable to customer support, healthcare, mobile apps and IoT, where typical LLMs could also be too computationally intensive to run effectively. Accordingly Evaluates reportsThe marketplace for these models is predicted to grow, with a compound annual growth rate of nearly 18% over the following five years.
What does Falcon 3 bring with it?
Based on 14 trillion tokens – greater than twice as many as its predecessor Falcon 2 – the Falcon 3 family uses a pure decoder architecture with grouped query attention to share parameters and manage memory usage for the key-value cache (KV) in the course of the Minimize inference. This enables faster and more efficient processes when processing a big selection of text-based tasks.
At their core, the models support 4 major languages - English, French, Spanish and Portuguese – and are equipped with a 32K context window so that they can handle long inputs, comparable to strongly worded documents.
“Falcon 3 is flexible, designed for each general and specialized tasks, offering users tremendous flexibility. Its base model is ideal for generative applications, while the Instruct variant is suitable for conversational tasks comparable to customer support or virtual assistants,” notes TII website.
According to the Leaderboard At Hugging Face, all 4 Falcon 3 models perform quite well, however the 10B and 7B versions are the celebrities of the show, achieving state-of-the-art ends in logical reasoning, language comprehension, command sequencing, and code and math tasks.
Among models within the 13B parameter size class, the 10B and 7B versions of the Falcon 3 outperform the competition, including Google's Gemma 2-9B, Meta's Llama 3.1-8B, Mistral-7B and Yi 1.5-9B. In most benchmarks – comparable to MUSR, MATH, GPQA and IFEval – they even outperform Alibaba's industry leader Qwen 2.5-7B, except for MMLU, the test for evaluating how well language models understand and process human language.
Cross-industry use
With the Falcon 3 models now available Hugging faceTII's goal is to serve a big selection of users and enable cost-effective AI deployments without computational bottlenecks. With their ability to handle specific, domain-related tasks with fast processing times, the models can support various applications at the sting and in privacy-sensitive environments, including customer support chatbots, personalized suggestion systems, data analytics, fraud detection, healthcare diagnostics, supply chain optimization and training.
In addition, the institute plans to further expand the Falcon family by introducing multimodal models. These models are expected to launch sometime in January 2025.
Notably, all models were released under the TII Falcon License 2.0, a permissive Apache 2.0-based license with an appropriate use policy that promotes responsible AI development and deployment. To help users start, TII has also launched a Falcon Playground, a testing environment where researchers and developers can check out Falcon 3 models before integrating them into their applications.
