Today, H2O AIthe corporate, which is committed to democratizing AI with a variety of open source and proprietary tools, announced the discharge of Danubea brand new super-tiny Large Language Model (LLM) for mobile devices.
Named after Europe's second-largest river, the open-source model has 1.8 billion parameters and is designed to rival and even outperform similarly sized models on a variety of natural language tasks. This puts it in the identical category because the strong offerings from Microsoft, Stability AI and Eleuther AI.
The timing of the announcement makes perfect sense. Companies that make consumer devices are wanting to explore the potential of offline generative AI, where models run locally on the product, providing users with quick support for all functions and eliminating the necessity to transfer information to the cloud.
“We are excited to release H2O-Danube-1.8B as a transportable LLM on small devices like your smartphone…The proliferation of smaller, lower-cost hardware and more efficient training is now making it possible to bring modest-sized models to a wider audience…We imagine that H2O-Danube-1.8B shall be a game changer for offline mobile applications,” said Sri Ambati, CEO and co-founder of H2O, in a press release.
What could be expected from the Donau-1.8B LLM?
Although “Donau” was just announced, H2O says it may be fine-tuned to handle a variety of natural language applications on small devices, including common sense reasoning, reading comprehension, summarizing, and translation.
To train the mini-model, the corporate collected a trillion tokens from various web sources and used techniques refined from Llama 2 and Mistral models to enhance its generation capabilities.
“We customized the Llama 2 architecture for a complete of roughly 1.8 billion parameters. We used (on the time) the unique Llama 2 tokenizer with a vocabulary size of 32,000 and trained our model to a context length of 16,384. “We incorporated Mistral’s 4,096 sash window,” the corporate noted when describing the model architecture on Hugging Face.
When tested on benchmarksThe model was found to perform equivalent or higher than most models within the 1-2B parameter category.
For example, within the Hellaswag test, which aimed to guage common sense natural language reasoning, it performed with an accuracy of 69.58%, just behind the 1.6 billion parameter Stability AI model Stable LM 2 , which was pre-trained on 2 trillion tokens. It also ranks third within the Arc benchmark for answering advanced questions with an accuracy of 39.42% behind Microsoft Phi 1.5 (1.3 billion parameter model) and Stable LM 2.
H2O has released Donau-1.8B for industrial use under an Apache 2.0 license. Any team that desires to implement the model for a mobile use case can download it from Hugging Face and perform application-specific fine-tuning.
To simplify this process, the corporate also plans to release additional tools soon. A chat-tuned version of the model was also released (H2O-Donau-1.8B-Chat), which could be implemented for conversational applications.
In the long run, the provision of Donau and similar small models is predicted to steer to an increase in offline generative AI applications on phones and laptops, helping with tasks comparable to email summarization, typing and image editing. In fact, Samsung has already gone on this direction with the launch of its S24 smartphone range.
