Google is attempting to make a splash with Gemini, its flagship suite of generative AI models, apps and services.
So what are twins? How can you utilize it? And how does it compare to the competition?
To make it easier to not sleep up to now with the newest Gemini developments, we've put together this handy guide. We'll keep it updated as latest Gemini models, features, and news about Google's plans for Gemini are released.
What are twins?
Gemini belongs to Google promised for a very long time, next-generation GenAI model family developed by Google's AI research labs DeepMind and Google Research. It is available in three flavors:
- Gemini Ultraprobably the most powerful Gemini model.
- Gemini Proa “lightweight” Gemini model.
- Gemini Nanoa smaller “distilled” model that runs on mobile devices just like the Pixel 8 Pro.
All Gemini models have been trained to be “natively multimodal” – in other words, they can work with and use greater than just words. They have been pre-trained and refined using a wide range of audio, image and video files, a considerable amount of code bases and texts in numerous languages.
This sets Gemini aside from models like Google's own LaMDA, which was trained solely on text data. LaMDA cannot understand or generate anything apart from text (e.g. essays, email drafts), but that will not be the case with Gemini models.
What is the difference between the Gemini apps and the Gemini models?
Photo credit: Google
Google has once more proven that it lacks a way of branding and has not made it clear from the beginning that Gemini is separate and distinct from the Gemini web and mobile apps (formerly Bard). The Gemini apps are simply an interface through which specific Gemini models might be accessed – consider it as a client for Google's GenAI.
By the way in which, the Gemini apps and models are also completely independent of Imagen 2, Google's text-to-image model, which is out there in a number of the company's development tools and environments.
What can Gemini do?
Because the Gemini models are multimodal, they’ll theoretically perform a variety of multimodal tasks, from transcribing speech to subtitling images and videos to creating artwork. Some of those features have already reached product stage (more on that later), and Google guarantees all of them – and more – in some unspecified time in the future within the not-too-distant future.
Of course, it's just a little difficult to take the corporate at its word.
Google significantly under-delivered when it originally launched Bard. And recently a video caused a stir that was imagined to display Gemini's capabilities, but turned out to be heavily manipulated and roughly ambitious.
Assuming that Google's claims are roughly true, here's what the several Gemini stages can do once they reach their full potential:
Gemini Ultra
Google says Gemini Ultra's multimodality means it may possibly be used for things like physics homework, solving problems step-by-step on a worksheet, and mentioning possible errors in already accomplished answers.
Gemini Ultra, in line with Google, may also be applied to tasks corresponding to identifying scientific papers relevant to a selected problem – extracting information from those papers and “updating” a graph from them by generating the formulas needed to try this Rebuild chart with newer data.
Gemini Ultra technically supports image creation, as already mentioned. However, this feature has not yet made its way into the product version of the model – perhaps since the mechanism is more complex than the way in which apps like ChatGPT generate images. Instead of passing prompts to a picture generator (like DALL-E 3 within the case of ChatGPT), Gemini outputs images “natively,” without an intermediate step.
Gemini Ultra is out there as an API through Vertex AI, Google's fully managed AI developer platform, and AI Studio, Google's web-based tool for app and platform developers. It also runs the Gemini apps – although not without spending a dime. Accessing Gemini Ultra through what Google calls Gemini Advanced requires a subscription to the Google One AI Premium plan, priced at $20 per thirty days.
The AI Premium plan also connects Gemini to your broader Google Workspace account – think emails in Gmail, documents in Docs, presentations in Sheets, and Google Meet recordings. This is helpful, for instance, for summarizing emails or having Gemini take notes during a video call.
Gemini Pro
According to Google, Gemini Pro represents an improvement over LaMDA in its reasoning, planning and comprehension capabilities.
An independent study from Carnegie Mellon and BerriAI researchers found that the primary version of Gemini Pro could actually handle longer and more complex reasoning chains higher than OpenAI's GPT-3.5. However, the study also found that this version of Gemini Pro, like all major language models, particularly struggled with multi-digit math problems, and users found examples of poor considering and obvious errors.
However, Google promised a treatment – and the primary got here in the shape of Gemini 1.5 Pro.
Gemini 1.5 Pro is designed as a alternative and features improvements over its predecessor in several areas, but most significantly in the quantity of information it may possibly process. Gemini 1.5 Pro can hold roughly 700,000 words or roughly 30,000 lines of code – 35 times the quantity that Gemini 1.0 Pro can handle. And for the reason that model is multimodal, it will not be limited to text. Gemini 1.5 Pro can analyze as much as 11 hours of audio or an hour of video in various languages, albeit slowly (e.g., looking for a scene in an hour-long video takes 30 seconds to a minute of processing time).
Gemini 1.5 Pro was introduced in April as a public preview of Vertex AI.
An additional endpoint, Gemini Pro Vision, can process text images – including photos and videos – and output text modeled on OpenAI's GPT-4 with Vision model.
![Twins](https://techcrunch.com/wp-content/uploads/2024/01/structured_prompt.png)
How to make use of Gemini Pro in Vertex AI. Photo credit: Twins
Within Vertex AI, developers can tailor Gemini Pro to specific contexts and use cases using a fine-tuning or “grounding” process. Gemini Pro may also hook up with external third-party APIs to perform certain actions.
There are workflows in AI Studio for creating structured chat prompts with Gemini Pro. Developers have access to the Gemini Pro and Gemini Pro Vision endpoints and might adjust model temperature to manage the creative range of the output, provide examples of tone and magnificence instructions – and in addition tweak security settings.
Gemini Nano
Gemini Nano is a much smaller version of the Gemini Pro and Ultra models and is efficient enough to run directly on (some) phones quite than sending the duty to a server somewhere. So far it supports a couple of features on the Pixel 8 Pro, Pixel 8 and Samsung Galaxy S24, including “Summarize in Recorder” and “Smart Reply” in Gboard.
The Recorder app, which allows users to record and transcribe audio with the touch of a button, features a summary of your recorded conversations, interviews, presentations and other excerpts provided by Gemini. Users receive these summaries even when there is no such thing as a signal or Wi-Fi connection available – and for privacy reasons, no data leaves their phone.
Gemini Nano can be included in Gboard, Google's keyboard app. A feature called “Smart Reply” will likely be enabled there to allow you to suggest what you wish to say next when you could have a conversation on a messaging app. The feature initially only works with WhatsApp, but will likely be available in additional apps over time, Google says.
And within the Google Messages app on supported devices, Nano enables Magic Compose, which may create messages in styles like “excited,” “formal,” and “lyrical.”
Is Gemini higher than OpenAI's GPT-4?
Google has several times touted Gemini's superiority in benchmarks and claims that Gemini Ultra outperforms current state-of-the-art results on “30 of the 32 widely used academic benchmarks utilized in large language model research and development.” The company says that in some scenarios, Gemini 1.5 Pro performs higher than Gemini Ultra at tasks corresponding to summarizing content, brainstorming, and writing; This will probably change with the discharge of the following Ultra model.
Leaving aside the query of whether benchmarks really indicate a greater model, Google's results look like only marginally higher than OpenAI's corresponding models. And – as mentioned – a number of the first impressions weren’t particularly good, amongst users and Academics points out that the older version of Gemini Pro tends to get basic facts flawed, has difficulty with translations, and makes poor coding suggestions.
How much does twins cost?
Gemini 1.5 Pro is free to make use of within the Gemini apps and currently also in AI Studio and Vertex AI.
However, once Gemini 1.5 Pro leaves preview in Vertex, the model will cost $0.0025 per character, while the output will cost $0.00005 per character. Vertex customers pay per 1,000 characters (roughly 140 to 250 words) and, on models like Gemini Pro Vision, per image ($0.0025).
Let's say a 500 word article comprises 2,000 characters. Summary of this text with Gemini 1.5 Pro would cost $5. Meanwhile, creating an article of comparable length would cost $0.1.
Ultra pricing has yet to be announced.
Where can you are trying Gemini?
Gemini Pro
The easiest place to experience Gemini Pro is the Gemini apps. Pro and Ultra answer queries in numerous languages.
Gemini Pro and Ultra are also accessible in preview in Vertex AI via an API. The API is free to make use of “inside borders” for now and supports certain regions, including Europe, in addition to features corresponding to chat functionality and filtering.
Elsewhere, Gemini Pro and Ultra might be present in AI Studio. The service allows developers to iterate on prompts and Gemini-based chatbots, then obtain API keys to make use of them of their apps – or export the code to a more feature-rich IDE.
Code Assist (formerly Duet AI for developers), Google's suite of AI-powered helper tools for code completion and generation, uses Gemini models. Developers could make “large-scale” changes across codebases, corresponding to updating cross-file dependencies and reviewing large blocks of code.
Google has integrated Gemini models into its development tools for Chrome and the Firebase mobile development platform, in addition to its database creation and management tools. And it has launched latest security products powered by Gemini Gemini in Threat Intelligence, a component of Google's Mandiant cybersecurity platform that may analyze large swaths of probably malicious code and permit users to go looking for current threats or signs of compromise in natural language.