OpenAI has just demonstrated its recent flagship base model, GPT-4o, with incredible speech recognition and translation capabilities.
As CEO Sam Altman He himself said we knew it OpenAIThe latest “spring update” had nothing to do with it GPT-5 or AI search.
But today at 10 a.m. PT, lots of of 1000’s attended the live-streamed presentation of the brand new model as Chief Technology Officer (CTO) Mira Murati demonstrated its benefits over its predecessor, the GPT-4.
Key announcements from the demo session include:
- GPT-4o (the o stands for omni) intends to switch GPT-4, with OpenAI calls it its recent flagship base model.
- Although broadly similar GPT-4, GPT-4o offers world-class multilingual and audiovisual processing. It can process and translate audio in near real-time.
- OpenAI might GPT-4o freely available, with restrictions. Pro users still get priority and the next message cap.
- OpenAI also releases a desktop version of ChatGPTinitially just for Mac, which can be rolled out immediately.
- Custom GPTs are also made available to free users.
- GPT-4o and its voice features can be slowly rolled out over the approaching weeks and months.
GPT-4oReal-time audio translation
The headline thing that has everyone talking is GPT-4o's impressive near real-time audio processing and translation.
Demonstrations showed that the AI conducted remarkably natural language conversations, offered fast translations, told stories, and gave coding advice.
For example, the model can analyze a picture of a foreign language menu, translate it, and supply cultural insights and suggestions.
OpenAI has just demonstrated its recent GPT-4o model that performs real-time translation 🤯 pic.twitter.com/Cl0gp9v3kN
— Tom Warren (@tomwarren) May 13, 2024
It also can detect emotions through respiration, facial expressions and other visual cues.
Clip of a real-time conversation with GPT4-o running ChatGPT app
NEW: Instead of just converting VOICE to text, GPT-4o also can understand and label other audio functions similar to BREATHING and EMOTION. I'm undecided how that is expressed within the model answer.#openai https://t.co/CpvCkjI0iA pic.twitter.com/24C8rhMFAw
— Andrew Gao (@itsandrewgao) May 13, 2024
GPT-4o's emotional detection capabilities will likely stir controversy once the dust settles.
Emotionally sensing AI could potentially develop nefarious use cases that depend on human impersonation, similar to deep fakes, social engineering, etc.
Another impressive capability the team demonstrates is real-time coding support via voice.
With the GPT-4o/ChatGPT Desktop app you possibly can have a coding buddy (black circle) that talks to you and sees what you see!#openai Announcement thread! https://t.co/CpvCkjI0iA pic.twitter.com/Tfh81mBHCv
— Andrew Gao (@itsandrewgao) May 13, 2024
In one demo, two examples of the model could even be seen singing together.
This demo of two GPT-4o singing together is one in all the craziest things I've ever seen. pic.twitter.com/UXFfbIpuF6
— Matt Shumer (@mattshumer_) May 13, 2024
The general core of OpenAIThe company's goal is to make AI multimodality truly useful in on a regular basis scenarios, difficult tools like Google Translate.
Another essential point is that these demos are lifelike. OpenAI identified, “All videos on this site are in 1x real time,” possibly in reference to Google, which heavily edited them Twins Demo video to spotlight its multimodal capabilities.
With GPT-4o, multimodal AI applications could evolve from a novelty buried deep in AI interfaces to something that average users can interact with each day.
While the demo was impressive, it's still a demo, and the outcomes from average users “within the wild” will really show how proficient these features are.
Aside from real-time language processing and translation that's within the highlight is the undeniable fact that OpenAI Making this recent model free from constraints is big.
WAlthough GPT-4o is *just* a rather higher GPT-4, it’ll equip everyone with a world-class AI model, leveling the playing field for thousands and thousands of individuals all over the world.
You can watch the announcement and demo below:
Everything we learn about GPT-4o
Here's a rundown of every little thing we learn about GPT-4o thus far:
- Multimodal integration: GPT-4o quickly processes and generates text, audio and image data, enabling dynamic interactions across different formats.
- Real-time answers: The model has impressive response times comparable to human conversational response speeds, with the audio response starting in only 232 milliseconds.
- Language and coding features: GPT-4o matches GPT-4 Turbo's performance on English and coding tasks and outperforms it on non-English word processing.
- Audio visual improvements: Compared to previous models, GPT-4o shows a greater understanding of visual and audio tasks, improving its ability to interact with multimedia content.
- Natural interactions: Demonstrations included two GPT-4os playing a song, helping with interview prep, playing games like rock-paper-scissors, and even providing humor with dad jokes.
- Reduced costs for developers: OpenAI has reduced costs for developers using GPT-4o by 50% and doubled processing speed.
- Benchmark performance: GPT-4o benchmarks are characterised by multilingual, audio and visual tasks.
GPT-4o is a big announcement for OpenAIespecially since it’ll be by far probably the most powerful free model.
It could usher in an era of practical, useful AI multimodality that individuals engage with en masse.
This could be an enormous milestone for each the corporate and the generative AI industry as a complete.
