ElevenLabsthe AI ​​voice startup known for its voice cloning, text-to-speech and speech-to-speech models has just added one other tool to its product portfolio: a AI Voice Isolator.
Available today on the ElevenLabs platform, the offering allows producers to remove unwanted ambient noise and sounds from all their content, from movies to podcasts or YouTube videos.
It comes just days after the corporate launched a reader app and is free to make use of (with some limitations). However, users must also note that the feature is nothing entirely recent in the marketplace. Many other providers of creative solutions, including Adobeoffer tools to enhance voice quality in content. It only stays to be seen how effective Voice Isolator is as compared to them.
How does the AI ​​Voice Isolator work?
When recording content corresponding to movies, podcasts or interviews, producers often encounter the issue of background noise, where unwanted sounds interfere with the content (think random people talking, wind or passing vehicles). These sounds is probably not noticeable when recording, but can affect the standard of the ultimate output – especially by sometimes drowning out the speaker's voice.
To solve this problem, many use ambient noise cancelling microphones that remove the background noise throughout the recording phase. They serve their purpose, but in lots of cases aren’t accessible, especially for early-stage creatives with limited resources. This is where AI-based tools like ElevenLabs' recent Voice Isolator come into play.
At its core, the product works within the post-production stage, where the user just must upload the content they need to boost. Once the file is uploaded, the underlying models process it, detect and take away the unwanted noise, and extract clear dialogues as output.
ElevenLabs says the product extracts speech at a high quality that is comparable to that of studio-recorded content. The company's head of design, Ammaar Reshi, also showed a demo where you may see how the tool removes the noise of a leaf blower to extract crystal-clear speech from the speaker.
We conducted three tests to check the sensible applicability of the speech isolator. In the primary, we spoke three single sentences, each interrupted by different background noises, while the opposite two consisted of three sentences with a combination of various noises occurring irregularly at random places.
In all cases, the tool was capable of process the audio data in a matter of seconds. Most importantly, in just about all cases it removed the sounds – from those related to opening/closing doors and knocking on the table to clapping and moving household objects – and extracted clear speech with none form of distortion. The only sounds it didn’t detect and take away were knocking on the wall and finger snapping.
Sam Sklar, who leads growth at the corporate, also told us that the feature doesn't currently work for vocals, but users can try it out in that use case and can have success on some songs.
Improvements likely on the way in which
While Voice Isolator's ability to remove irregular background noise definitely makes it higher than most other tools that only work with flat sounds, there remains to be room for improvement. Hopefully, like all other tools, ElevenLabs will proceed to enhance its performance.
It is essential to notice that the corporate has not shared much in regards to the underlying models that power the tool or whether the recordings that go into the tool are utilized in any method to train its models. Sklar said he couldn’t share specifics about what goes into model constructing, but stressed that the corporate has a form The privacy policy comprises a link through which users can object to the use of non-public data for training purposes.
The company now offers Voice Isolator only through its platform. It plans to open API access in the approaching weeks, although the precise timeline remains to be unclear. For users who visit the web site or app to check out the tool, ElevenLabs offers free access with certain usage limits.
“The Voice Isolator model costs 1000 characters per minute of audio. We have a free plan on our website that's 10,000 characters/month, so you should utilize it free of charge with 10 minutes of audio per thirty days,” Sklar explained. This signifies that users who wish to remove background noise from larger audio files can have to upgrade to paid plans that start at $5/month and are billed monthly.
