Stability AI, the startup behind AI-powered art generator Stable Diffusion, has released an open AI model for generating sounds and songs that it says was trained exclusively on royalty-free recordings.
The generative model, called Stable Audio Open, takes a text description (e.g., “rock beat in a treated studio, session drumming on an acoustic kit”) and outputs a recording as much as 47 seconds long. The model was trained with around 486,000 samples from the free music libraries FreeSound and the Free Music Archive.
According to Stability AI, the model can create drum beats, instrument riffs, ambient sounds and “production elements” for videos, movies and tv shows, in addition to “edit” existing songs or apply the kind of one song (e.g. smooth jazz) to a different.
“A key good thing about this open source version is that users can optimize the model based on their very own custom audio data,” says Stability AI wrote in a post on his company blog“For example, a drummer could fine-tune samples of his own drum recordings to create recent beats.”
Stable Audio Open has its limitations, nevertheless. It can't play full songs, melodies, or vocals — no less than not good ones. Stability AI says it's not optimized for that, and suggests users in search of those features go for the corporate's premium Stable Audio service.
Stable Audio Open also can’t be used commercially; the terms of use prohibit this. And it doesn’t work equally well in all music styles and cultures, or with descriptions in languages apart from English – the bias that Stable AI attributes to the training data is skewed.
“The data source may lack diversity and never all cultures are equally represented within the dataset,” writes Stability AI in a Description of the model. “The samples generated by the model reflect the biases within the training data.”
Stability AI – the fought long to revive its ailing business – recently became the topic of controversy after Vice President of Generative Audio, Ed Newton-Rexresigned because he disagreed with the corporate's stance that training generative AI models on copyrighted works constitutes “fair use.” Stable Audio Open appears to be an try to flip that narrative while not-so-subtly promoting Stability AI's paid products.
With the increasing popularity of music generators like Stability, copyright law and the ways wherein some generator developers may abuse it have gotten the main target of attention.
In May, Sony Music, which represents artists akin to Billy Joel, Doja Cat and Lil Nas X, released sent a letter to 700 AI corporations warning against the “unauthorized use” of their content to coach audio generators. And in March, the primary law geared toward curbing the misuse of AI in music was passed within the US. law entered into force in Tennessee.
