¶ Stability AI Launches Audio Model
Today on the podcast we're going to be talking about stability AI and a brand new feature that has just rolled out and that is the ability for them to do audio. So this is a new um update that they've rolled out recently and stability is kind of an interesting company you'll probably remember it just for the fact that it was one of like the leaders in the ai revolution they literally invented a stable diffusion and the way that we
use ai to generate images and yet they really got left behind as a company that's had a lot of financial issues um but i think that they're about to make a big turnaround and so because of this i don't think it's a company that you should count out just quite yet the one thing i did want to mention before we get into this if you haven't tried it already my startup aibox.ai is officially out of
It is officially launched and our first product is the AI Box Playground. We have a beta out right now that essentially allows you to access the top 20 AI models all on one platform. You can chat with them all in the same chat. We have audio, image, and text all in the same chat.
for $20 a month, so you don't have to have subscriptions to 20 different platforms. You pay one time for that, and then you get access to all the different platforms. So you can check it out. The link's in the description, AIbox.ai. All right, let's get into what's happening with Stability AI.
¶ Model Features and Limitations
So the new update they have, the thing that's really interesting about it beyond the fact that, you know, they came out with kind of like an audio model. And I should preface this by saying. They have a big announcement about an audio model, but this isn't like a vocal model. This is a music model. So specifically, it does music. There's a bunch of different competitors. There's Suno and Udeo that are doing this.
But most of these ones that are kind of doing this generated music, people criticize them for the copyright. So they're like, look, these guys, they grabbed all of this data from the Internet. They grabbed everyone's music. They trained a model and now it. creates music so people are upset about kind of the copyright in the data set for this stability tried to avoid this essentially and they did a couple two cool things
Number one, it's a really lightweight, small model that actually can run on your phone, meaning like Suno and UDO have apps that can run on your phone. But obviously that's going up to the server, to the cloud and running off of their own.
their own websites and servers and stuff you have to have access to the internet with this application you technically could just do everything on your phone your phone is powerful enough to run this model and it can generate you stuff now i will put a caveat on this by saying This is not as good as Suno or UDO. It's just that's just the nature of the beast. So Stability trained this only on content that they had copyright for, which is fantastic, right? They don't want any sort of IP risk.
uh involved with this when they're releasing it so they said that it's entirely made out of royalty free audio libraries and free the free music archive um and free sounds those are kind of their sources and they're allowed to do this which is technically great except that it's not as good. So that's I think the big thing. It is really small. It's 341 million parameters in size.
and it was specifically optimized to run on arm cpus so arm makes chips these are built on you know the this model was essentially built so that it's able to run on an arm cpu right on a phone these arm cpus are often put into phones so the thing that it's specifically made for doing though is for quick um kind of shorter audio samples and sound effects so you can do drums you can do instruments you can do riffs
And it can make up to 11 seconds of audio. You can do it on a smartphone and it takes about eight seconds to do this. So this is, you know, definitely faster than your average UDO or Suno AI piece.
but and i'm not saying it's bad i actually think it's fairly decent for what it can do but like it doesn't do vocals and so if you're trying to make a fully fledged song or honestly uh a really great song like suno and yudio are gonna do a much better job in my opinion of making music i've tried both of the um i've extensively tried suno and uh it does incredible work makes amazing music
people criticize that it was trained off of the copyrighted data. I'm not too concerned about that. That's not really my problem. Um, you know, I'm sure people get mad at me or criticize me for that, but that's just my opinion is just like,
That's, you know, their copyright issue to deal with the model so much better as a user and a consumer and someone that would like to create things. I'm going to use the best model. So that's kind of what I'm getting out of sooner or UDO. All right. I want to give you a sample though, because I'm actually.
quite impressed by what they have been able to produce completely copyright free there's no issues there um so they have a couple samples of what it's able to actually so um you can actually go online check out soundcloud they got a bunch of different samples and all of their samples are like much shorter but they are you know showing you exactly what it's capable of doing they could do some drums some music
They have a bunch of limitations in addition to the all ones I've mentioned already. One, it can only do English prompts written in English. So if you speak another language, you'd have to translate your prompts into English and Google Translate or something like that.
it can't generate realistic vocals or high quality songs it's kind of low quality and it doesn't do a lot of different musical styles it was really just built on a bunch of kind of western they call them western bias training data so these free music libraries are not very extensive it's just mostly kind of like western music so
It also has a little bit of restrictive usage. It's not the end of the world. You got to make money somewhere. So it's free for researchers and hobbyists and businesses that make less than a million dollars annual revenue.
But if you're making over a million dollars, you have to pay Stability's enterprise license. This isn't the end of the world. And I think this is a pretty standard licensing kind of deal. Although, yeah, it feels like they'd be making something open source. So I guess some people...
¶ Company Turnaround and Future Plans
are upset about that. Now, Stable Diffusion is a company that has had a ton of issues in the past. They raised some new money last year. A bunch of their investors, including Eric Schmidt from Google. The Napster founder, Sean Parker, famously, who, you know, invested in Meta. We're really trying to turn the business around. So Imad Mostak was their co-founder and he was kind of the former CEO. He apparently really mismanaged all of their finances, almost completely destroyed the company.
Tons of staff resigned. There was a partnership they had with Canva that fell through. Investors were super concerned about this. So in the last few months, they actually got a new CEO. and they appointed james cameron to their board of directors which is interesting because typically this has kind of been famous as a image company and with james cameron you can kind of imagine where they're going with this is going to become a video company all
These AI generated images are perfectly poised to create AI generated videos. And they've also released a bunch of new image generation models. So it seems like stability is on track to do some cool things. I think specifically if we're looking at video. doing these sound effects and kind of these like smaller music bits makes a lot of sense. They want this in the background of if you know, they're making music tracks.
to be able to, or sorry, videos, it'd be really cool to have also AI generated music in the background. So this makes a lot of sense with kind of their strategic direction. I'll be super curious to see where they go. This is a very prolific company. It's raised a lot of money. It's done a lot of interesting things, but again,
It has faced a lot of challenges. So I'll keep you up to date on everything happening with stability. Make sure to leave a rating or review wherever you listen to your podcast. And again, if you haven't tried AI box already, there's a link in the description. I would love to have you try it. You can dump a ton of your subscriptions.
20 a month you get access to all the top ai models you can compare results side by side of different models you can chat with all the models in the same chat you don't have to switch or you know not have the ability to keep talking to different models And it's a lot of fun. So check it out, AIbox.ai, and I will catch you next time.