Source: Cailian
After the chat robot ChatGPT, AI video model Sora, OpenAI is to expand again.
According to reports, OpenAI is actively developing AI music models.OpenAI’s engineers are working with some students at the Yulia Academy of Music to tag music spectrum as training data for music models.
In internal discussions, OpenAI explored specific application scenarios for using text and audio prompt words to generate music. For example, a user can enter a description and ask the AI to add guitar accompaniment to an existing vocal track.
This feature can help users add music to short videos, making the video entirely AI-generated. Imagine that users generate a TikTok-style dance video with Sora, then instantly add dynamic background music through music AI, and then share it to the AI social platform that ChatGPT is developing, which will greatly lower the threshold for content creation.
With over 800 million active users, the music model will help OpenAI build a more comprehensive AI ecosystem that will further enhance user viscosity.
Whether the music model will merge seamlessly with ChatGPT or Sora, or become a standalone app, however, is unknown, a spokesman for OpenAI declined to comment.
Music models are not only suitable for personal entertainment, but can also be combined with commercial scenarios to help OpenAI expand into the advertising field. Advertising companies will be able to use OpenAI’s music models to create lyrics and melodies for ads.
In fact, OpenAI’s attempts in the field of music have long been traceable. In 2019, OpenAI launched the music-generating model MuseNet, which combines up to 10 different instruments to generate many different styles, such as classics, rock, country, and songs of up to 4 minutes, but can not be sung.
But both MuseNet and Jukebox were not integrated into ChatGPT and Sora. Due to the limitations of technology and computing costs, there is still a big gap between the music they produce and human creation.
The Global Music Race
Nowadays, with the advancement of computing power and model architecture, music generation technology has finally ushered in the possibility of practical use, and may become the focus of a new round of AI technology competition after text and video.
In May this year, Google launched the second-generation music production model Lyria. Google emphasized that Lyria can produce music for advertising, which heavily overlap with the potential commercialization direction of the OpenAI music model.
The AI music generation products of startups Suno and Udio have been successfully commercialized.Suno, as a company founded just three years ago, has annual regular revenue of $1.5 billion, almost four times more than a year ago.
The journal noted that China’s AI music model is also rising rapidly.
Last year, the big model team launched the Seed-Music family of music-generating models with flexible controls.
Earlier this year, Alipay Lab opened the music generation model InspireMusic, aiming to create an open-source AIGC toolkit that combines music generation, song generation and audio generation capabilities.
On March 26, Kunlun Wanwei released the world's first music reasoning model, Mureka O1, which surpassed Suno V4 in many performances and reached the top of SOTA.
On June 16, Tencent AI Lab open-source SongGeneration music generated a large model, focusing on solving the three common problems of sound quality, musicality and production speed in music AIGC.
On September 12, MiniMax launched the music generation model Music 1.5 and achieved four new breakthroughs: strong control, full of people, rich composition level and clear song structure.