YuE, a free music generation AI model that can create songs from lyrics, supports Japanese and can also generate songs that mix multiple languages

' YuE ' is a music generation AI jointly developed by the AI research community Multimodal Art Projection and the Hong Kong University of Science and Technology. It can generate music up to five minutes long, including melody and lyrics. The model data is available for free and supports Japanese.

YuE
https://map-yue.github.io/

GitHub - multimodal-art-projection/YuE: YuE: Open Full-song Music Generation Foundation Model, something similar to Suno.ai but open
https://github.com/multimodal-art-projection/YuE?tab=readme-ov-file

YuE is a music generation AI developed based on the Llama2 architecture. The name YuE comes from the Chinese word '乐,' which means 'music' or 'happiness.' The pronunciation of '乐' is similar to the English word 'yeah,' so the research team has released a pop song that repeatedly calls out '乐' as a demo sound source.


YuE can generate music by specifying lyrics and a melody. The song below was generated by specifying a metal-style melody. Unlike music generation AI that automatically generates lyrics by specifying the overall atmosphere of the song, YuE's unique feature is that you can explicitly specify the lyrics for the intro and chorus.


YuE also has variations, including models specialized for English and China. ' YuE-s1-7B-anneal-jp-kr-cot ' and ' YuE-s1-7B-anneal-jp-kr-icl ' are specialized for Japanese and Korean output. The song below is an arrangement of 'OFFICIAL' by the Korean music group EXID, performed using YuE. When played, you can hear the lyrics 'A perfect relationship, as seen from the outside,' 'Pain with sweet thorns still stuck in my throat,' and 'Drowning in the magic of love. Abracadabra... in love with you' sung skillfully. However, the accuracy of Japanese is not as good as that of a native speaker, and is more like a Korean idol practicing and singing Japanese.


The YuE model data is available at the following link:

YuE - a map Collection
https://huggingface.co/collections/map/yue-6797d55e22990ae89b90a3d6