- 1 Introduction
- 2 Syncing Lyrics and Audio
- 3 Conclusion
- 4 Credits:
Karaoke games like WeSing or Singstar are quite funny to play: having the ability to see if you sing in tune almost in real time is very stimulating, for both individual practice and parties with your fiends. Though these games often have two major flaws:
- The songs doesn’t always match what you would like to sing apart few tracks.
- The songs doesn’t match your vocal range or vocal skills (they are often quite impressive vocally, but if you are a beginner it can be frustrating to try reproduce), or even language.
This makes karaoke games usually quite expensive for the songs there are left to sing.
That’s why community based Karaoke games are attractive: users can create and share their own lyrics/songs synchronization and share them to everyone else.
In the open source community, there is two major games : UltraStar Deluxe (USDX) and Vocaluxe. They are pretty similar (Vocaluxe is a fork of UltraStar which has evolved a bit differently). Pick one, or the other, or even both: they are generously shared for free by their developers!
You can then download songs from communities website / forum (but you have to consider that some mp3/video are under copyright, and so download may not be legal, especially if you don’t purchase the song before – that said as there is no profit and that song/videos are for most of them already watchable for free on YouTube, I suppose it’s quite ok ethically. The .txt files handling lyrics/sync should be good too. I think people sharing signed artists songs have more legal concerns; regular karaoke games have to give royalties fees to artist/producer on their set list.
Anyway, these legal concerns are out of the scope of this article. Here, we will focus on creating synced lyrics file for our own songs (or any other royalty free songs) with these open source karaoke games for personal usage: no legal issue with that!
Syncing Lyrics and Audio
Using available Software
Creating a custom song for USDX is four different steps:
- Get media (audio of course, but optionally video and cover picture).
- Get lyrics and mark syllables separation manually
- Get notes pitch and timing for each syllables
- Generate a .txt file formatted accordingly (with right infos)
Je col+le+ctionne des ca+nards oh+oh Je col+le+ctionne des ca+nards par mil+lier Je col+le+ctionne des ca+nards oh+oh Je col+le+ctionne des ca+nards+~ vi+vants
There is free softwares to help you during the steps 2-3-4. Here what a lyrics with syllables can look like, with a “+” separator:
The most advanced software is probably YASS, as contrary to UltraStar-Creator, YASS has a timeline visualization and allows you to tweak pitch, and a nice lyrics display. Also, it allows to manage your whole songs library, which can be helpful (UltraStar-Manager can do that too).
Seduced by the customization possibility, I decided to try to map a song from scratch: Ultra Vomit feat Andréas – Je Collectionne des Canards (Vivants). It’s in French, it’s funny, I like the band, I like the music, it is in my vocal range… I had to try! Considering all songs already shared on certain websites like UltraStar Espana, I was curious to see how hard it would be.
Well, it went pretty hard.
UltraStar-Creator is way too basic for mapping something from scratch IMHO, despite some syncing helpers, not having timeline and pitch representation makes me think it is not what it is supposed to do. But I found that YASS had a MIDI importing function, so this would be a good place to start.
So, for finding notes pitch and timing, I used REAPER and a MIDI keyboard. It was the easy part: I made an initial beat mapping (the song was a bit complex: different tempo, different time signatures…), then I recorded at the piano (she is pretty good at finding melodies notes), and adjust the MIDI afterward to sync it perfectly with the vocals. It takes a bit of time but it stays reasonable, as it’s basic REAPER editing. Then, it was time for MIDI to Notes Syncing.
I imported the MIDI export into YASS, and the lyrics ready to be split. But I noticed something: all my tempo mapping was gone, replaced by one BPM grid only. Most notes were still in sync, but not all of them. So I tried to manually adjusted them, but because of different quantization on the grid they endure during the click and drag, their relative position was totally messed-up: that was supposed to be a regular arpeggio became a weird swing pattern.
I don’t blame the software for the bad results, it is probably me who didn’t get it. I wasn’t comfy with the software ergonomic, some basic action like adding a “-” in the lyrics in the middle of a word were way harder than expected. I surely missed some basic infos to start well, but in the end, it took me hours just to put lyrics on the imported notes…
… and if I choose to change something on the REAPER project (because I like REAPER ergonomics), all of this had to be done again. I’m sure it would have been easier with regular 4/4 fixed tempo song because of the way YASS grid works (based on UltraStar .txt recommendations which asks for one fixed BPM tag only) but this wasn’t my case: I have a complex tempo grid, I makes more sense to me to use a complex tempo map.
I had to find a more flexible solution.
Disclaimer: I will not teach you how to use and install REAPER from scratch, just to tell you how I extended REAPER for UltraStar interoperability. If you don’t know REAPER, but find the concept explained below interesting anyway, go take a look at my REAPER Exhaustive Review. You will learn more about the software and where to learn about it.
Advanced macros can be made for REAPER with scripts called ReaScript. If you are not familiar with the concept, you can check this article.
Considering the .txt UltraStar text definition, I knew from the start that making few scripts to export a MIDI track with synced lyrics as UltraStar project .txt file would be possible. Just of matter coding time. In the end, I end up with various UltraStar scripts, for import and export of these .txt files. Here is an example and what data should be interpreted and exported:
#ARTIST:Ultra Vomit #TITLE:Je Collection Des Canards Vivants #GENRE:Unknown #EDITION: #LANGUAGE: #MP3:Ultra Vomit - Je Collection Des Canards Vivants.mp3 #COVER:Ultra Vomit - Je Collection Des Canards Vivants.jpg #VIDEO:Ultra Vomit - Je Collection Des Canards Vivants.mp4 #BPM:167.0 #GAP:11425,67 : 0 1 -12 Dès : 2 1 -12 qu'ya : 4 1 -12 un : 6 2 0 spot : 8 2 0 a : 10 2 -1 la : 12 1 -3 té : 13 4 -5 -lé - 17 : 18 2 -8 qui : 20 2 -5 me : 22 2 -3 par : 24 2 -3 -le : 26 2 -5 d'un * 28 2 -7 can : 30 4 -5 -ard - 33
Type of notes, potion, length, pitch and syllables. A part that it use different unit, it’s pretty straightforward.
But why preparing your karaoke file inside a DAW like REAPER ?
- Ergonomic editing: macros, preference customization, item based workflow…
- Powerful MIDI editing: a lot of feature to make song transcription faster (MIDI quantization, auto-harmonization…)
- Score Editing
- MIDI Input possibilities (via keyboard for eg)
- Interoperability: by having a MIDI track, you will be able to export it and use it in other software if you want.
- Video support
- Cross-Platform, download for free
- Esoteric workflow (like using a voice to MIDI software)
- Virtual instrument support (if you want to hear your notes)
- Multi-bpm timeline grid!
It has an inconvenience though: DAW are complex softwares meant to answer a lot of different things (orchestral composition, studio recording, video post production etc…), so a minimum learning is required (how to create items, a track etc). Though, with flexible DAW like REAPER, all this complexity is hidden if you don’t want it. At first start, REAPER only has few button on a toolbar. But once you get the basic of REAPER… It’s all good!
How to set up the project? I think a lot about this and came with these solutions:
Lyrics in REAPER can be entered in various ways. I chose to use Lyrics MIDI events:
- They are inside the items and so supports item transformation (rate change, splitting, copy pasting).
- MIDI Lyrics are the one displayed in REAPER Notation editor over the score.
- Easy to insert under MIDI Notes.
- There is native Lyrics navigation actions.
- Interoperability: exported MIDI files will contain this text events.
A script to set lyrics from project notes allows to quickly set Lyrics to MIDI Lyrics. It can be either UltraStar-Creator or YASS lyrics syntax:
I chose unnamed markers for new page event cause marker
- Can be displayed in MIDI Editor, without visually interfere with the other text event (like lyrics).
- They are visible from arrange view,
- They can be exported within a MIDI file.
- There is a native action to insert unnamed markers. (Markers: Insert marker at current position)
I simply chose MIDI notes channel to take care of the different UltraStar notes. There is native action to offset selected note channel.
- Channel 1 for Normal notes
- Channel 2 for Golden notes
- Channel 3 for Freestyle
Metadata are written at .txt import and register as project ExtState key/value (some kind of project exta metadata). I create an script to modify exiting keys. Also, at export, if no keys are present, it can try to find the Artist – Title of the song if the .rpp (REAPER project files) is written with a similar pattern.
Video, Audio and Cover tag are automatically added based on project name.
For .txt exportation, BPM is equal to BPM at edit cursor position.
Just select the track that contains your MIDI item(s) and hit export. It will output a .txt file right next to your project file.
Use the Import script on a empty project. It will:
- Set BPM to BPM tag value of the .txt
- Create a new track with a MIDI items containing all MIDI notes and lyrics associated to them
- Add project marker at page split
- Create an audio track with the linked MP3
- Create a video track with the linked video
And of course, all of this is in synced!
REAPER Karaoke Lyrics Viewer (2023-02-09)
I just released a new script, Script: X-Raym_MIDI Lyrics karaoke viewer for Ultrastar_GUI.lua, to have a visual display Karaoke Style of MIDI lyrics event content of your project. It works with this workflow, meaning that pages are defined by markers.
Here is a video demo of the Import / Export workflow.
Here is a recap of the script I made for this workflow:
- Import UltraStar lyrics
- Export first selected track MIDI as UltraStar txt file
- Insert UltraStar lyrics from project to items MIDI notes on first selected track
- Write Ultrastar project metadata
Few word of tricks and caution to be sure it works as expected, without trouble:
- Have a MIDI lyrics text event under each MIDI notes.
- Save your .rpp in the same folder as your audio, video and cover file.
- Have all these files named similarly (they only differ by extension).
- Modify the .rpp and remake export if you need to make a modification (rather than modify the .txt directly).
- Make backups if you export over existing .txt files
It’s possible to import/export an UltraStar .txt with these scripts in REAPER without altering any data: the conversion seems good! This can make REAPER the most advanced Karaoke song editor software 😛 It shows, once again, how having complex API can let users extend softwares, and how it can be valuable in various audio workflows.
Just like the softwares they are meant for, I decided to share these scripts free and open source to anyone. Feel free to mod them! You will find them within my Free Script Pack available with the Reapack REAPER extension.
It is possible that I missed some tags, but you can either request their support or fork the scripts and propose the change.
Have fun syncing… and singing!
Note: This tooks a incredible amount of time to elaborate, I invite to consider making a donation if you like it. Thx!
Screenshot: Ultra Vomit feat Andréas – Je Collectionne des Canards (Vivants)
UltraStar conversion by X-Raym and Marie.
Thanks basisbit from UltraStar Deluxe dev team for its supports about the .txt BPM format!
Cockos Forum Thread: Scripts: Creating Karaoke Songs for UltraStar and Vocaluxe with REAPER