ControllableTalkNet icon indicating copy to clipboard operation
ControllableTalkNet copied to clipboard

MIDI Support

Open RAYTRAC3R opened this issue 3 years ago • 0 comments

Would it be possible to make the script able to directly read MIDI files to get durations and pitch? It'd be very helpful in cases where you don't have clean vocals, but you have a MIDI based on the vocals.

I've been looking into the code, and it looks like it might be possible if you make it able to read the note durations and pitches in a MIDI and convert it to the proper format, but I'm not a skilled enough coder to do it.

Alternatively, I use a concatenative singing synthesizer called UTAU, and the .ust files made with it seem fairly simple in terms of structure, so it might even be possible to import durations and pitch from it instead. UST files even contain lyrics for each note, so a transcript could be extracted.

RAYTRAC3R avatar Feb 05 '22 00:02 RAYTRAC3R