MIDI Import feature ported to drumstick and new MIDI Export feature

[senseab]

Now I have changed origin portsmf of MIDI Import to drumstick And with some new features.

• Fix issue #1967
• Adding Overture (.ove) and Cakewalk (.wrk) project file support. with drumstick
• New MidiExport plugin

Origin MidiImport plugin changed to SmfImport

[Umcaruje]

@tonychee7000 locale updates are done over transifex now, this should not be in the commit for other languages, just English. CC @liushuyu

[senseab]

Sorry, i would fix it

[Umcaruje]

Also you'll need to add drumstick to packages needed for travis, and mingw packages will need to be made. What are the advantages of drumstick over our current implementation?

[senseab]

Reply @Umcaruje

1. I have not understood travis yet, and I don't know how to make mingw packages. So it will be better if there were someone help.
2. As you see, drumstick can read Overture and Cakewalk project. Then we can support more file types.
3. Drumstick is Qt-based library for MIDI, using Signal/Slot mechanism.
4. It would be easy to procceed with RPN Event like pitch bend range was changed. See #1967 and #3206

If any doubts, ask @liushuyu

[zonkmachine]

MMS requires libdrumstick-file and drumstick-dev >= 1.0.2

Debian Jessie use drumstick 0.5.0

Changes for v1.0.2
...
Compilation minimum requirements for all platforms: CMake 3.0 and Qt 5.1 

We still use Qt 4 on Linux

[senseab]

is there any libdrumstick newer version at Debian ？if not, i will test it

[zonkmachine]

is there any libdrumstick newer version at Debian ？

In experimental only. 1.0.2 https://packages.debian.org/search?suite=experimental&arch=any&searchon=names&keywords=drumstick I don't know if Debian folks have a problem using packages from experimental.

QT 4 is about to get dropped anyway so that is probably not a big thing. Also I don't know if the libstick version in Debian is a problem. I only use Ubuntu based systems so I'm used to using ppa's. It's a thing with Debian to not update their stable releases so I guess they are cool with running an older lmms too.

[senseab]

Got that, I would say that Qt 5 might be more effective. But I still wonder the lowest version for my codes, to get the most widely compatibility

[zonkmachine]

But I still wonder the lowest version for my codes, to get the most widely compatibility

Then it looks like maybe 0.5.0 is the best candidate.

[senseab]

i will test it

[liushuyu]

I would like to say:

1. This PR should be squashed into one commit, for the early commit messages were non-sense.
2. Need more testing, for the changes affected many aspects
3. Test drumstick package under Windows. Seems like drumstick is intended for Linux only, so we may need some workarounds to port or compile it for Windows (via MSYS or Cygwin, or just dig out drumstick-file components), Darwin, BSDs, Haiku, etc.
4. For the sake of new comers, @tonychee7000 had expressed the hardness of finding "clues" of each and every APIs in LMMS, let's just try to leave more comments in our code. Not only for developers. A bunch of translators asked me about the context of messages in LMMS, I feel like we need to do something about this.

[senseab]

Tested on Debian with drumstick 0.5.0, works well.

[senseab]

@liushuyu It seems that I overwrite your commit for mistake. so could you repair it again?

[senseab]

I was trying to solve travis problem and drumstick-file needs build from source. Now I can build libdrumstick-file.dll successfully. But travis seems need some tests. Code will be merged after all is okay. @Umcaruje

[senseab]

In addition, MIDI Export will be availiable with help of drumstick

[senseab]

Okay, here is a script that can build drumstick-file with mingw

drumstick_ver=$1

cd /tmp

wget https://master.dl.sourceforge.net/project/drumstick/${drumstick_ver}/drumstick-${drumstick_ver}.tar.gz
tar xf drumstick-${drumstick_ver}.tar.gz
cd drumstick-${drumstick_ver}

# We just need library
sed -i "/ADD_SUBDIRECTORY(utils)/d" CMakeLists.txt
sed -i "/add_subdirectory(utils)/d" CMakeLists.txt

mkdir -p build && cd build

export MINGW=/opt/mingw64
export MINGW32=/opt/mingw32
export PROCESSOR=x86_64
export PROCESSOR32=i686
export MINGW_TOOL_PREFIX=${MINGW}/bin/${PROCESSOR}-w64-mingw32-
export MINGW_TOOL_PREFIX32=${MINGW32}/bin/${PROCESSOR32}-w64-mingw32-

export CMAKE_OPTS="-DUSE_WERROR=ON
                   -DLIB_SUFFIX=''
                   -DCMAKE_INSTALL_PREFIX=$MINGW
                   -DCMAKE_PREFIX_PATH=$MINGW 
                   -DMINGW_PREFIX=$MINGW 
                   -DMINGW_PREFIX32=$MINGW32
                   -DCMAKE_SYSTEM_PROCESSOR=$PROCESSOR
                   -DCMAKE_SYSTEM_PROCESSOR32=$PRPCESSOR32
                   -DWIN64=TRUE 
                   -DCMAKE_SYSTEM_NAME=Windows 
                   -DCMAKE_SYSTEM_VERSION=1 
                   -DCMAKE_FIND_ROOT_PATH=$MINGW 
                   -DCMAKE_C_COMPILER=${MINGW_TOOL_PREFIX}gcc 
                   -DCMAKE_CXX_COMPILER=${MINGW_TOOL_PREFIX}g++ 
                   -DCMAKE_RC_COMPILER=${MINGW_TOOL_PREFIX}gcc
                   -DSTRIP=${MINGW_TOOL_PREFIX}strip
                   -DWINDRES=${MINGW_TOOL_PREFIX}windres
                   -DPKG_CONFIG_EXECUTABLE=${MINGW_TOOL_PREFIX}pkgconfig
                   -DQT_BINARY_DIR=$MINGW/bin
                   -DQT_QMAKE_EXECUTABLE=$MINGW/bin/qmake"
                     
export PATH=$PATH:$MINGW/bin

cmake ../ $CMAKE_OPTS
make && sudo make install

There is an important thing that if use Qt4, drumstick should use 0.5.0 or use >=1.0.0 if use Qt5.

But I was not sure what can I do for travis. It was f***ing difficult for me.

So anyone help?

PS: OSX build might use this

drumstick_ver=$1

cd /tmp

wget https://master.dl.sourceforge.net/project/drumstick/${drumstick_ver}/drumstick-${drumstick_ver}.tar.gz
tar xf drumstick-${drumstick_ver}.tar.gz
cd drumstick-${drumstick_ver}

# We just need library
sed -i "/ADD_SUBDIRECTORY(utils)/d" CMakeLists.txt
sed -i "/add_subdirectory(utils)/d" CMakeLists.txt

mkdir -p build && cd build

cmake ../ -DCMAKE_BUILD_TYPE=RelWithDebInfo -DUSE_WERROR=OFF -DLIB_SUFFIX=''
make && sudo make install

[tresf]

if use Qt4, drumstick should use 0.5.0 or use >=1.0.0 if use Qt5.

Can you try:

# Drumstick is Qt version dependent
IF(QT_VERSION_MAJOR EQUAL 4)
   PKG_CHECK_MODULES(REQUIRED DRUMSTICK drumstick-file>=0.5)
ELSE()
   PKG_CHECK_MODULES(REQUIRED DRUMSTICK drumstick-file>=1.0)
ENDIF()

If that doesn't work, you can check WANT_QT5 variable.

But I was not sure what can I do for travis. It was f***ing difficult for me.

For travis, we use three sources for precompiled binaries:

1. apt (Linux)
2. brew (MacOS)
3. @tobydox's apt (Mingw)

If the package is not available in all locations, the build should be commented (or naturally skipped) on that platform until the repos have binaries ready.

[senseab]

@tresf

Can you try:

# Drumstick is Qt version dependent
IF(QT_VERSION_MAJOR EQUAL 4)
  PKG_CHECK_MODULES(REQUIRED DRUMSTICK drumstick-file>=0.5)
ELSE()
  PKG_CHECK_MODULES(REQUIRED DRUMSTICK drumstick-file>=1.0)
ENDIF()

If that doesn't work, you can check WANT_QT5 variable.

Thanks for your help, CMake can require different version of drumstick with Qt version.

For travis, we use three sources for precompiled binaries:

1. apt (Linux)
2. brew (MacOS)
3. @tobydox's apt (Mingw)

If the package is not available in all locations, the build should be commented (or naturally skipped) on that platform until the repos have binaries ready.

Actually, I was tested them on travis. drumstick-file can build well, but LMMS source always have some problems

And I have no any Mac

Qt4 with MinGW (32/64)

Linking CXX shared module ../midiexport.dll
/opt/mingw32/lib/gcc/i686-w64-mingw32/4.9.4/../../../../i686-w64-mingw32/bin/ld: cannot find -lQtCore
collect2: error: ld returned 1 exit status
make[2]: *** [plugins/midiexport.dll] Error 1
make[1]: *** [plugins/MidiExport/CMakeFiles/midiexport.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....

Linking CXX shared module ../smfimport.dll
/opt/mingw32/lib/gcc/i686-w64-mingw32/4.9.4/../../../../i686-w64-mingw32/bin/ld: cannot find -lQtCore
collect2: error: ld returned 1 exit status
make[2]: *** [plugins/smfimport.dll] Error 1
make[1]: *** [plugins/SmfImport/CMakeFiles/smfimport.dir/all] Error 2
make: *** [all] Error 2

Qt4 on OSX

/opt/mingw64/lib/gcc/x86_64-w64-mingw32/4.9.4/../../../../x86_64-w64-mingw32/bin/ld: cannot find -lQtCore
collect2: error: ld returned 1 exit status
make[2]: *** [plugins/midiexport.dll] Error 1
make[1]: *** [plugins/MidiExport/CMakeFiles/midiexport.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....

Linking CXX shared module ../smfimport.dll
/opt/mingw64/lib/gcc/x86_64-w64-mingw32/4.9.4/../../../../x86_64-w64-mingw32/bin/ld: cannot find -lQtCore
collect2: error: ld returned 1 exit status
make[2]: *** [plugins/smfimport.dll] Error 1
make[1]: *** [plugins/SmfImport/CMakeFiles/smfimport.dir/all] Error 2
make: *** [all] Error 2

Qt5 with MinGW (32/64)

Linking CXX shared module ../smfimport.dll
CMakeFiles/smfimport.dir/objects.a(commonReader.cpp.obj):commonReader.cpp:(.text+0x628): undefined reference to `SmfMidiChannel::setName(QString)'
CMakeFiles/smfimport.dir/objects.a(commonReader.cpp.obj):commonReader.cpp:(.text+0x757): undefined reference to `SmfMidiChannel::create(TrackContainer*, QString)'
CMakeFiles/smfimport.dir/objects.a(commonReader.cpp.obj):commonReader.cpp:(.text+0xd86): undefined reference to `SmfMidiChannel::create(TrackContainer*, QString)'
CMakeFiles/smfimport.dir/objects.a(commonReader.cpp.obj):commonReader.cpp:(.text+0x1130): undefined reference to `SmfMidiChannel::create(TrackContainer*, QString)'
CMakeFiles/smfimport.dir/objects.a(commonReader.cpp.obj):commonReader.cpp:(.text+0x12c5): undefined reference to `SmfMidiCC::putValue(MidiTime, AutomatableModel*, float)'
CMakeFiles/smfimport.dir/objects.a(commonReader.cpp.obj):commonReader.cpp:(.text+0x1ac6): undefined reference to `SmfMidiCC::create(TrackContainer*, QString)'
CMakeFiles/smfimport.dir/objects.a(commonReader.cpp.obj):commonReader.cpp:(.text+0x1e62): undefined reference to `SmfMidiChannel::create(TrackContainer*, QString)'
CMakeFiles/smfimport.dir/objects.a(commonReader.cpp.obj):commonReader.cpp:(.text+0x2082): undefined reference to `SmfMidiCC::putValue(MidiTime, AutomatableModel*, float)'
CMakeFiles/smfimport.dir/objects.a(commonReader.cpp.obj):commonReader.cpp:(.text+0x285b): undefined reference to `SmfMidiCC::create(TrackContainer*, QString)'
CMakeFiles/smfimport.dir/objects.a(commonReader.cpp.obj):commonReader.cpp:(.text+0x2adb): undefined reference to `SmfMidiCC::SmfMidiCC()'
CMakeFiles/smfimport.dir/objects.a(commonReader.cpp.obj):commonReader.cpp:(.text+0x2b1e): undefined reference to `SmfMidiChannel::SmfMidiChannel()'
CMakeFiles/smfimport.dir/objects.a(commonReader.cpp.obj):commonReader.cpp:(.text+0x32ee): undefined reference to `SmfMidiChannel::create(TrackContainer*, QString)'
CMakeFiles/smfimport.dir/objects.a(commonReader.cpp.obj):commonReader.cpp:(.text+0x3460): undefined reference to `SmfMidiChannel::addNote(Note&)'
collect2: error: ld returned 1 exit status
make[2]: *** [plugins/smfimport.dll] Error 1
make[1]: *** [plugins/SmfImport/CMakeFiles/smfimport.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....

Qt5 on OSX

ld: library not found for -ldrumstick-file
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [plugins/libsmfimport.so] Error 1
make[1]: *** [plugins/SmfImport/CMakeFiles/smfimport.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....

So what can I do next?

For more information, see here

[senseab]

For travis, now Qt5 with MinGW is okay

see here

But don't worry, i will squash that branch after all works well.

[tresf]

For travis, now Qt5 with MinGW is okay

Great. Linux Qt5 won't work unless you shim yet another compile script for 14.04 since it defaults to libdrumstick0, which is Qt4 only. When I had previously stated to disable this for platforms which didn't offer binaries, I spoke prematurely since this is a replacement for MIDI import.

We generally don't script tarball downloads and installs into our build process, but rather work with the upstream mirror process. @tobydox is usually pretty quick to offer up the DLLs but we'll need someone comfortable with issuing a PR to homebrew as well.

I'm still concerned about the lack of libdrumstick1 for Ubuntu 14.04 as it breaks our build system. Any advice from apt packagers is appreciated.

[tresf]

@tonychee7000 Now I see you're working from two branches. Please merge the build helpers from travis-t into this branch until the mirrors can be setup.

[senseab]

@tresf Yep, so I would say all the things I was doing was just a temporarily solution.

Codes in my travis-t branch need to be reviewed again and to be squashed when merging

But you may also see my another branch export_midi_support, lmms can generate a pretty MIDI file.

I would merge it to the current branch we are at now. after codes of midi-export branch is adapted to your standard.

So could we merge that to a new branch?

Great. Linux Qt5 won't work unless you shim yet another compile script for 14.04 since it defaults to libdrumstick0, which is Qt4 only. When I had previously stated to disable this for platforms which didn't offer binaries, I spoke prematurely since this is a replacement for MIDI import.

Just because I am using Archlinux now, for some library may be newer. But you're right, it seems a tough thing.

PS: travis haven't passed all tests yet, and I need some help for that. see here

[tresf]

I can't get past pkgconfig step on my system (14.04, mingw64).

I think anyone helping will need exactly the changes you've made. (and yes, I've run the windows libdrumstick installation script, which copies the 64-bit binaries to /opt/mingw64/... AND deleted my CMakeCache.txt )

CMake Error at /usr/share/cmake-2.8/Modules/FindPkgConfig.cmake:283 (message):
  A required package was not found
Call Stack (most recent call first):
  /usr/share/cmake-2.8/Modules/FindPkgConfig.cmake:337 (_pkg_check_modules_internal)
  plugins/SmfImport/CMakeLists.txt:5 (PKG_CHECK_MODULES)

[senseab]

So what can I do?

[senseab]

I am a newbie on Qt and cross compile and need help for this.

[tresf]

I need some help for that. see here.... I am a newbie on Qt and cross compile and need help for this.

You just need to adjust your Windows compilation script to use separate prefixes for 64 and 32 bit. I'll comment on the travis-t branch. This would be much easier if we were working from a single branch.

[tresf]

You just need to adjust your Windows compilation script to use separate prefixes for 64 and 32 bit

Nevermind, your script above is different from that on travis-t. Are you on discord?

[senseab]

Nevermind, your script above is different from that on travis-t. Are you on discord?

I have just modified them, so you may subject to travis-t

[tresf]

I have just modified them, so you may subject to travis-t

The following variables aren't needed in your 64-bit mingw script: MINGW32, PROCESSOR32, MINGW_TOOL_PREFIX32, -DMINGW_PREFIX32, -DCMAKE_SYSTEM_PROCESSOR32,-DWIN64.

The only differences between 32-bit and 64-bit compilations should be the prefix and arch. I would recommend combining the build scripts into one.

Why are you rewriting the CMakeLists.txt file? And why aren't you doing this for 32-bit?

[tresf]

Also, it appears the -lQCore4 syntax for Qt5 can be Qt5::Core per https://github.com/LMMS/lmms/blob/master/plugins/vst_base/CMakeLists.txt#L9-L13.