OpenLightingProject
Is there a recipe for yocto building?
I tried the trivial, but got confused as ola requires a native ola to build itself, so the trivial way is not that trivial. I wonder if anyone has already prepared a yocto recipe?
Not that I'm aware of, some discussion here: http://lists.openembedded.org/pipermail/openembedded-devel/2014-July/096999.html
Or Buildroot: https://github.com/beagleboard/buildroot/blob/master/package/ola/ola.mk
You just need to deal with cross-compilation.
See also: https://gist.github.com/simark/1167861bfabe21dd7658 https://github.com/s-light/acme_ola_crosscompile https://github.com/OpenLightingProject/ola/issues/489 https://opendmx.net/index.php/Cross_Compiling_OLA
Well that's an awesome start, I tried creating the recipe but stopped around the native build. It seems the first link is already beyond what I achieved. I will test it and fix the build, maybe I can make it work.
I have routinely built ola for ARM on the pi, but not using yocto, this helps a lot.
Yocto rocko uses libftdi1 not libftdi0 so configure fails
$PKG_CONFIG --exists --print-errors "libftdi1 >= 0.18"
But modifying line:
https://github.com/OpenLightingProject/ola/blob/00dc86a48ec4c528cec90166435b440f283a9c86/configure.ac#L624
to
[PKG_CHECK_MODULES([libftdi], [libftdi1 >= 0.18], [have_libftdi="yes"],
Makes it finish ok. I think this is related to #1012
Right now with the libftdi1 fix, I can build ola natively on a yocto-generated image (if I provide all necessary *dev pakages). But when cross compiling I get into errors. Yocto first builds ola-native (in my case for x86_64) then it should build for the target (ARM). But ola-native fails, it seems it is looking for an folder (common/rdm/) that is just not there while running protoc.
ERROR: ola-native-0.10.6-r0 do_compile: Function failed: do_compile (log file is located at /home/ariel/projects/poky/rpi/tmp/work/x86_64-linux/ola-native/0.10.6-r0/temp/log.do_compile.30212) ERROR: Logfile of failure stored in: /home/ariel/projects/poky/rpi/tmp/work/x86_64-linux/ola-native/0.10.6-r0/temp/log.do_compile.30212 Log data follows: | DEBUG: Executing shell function do_compile | NOTE: make -j 8 | + cd /home/ariel/projects/poky/rpi/tmp/work/x86_64-linux/ola-native/0.10.6-r0/build | + do_compile | + autotools_do_compile | + oe_runmake | + oe_runmake_call | + bbnote make -j 8 | + [ -p /home/ariel/projects/poky/rpi/tmp/work/x86_64-linux/ola-native/0.10.6-r0/temp/fifo.30212 ] | + printf %b\0 bbnote make -j 8 | + make -j 8 | | + cd /home/ariel/projects/poky/rpi/tmp/work/x86_64-linux/ola-native/0.10.6-r0/build | + do_compile | + autotools_do_compile | + oe_runmake | + oe_runmake_call | + bbnote make -j 8 | + [ -p /home/ariel/projects/poky/rpi/tmp/work/x86_64-linux/ola-native/0.10.6-r0/temp/fifo.30212 ] | + printf %b\0 bbnote make -j 8 | + make -j 8 | /home/ariel/projects/poky/rpi/tmp/work/x86_64-linux/ola-native/0.10.6-r0/recipe-sysroot-native/usr/bin/protoc --cpp_out common/protocol --proto_path ../git/common/protocol ../git/common/protocol/Ola.proto | /home/ariel/projects/poky/rpi/tmp/work/x86_64-linux/ola-native/0.10.6-r0/recipe-sysroot-native/usr/bin/protoc --cpp_out common/protocol --proto_path ../git/common/protocol ../git/common/protocol/Ola.proto | | /home/ariel/projects/poky/rpi/tmp/work/x86_64-linux/ola-native/0.10.6-r0/recipe-sysroot-native/usr/bin/protoc --cpp_out common/protocol --proto_path ../git/common/protocol ../git/common/protocol/Ola.proto | /home/ariel/projects/poky/rpi/tmp/work/x86_64-linux/ola-native/0.10.6-r0/recipe-sysroot-native/usr/bin/protoc --cpp_out common/protocol --proto_path ../git/common/protocol ../git/common/protocol/Ola.proto | /home/ariel/projects/poky/rpi/tmp/work/x86_64-linux/ola-native/0.10.6-r0/recipe-sysroot-native/usr/bin/protoc --plugin=protoc-gen-cppservice=./protoc/ola_protoc_plugin --cppservice_out common/protocol --proto_path ../git/common/protocol ../git/common/protocol/Ola.proto | /home/ariel/projects/poky/rpi/tmp/work/x86_64-linux/ola-native/0.10.6-r0/recipe-sysroot-native/usr/bin/protoc --plugin=protoc-gen-cppservice=./protoc/ola_protoc_plugin --cppservice_out common/protocol --proto_path ../git/common/protocol ../git/common/protocol/Ola.proto | | /home/ariel/projects/poky/rpi/tmp/work/x86_64-linux/ola-native/0.10.6-r0/recipe-sysroot-native/usr/bin/protoc --plugin=protoc-gen-cppservice=./protoc/ola_protoc_plugin --cppservice_out common/protocol --proto_path ../git/common/protocol ../git/common/protocol/Ola.proto | /home/ariel/projects/poky/rpi/tmp/work/x86_64-linux/ola-native/0.10.6-r0/recipe-sysroot-native/usr/bin/protoc --plugin=protoc-gen-cppservice=./protoc/ola_protoc_plugin --cppservice_out common/protocol --proto_path ../git/common/protocol ../git/common/protocol/Ola.proto | /home/ariel/projects/poky/rpi/tmp/work/x86_64-linux/ola-native/0.10.6-r0/recipe-sysroot-native/usr/bin/protoc --cpp_out common/rdm --proto_path ../git/common/rdm ../git/common/rdm/Pids.proto | | /home/ariel/projects/poky/rpi/tmp/work/x86_64-linux/ola-native/0.10.6-r0/recipe-sysroot-native/usr/bin/protoc --cpp_out common/rdm --proto_path ../git/common/rdm ../git/common/rdm/Pids.proto | /home/ariel/projects/poky/rpi/tmp/work/x86_64-linux/ola-native/0.10.6-r0/recipe-sysroot-native/usr/bin/protoc --cpp_out common/rdm --proto_path ../git/common/rdm ../git/common/rdm/Pids.proto | | /home/ariel/projects/poky/rpi/tmp/work/x86_64-linux/ola-native/0.10.6-r0/recipe-sysroot-native/usr/bin/protoc --cpp_out common/rdm --proto_path ../git/common/rdm ../git/common/rdm/Pids.proto | /home/ariel/projects/poky/rpi/tmp/work/x86_64-linux/ola-native/0.10.6-r0/recipe-sysroot-native/usr/bin/protoc --cpp_out common/rpc --proto_path ../git/common/rpc ../git/common/rpc/Rpc.proto | | /home/ariel/projects/poky/rpi/tmp/work/x86_64-linux/ola-native/0.10.6-r0/recipe-sysroot-native/usr/bin/protoc --cpp_out common/rpc --proto_path ../git/common/rpc ../git/common/rpc/Rpc.proto | /home/ariel/projects/poky/rpi/tmp/work/x86_64-linux/ola-native/0.10.6-r0/recipe-sysroot-native/usr/bin/protoc --cpp_out common/rpc --proto_path ../git/common/rpc ../git/common/rpc/Rpc.proto | | /home/ariel/projects/poky/rpi/tmp/work/x86_64-linux/ola-native/0.10.6-r0/recipe-sysroot-native/usr/bin/protoc --cpp_out common/rpc --proto_path ../git/common/rpc ../git/common/rpc/Rpc.proto | [libprotobuf WARNING google/protobuf/compiler/parser.cc:546] No syntax specified for the proto file: Ola.proto. Please use 'syntax = "proto2";' or 'syntax = "proto3";' to specify a syntax version. (Defaulted to proto2 syntax.) | | [libprotobuf WARNING google/protobuf/compiler/parser.cc:546] No syntax specified for the proto file: Ola.proto. Please use 'syntax = "proto2";' or 'syntax = "proto3";' to specify a syntax version. (Defaulted to proto2 syntax.) | [libprotobuf WARNING google/protobuf/compiler/parser.cc:546] No syntax specified for the proto file: Pids.proto. Please use 'syntax = "proto2";' or 'syntax = "proto3";' to specify a syntax version. (Defaulted to proto2 syntax.) | | [libprotobuf WARNING google/protobuf/compiler/parser.cc:546] No syntax specified for the proto file: Pids.proto. Please use 'syntax = "proto2";' or 'syntax = "proto3";' to specify a syntax version. (Defaulted to proto2 syntax.) | [libprotobuf WARNING google/protobuf/compiler/parser.cc:546] No syntax specified for the proto file: Rpc.proto. Please use 'syntax = "proto2";' or 'syntax = "proto3";' to specify a syntax version. (Defaulted to proto2 syntax.) | | [libprotobuf WARNING google/protobuf/compiler/parser.cc:546] No syntax specified for the proto file: Rpc.proto. Please use 'syntax = "proto2";' or 'syntax = "proto3";' to specify a syntax version. (Defaulted to proto2 syntax.) | [libprotobuf WARNING google/protobuf/compiler/parser.cc:546] No syntax specified for the proto file: Ola.proto. Please use 'syntax = "proto2";' or 'syntax = "proto3";' to specify a syntax version. (Defaulted to proto2 syntax.) | common/rpc/: No such file or directory | | [libprotobuf WARNING google/protobuf/compiler/parser.cc:546] No syntax specified for the proto file: Ola.proto. Please use 'syntax = "proto2";' or 'syntax = "proto3";' to specify a syntax version. (Defaulted to proto2 syntax.) | common/rpc/: No such file or directory | Makefile:18000: recipe for target 'common/rpc/Rpc.pb.cc' failed | make: *** [common/rpc/Rpc.pb.cc] Error 1 | make: *** Waiting for unfinished jobs.... | | Makefile:18000: recipe for target 'common/rpc/Rpc.pb.cc' failed | make: *** [common/rpc/Rpc.pb.cc] Error 1 | make: *** Waiting for unfinished jobs.... | [libprotobuf WARNING google/protobuf/compiler/parser.cc:546] No syntax specified for the proto file: Ola.proto. Please use 'syntax = "proto2";' or 'syntax = "proto3";' to specify a syntax version. (Defaulted to proto2 syntax.) | | [libprotobuf WARNING google/protobuf/compiler/parser.cc:546] No syntax specified for the proto file: Ola.proto. Please use 'syntax = "proto2";' or 'syntax = "proto3";' to specify a syntax version. (Defaulted to proto2 syntax.) | common/protocol/ | common/protocol/: | : No such file or directory | No such file or directory | | | [libprotobuf WARNING google/protobuf/compiler/parser.cc:546] No syntax specified for the proto file: Ola.proto. Please use 'syntax = "proto2";' or 'syntax = "proto3";' to specify a syntax version. (Defaulted to proto2 syntax.) | | [libprotobuf WARNING google/protobuf/compiler/parser.cc:546] No syntax specified for the proto file: Ola.proto. Please use 'syntax = "proto2";' or 'syntax = "proto3";' to specify a syntax version. (Defaulted to proto2 syntax.) | Makefile:17994: recipe for target 'common/protocol/OlaService.pb.cpp' failed | make: *** [common/protocol/OlaService.pb.cpp] Error 1 | | Makefile:17994: recipe for target 'common/protocol/OlaService.pb.cpp' failed | make: *** [common/protocol/OlaService.pb.cpp] Error 1 | [libprotobuf WARNING google/protobuf/compiler/parser.cc:546] No syntax specified for the proto file: Pids.proto. Please use 'syntax = "proto2";' or 'syntax = "proto3";' to specify a syntax version. (Defaulted to proto2 syntax.) | common/rdm/: No such file or directory | Makefile:17997: recipe for target 'common/rdm/Pids.pb.cc' failed | make: *** [common/rdm/Pids.pb.cc] Error 1 | | [libprotobuf WARNING google/protobuf/compiler/parser.cc:546] No syntax specified for the proto file: Pids.proto. Please use 'syntax = "proto2";' or 'syntax = "proto3";' to specify a syntax version. (Defaulted to proto2 syntax.) | common/rdm/: No such file or directory | Makefile:17997: recipe for target 'common/rdm/Pids.pb.cc' failed | make: *** [common/rdm/Pids.pb.cc] Error 1 | common/rdm/: No such file or directory | [libprotobuf WARNING google/protobuf/compiler/parser.cc:546] No syntax specified for the proto file: Rpc.proto. Please use 'syntax = "proto2";' or 'syntax = "proto3";' to specify a syntax version. (Defaulted to proto2 syntax.) | | common/rdm/: No such file or directory | [libprotobuf WARNING google/protobuf/compiler/parser.cc:546] No syntax specified for the proto file: Rpc.proto. Please use 'syntax = "proto2";' or 'syntax = "proto3";' to specify a syntax version. (Defaulted to proto2 syntax.) | Makefile:17997: recipe for target 'common/rdm/Pids.pb.h' failed | make: *** [common/rdm/Pids.pb.h] Error 1 | | Makefile:17997: recipe for target 'common/rdm/Pids.pb.h' failed | make: *** [common/rdm/Pids.pb.h] Error 1 | common/rpc/: No such file or directory | | common/rpc/: No such file or directory | common/protocol/: No such file or directory | Makefile:17994: recipe for target 'common/protocol/OlaService.pb.h' failed | make: *** [common/protocol/OlaService.pb.h] Error 1 | | common/protocol/: No such file or directory | Makefile:17994: recipe for target 'common/protocol/OlaService.pb.h' failed | make: *** [common/protocol/OlaService.pb.h] Error 1 | common/protocol/: No such file or directory | | common/protocol/: No such file or directory | Makefile:17991: recipe for target 'common/protocol/Ola.pb.h' failed | make: *** [common/protocol/Ola.pb.h] Error 1 | | Makefile:17991: recipe for target 'common/protocol/Ola.pb.h' failed | make: *** [common/protocol/Ola.pb.h] Error 1 | Makefile:18000: recipe for target 'common/rpc/Rpc.pb.h' failed | make: *** [common/rpc/Rpc.pb.h] Error 1 | | Makefile:18000: recipe for target 'common/rpc/Rpc.pb.h' failed | make: *** [common/rpc/Rpc.pb.h] Error 1 | common/protocol/: No such file or directory | | common/protocol/: No such file or directory | Makefile:17991: recipe for target 'common/protocol/Ola.pb.cc' failed | make: *** [common/protocol/Ola.pb.cc] Error 1 | + die oe_runmake failed | + bbfatal_log oe_runmake failed | + [ -p /home/ariel/projects/poky/rpi/tmp/work/x86_64-linux/ola-native/0.10.6-r0/temp/fifo.30212 ] | | Makefile:17991: recipe for target 'common/protocol/Ola.pb.cc' failed | make: *** [common/protocol/Ola.pb.cc] Error 1 | + die oe_runmake failed | + bbfatal_log oe_runmake failed | + [ -p /home/ariel/projects/poky/rpi/tmp/work/x86_64-linux/ola-native/0.10.6-r0/temp/fifo.30212 ] | ERROR: oe_runmake failed | + printf %b\0 bbfatal_log oe_runmake failed | + exit 1 | + bb_exit_handler | + ret=1 | + echo WARNING: exit code 1 from a shell command. | WARNING: exit code 1 from a shell command. | + exit 1 | | + printf %b\0 bbfatal_log oe_runmake failed | + exit 1 | + bb_exit_handler | + ret=1 | + echo WARNING: exit code 1 from a shell command. | WARNING: exit code 1 from a shell command. | + exit 1 | ERROR: Function failed: do_compile (log file is located at /home/ariel/projects/poky/rpi/tmp/work/x86_64-linux/ola-native/0.10.6-r0/temp/log.do_compile.30212)
- do_rm_work
- mkdir -p /home/ariel/projects/poky/rpi/tmp/work/x86_64-linux/debianutils-native/4.8.1.1-r0/pseudo/ ERROR: Task (virtual:native:/home/ariel/projects/poky/meta-rpi/recipes-misc/ola/ola_git.bb:do_compile) failed with exit code '1'
I can confirm common/rdm is not there: /home/ariel/projects/poky/rpi/tmp/work/x86_64-linux/ola-native/0.10.6-r0/build
I removed the rdm-tests and ran with config flags:
--disable-unittests --disable-rdm-tests --disable-python-libs --disable-java-libs --disable-all-plugins --enable-dummy
Got less errors but still specific to missing folder, attached the log: log.txt
Do you need FTDI? I'd suggest adding --disable-libftdi to configure instead, which should stop the error, given we don't yet support libftdi1 (as you say, due to #1012 ).
common/rpc will likely be missing too etc I suspect.
Can we see the Yocto config you're using, or ideally can you try and run the cross compile outside of Yocto?
It's failing on the core protobuf stuff, rather than anything specific to the RDM tests, so disabling them won't help. Basically, without this core stuff, OLA can't be compiled.
It's trying to write the file into common/rpc:
/home/ariel/projects/poky/rpi/tmp/work/x86_64-linux/ola-native/0.10.6-r0/recipe-sysroot-native/usr/bin/protoc --cpp_out common/rpc --proto_path ../git/common/rpc ../git/common/rpc/TestService.proto
[libprotobuf WARNING google/protobuf/compiler/parser.cc:546] No syntax specified for the proto file: Ola.proto. Please use 'syntax = "proto2";' or 'syntax = "proto3";' to specify a syntax version. (Defaulted to proto2 syntax.)
[libprotobuf WARNING google/protobuf/compiler/parser.cc:546] No syntax specified for the proto file: Ola.proto. Please use 'syntax = "proto2";' or 'syntax = "proto3";' to specify a syntax version. (Defaulted to proto2 syntax.)
[libprotobuf WARNING google/protobuf/compiler/parser.cc:546] No syntax specified for the proto file: Pids.proto. Please use 'syntax = "proto2";' or 'syntax = "proto3";' to specify a syntax version. (Defaulted to proto2 syntax.)
[libprotobuf WARNING google/protobuf/compiler/parser.cc:546] No syntax specified for the proto file: Rpc.proto. Please use 'syntax = "proto2";' or 'syntax = "proto3";' to specify a syntax version. (Defaulted to proto2 syntax.)
common/rpc/: No such file or directory
[libprotobuf WARNING google/protobuf/compiler/parser.cc:546] No syntax specified for the proto file: Rpc.proto. Please use 'syntax = "proto2";' or 'syntax = "proto3";' to specify a syntax version. (Defaulted to proto2 syntax.)
Makefile:18000: recipe for target 'common/rpc/Rpc.pb.cc' failed
make: *** [common/rpc/Rpc.pb.cc] Error 1
So I think it's actually complaining that the folder path doesn't exist (common/rpc).
As you can see, in my latest run i got --disable-all-plugins --enable-dummy
Please see the log.txt attached before. No plugins still give
common/rpc/: No such file or directory
I think the line should be ../git/common/rpc
/home/<snip>/usr/bin/protoc --cpp_out ../git/common/rpc --proto_path ../git/common/rpc ../git/common/rpc/TestService.proto
Right now it is
/home/<snip>/usr/bin/protoc --cpp_out common/rpc --proto_path ../git/common/rpc ../git/common/rpc/TestService.proto
../git/common/rpc does exist.
It shouldn't be ../git/, I'm assuming that's where you're source is, but you're trying to build into the build folder?
Everywhere it says --cpp_out common/rpc or equivalent in common/rpc/Makefile.mk and similar, add $(top_builddir) to the start of that path, e.g. --cpp_out $(top_builddir)/common/rpc and see if that improves it.
If it does, I'll open a PR with a more complete set of changes.
Just to test, I generated a symlink to the missing path, and it passes. So it's just a matter to replace that path with the correct one.
Please check this: https://gist.github.com/desert/e10c17c8da12d0ff31f0e1799cc22f06
I guess LINE 15 is the needed path. But how to add it?
No it's not @desert . Please try as I suggested changing the Makefile to add top_builddir. Yes, your build may progress with a symlink, but it's not the correct fix for the issue you're experiencing.
I just put what you suggested, but didn't work:
$(PROTOC) --cpp_out $(top_builddir)/common/rpc --proto_path $(srcdir)/common/rpc $(srcdir)/common/rpc/Rpc.(proto
..two more...
I tried this, and passed
$(PROTOC) --cpp_out $(srcdir)/common/rpc --proto_path $(srcdir)/common/rpc $(srcdir)/common/rpc/Rpc.proto
But got the same error on common/rdm, I detected another --cpp_out at the Makefile.mk there. So I added again
LINE 59: $(PROTOC) --cpp_out $(srcdir)/common/rdm --proto_path $(srcdir)/common/rdm $(srcdir)/common/rdm/Pids.proto
I kept getting more similar errors. This time on common/protocol. I changed them all.
Am I on the right track using $(srcdir)/ ?
After using $(srcdir)/ (can't tell if it's correct) the build advances but gets stopped at a different place.
As you can see here: https://gist.github.com/desert/7e989b68ba10aa00562040aa1202969f
Bison/Flex has similar problems, where can I check them?
Can I see the error with top_builddir please? I'm not sure if srcdir is the right choice either, it really depends on what it's trying to do next. Essentially we're generating some intermediate code, I think that should probably go into builddir, as I believe srcdir could be read-only in some modes, but perhaps we then need to tweak the Makefile to source the built stuff from builddir.
It may also depend how you're initialising the build and what options you're passing in.
even this should be fixed:
sh ../git/plugins/convert_README_to_header.sh ../git/plugins/dummy ./plugins/dummy/DummyPluginDescription.h
to be at ../git/plugins/dummy/DummyPluginDescription.h
Can I see the error with top_builddir please?
Seems $(top_builddir) is replaced with a dot (.) so I get https://gist.github.com/desert/8f69602a9f80d49c3fb2fcd2f85e7178
No, that definitely shouldn't, and is that makes me think your issue is actually with your make command, that's already using top_builddir and working for our distcheck builds, so I think it's how you're calling stuff.
Okay, this says top_builddir == builddir == '.', so the issue is actually that you need to run make from within the directory you want the files to appear in, which doesn't want/need to be ../git/
See e.g. https://travis-ci.org/OpenLightingProject/ola/jobs/350703970#L1728 .
I see the command:
cd /home/ariel/projects/poky/rpi/tmp/work/x86_64-linux/ola-native/0.10.6-r0/build
But there is no common/blah path, who should create them? Why are they not being created?
See also https://www.gnu.org/software/automake/manual/html_node/Checking-the-Distribution.html and https://www.gnu.org/software/automake/manual/html_node/VPATH-Builds.html#VPATH-Builds for more explanation.
The build tree is rooted in the directory in which configure was run, and is populated with all object files, programs, libraries, and other derived files built from the sources (and hence not distributed). The build tree usually has the same subdirectory layout as the source tree; its subdirectories are created automatically by the build system.
I guess an in-source build would make life easier right? At least for yocto building.
It might help, but I don't think you can do it entirely in-source @desert .
I'd still like to see your current Yocto setup/config, as I think it might help.
If we look at this list: https://gist.github.com/simark/1167861bfabe21dd7658#file-1-steps-L39-L43
Then you probably want the host OLA protoc in a different folder to the target build, hence needing to do an out of source build anyway.
It works fine for us with distcheck, so I think it's just a case of running configure in your /build dir.
Have you tried just manually following the steps in simark's gist on your machine to see if that works as expected, before trying to automate it with yocto?
following the steps in simark's gist on your machine to see if that works as expected, before trying to automate it with yocto?
Yes, I've been cross compiling and also doing native x86 and ARM building, it all works right.
It might help, but I don't think you can do it entirely in-source @desert .
Turns out it works with in-source building. Right now I cross compiled ola for several ARM chips using yocto, just by inheriting autotools-brokensep this means the bug is that ola out-of-tree build is somehow broken. I'm no expert in autotools so can't say much.
Excellent @desert . Glad you've got a bit of a workaround in place.
Please can you share the working Yocto config for others benefit?
I'm still slightly confused you find out-of-tree builds broken; we do a make distcheck every time we build on Travis, which I understood was doing an out-of-tree build, but that may not be the case. Again we'd want the config/recipe to be able to try and reproduce and fix it. Are you still getting the same errors you were before; I have pushed a number of changes to master which ought to improve things?
If I were to add the recipe, should i clone, git commit then PR, or is it better to paste it here?