mavlink
Can Android connect over serial?
Asked on Slack, copying here for the record:
Hello!
Just wanted to know whats the best way to create a custom android groundstation app with mavsdk? There doesn't seem to be any framework for connecting to a serial connection on android to a telemetry module. I tried modifying the qgroundcontrol app but it isn't java based so it doesn't work for my purposes. I also tried using dronekit but their code is so old, much of it is unsupported by android studio, same story with trying to fork Android Tower.
Thanks in advance!
MAVSDK-Java will always look like "pure Java" to you, because we hide the C++ stuff :smile:. Once the C++ will be packaged into an Android library, you will just have to do something like:
startMavlink(<connectionUrl>);
Action action = new Action();
action.arm().andThen(action.takeoff()).subscribe();
And the drone will takeoff. With connectionUrl being something like udp://:14540.
Before that (i.e. right now), you will have to run mavsdk_server on your computer (you'll find binaries in the "assets" for different platforms on the release page), and the code will look like:
Action action = new Action(<ipOfYourComputer>);
action.arm().andThen(action.takeoff()).subscribe();
You see here that the first line (that would start mavsdk_server on your phone) is missing, because you run mavsdk_server on your computer instead. And also, you enter an IP in new Action(...), because you need to tell the SDK where to contact mavsdk_server (default is 'localhost').
This means that you can totally develop your app with the current SDK, but your app has to connect to your computer (with <ipOfYourComputer> as described above), and your computer has to connect to the drone (or simulator). When we release mavsdk_server for Android, it will be a matter of adding the startMavlink() line and your app will run completely on your phone :+1:.
Also, note that the Android wrappers are not super mature yet, but they are already quite advanced. So feel free to report bugs and to contribute fixes! Please get started with the java-client example!
@JonasVautherin The question was specifically around how you get the serial port of Android into an application. So what would help this user is if MAVSDK included the logic to open and read the UART: https://developer.android.com/things/sdk/pio/uart
From previous experience the simplest approach might be to read it in Java, push the payload down to C++ (mavsdk core) and then have the same flow as in C++.
This library kind of does this: https://github.com/chzhong/serial-android
Right, I missed that, and I was mainly hoping to actually move the discussion here.
I'm not sure we even need to add a new library. The app would definitely need the permission:
<uses-permission android:name="com.google.android.things.permission.USE_PERIPHERAL_IO" />
And then I wouldn't be surprised if mavsdk_server could "just" use it, given it has the right connection_url. Anyway I've been using that in the past, both from Tower or with proprietary software, and I'm sure there is a way.
It is probably not so difficult to try, but for that one would need to run the example first, as described above.
Would this help? https://github.com/DroidPlanner/usb-serial-for-android/tree/develop/UsbSerialLibrary if i "loaded" the mavsdk_server onboard the android app by just putting the mavsdk C++ library onboard? I could use these series of drivers. Sorry if I am completely misunderstanding the functions I'm still relatively new to this
The serial connection would be done by mavsdk_server. Currently, that is done in serial_connection there. On Linux we would do something like:
./mavsdk_server serial:///dev/ttyUSB0
And my question would be: could we get that kind of address from Android (in Java), and would that be enough to access it (together with the permission mentioned above)?
I will try to have a look at that ASAP, but I would love your input on it!
Hi
First of all I would like to say it is really cool that there is a Java frontend implementation to MAVSDK 👍. You said:
When we release mavsdk_server for Android, it will be a matter of adding the startMavlink() line and your app will run completely on your phone
Is someone working on it right now and will it come out soon or can it still take a long time? Because I have no idea how to program a serial connection on Android.
Hello @dunifi91!
I am currently looking into that :slightly_smiling_face:. If everything goes well, it may work in a few days! Otherwise, at least I'll be able to tell you what I tried that doesn't work :laughing:.
Hi @JonasVautherin Jonas, I am interested in using the MAVSDK-Java for an Android App as well. Is there further progress with the mavsdk_server running on Android, or is there any timeline when it will be available? Thanks in advanced!
Hej! Sorry I've been pretty busy these days. I have a proof of concept running mavsdk_server on Android, I need to wrap it up and push it. And then I'll need help to test the serial stuff. Sorry for the delay, I'll try hard to get that out next week!
@JonasVautherin Thanks for the fast reply. We can do some testing of the mavsdk_server for Android if the server is available.
I pushed my changes, it is working (at least for me) with the android example app. Here is what I changed there, with link to the relevant code:
This is not the final version (I'd like to expose it through RxJava, and we should be able to stop mavsdk_server from the app as well), but that's already something.
Why does it concern this issue?
You can pass the connection url to mavsdk_server. Here I pass udp://:14540, but for trying a serial connection, that may be something like serial:///dev/bus/usb/001/002. And probably the app would need some more permissions.
I won't have time to try the serial connection in the next days, so please let me know here if you make tests! Also if the udp connection works.
Note:
The library only supports arm64-v8a (I published mavsdk-server with libmavsdk_server.so here). I hope it's enough for you to test for the moment!
@JonasVautherin Thanks for pushing your changes. At the moment stuck at mavsdk-server on Linux because we want to use it with ArduPilot.
@dunifi91: Did you have an opportunity to test if the serial connection "just works" when the path is hardcoded, as per my suggestion above ("something like serial:///dev/bus/usb/001/002")?
Would be helpful :slightly_smiling_face:.
@JonasVautherin Unfortunately i have no experience with serial connection on Android yet. But what i read it's not that straightforward as on a PC. So i think simply hardcoding the bus and adding the usb host permission <uses-feature android:name="android.hardware.usb.host" /> will not be enough. But maybe I'm wrong. I haven't tested it yet. A library like https://github.com/mik3y/usb-serial-for-android could be a solution.
Just wondering if anyone has had any success using mavsdk android to communicate over mavlink using a telemetry module connected to the usb port?
int mavsdkServerPort = mavsdkServer.run("serial:///dev/bus/usb/001/002");
drone = new System(BACKEND_IP_ADDRESS, mavsdkServerPort)
I try to use serial on android,I got this permission error.I have granted the usb permission in my app,but I still got this.Does anybody know how to fix this?
2020-07-17 18:04:19.085 26853-26853/com.leopard.droneclient D/MAVSDK-Server: Running mavsdk_server with connection url: serial:///dev/bus/usb/001/002 2020-07-17 18:04:19.085 26853-26853/com.leopard.droneclient I/Mavsdk: MAVSDK version: 0.28.0 2020-07-17 18:04:19.085 26853-26853/com.leopard.droneclient D/Mavsdk: New: System ID: 0 Comp ID: 0 2020-07-17 18:04:19.088 26853-26853/com.leopard.droneclient I/Mavsdk: Server started 2020-07-17 18:04:19.088 26853-26853/com.leopard.droneclient I/Mavsdk: Server set to listen on 0.0.0.0:60096 2020-07-17 18:04:19.088 26853-26853/com.leopard.droneclient I/Mavsdk: Waiting to discover system on serial:///dev/bus/usb/001/002... 2020-07-17 18:04:19.088 26853-26853/com.leopard.droneclient E/Mavsdk: open failed: Permission denied 2020-07-17 18:04:19.088 26853-26853/com.leopard.droneclient E/Mavsdk: Connection failed: Connection error
Maybe it cannot be done this way :confused:. One way that I'm pretty sure would work would be to allow to pass a file descriptor to mavsdk_server. So that the Android code would get the file descriptor to access USB, and pass it directly to mavsdk_server. Though that requires a change in C++ in mavsdk_server.
Would you be willing to look into that?
Maybe it cannot be done this way . One way that I'm pretty sure would work would be to allow to pass a file descriptor to
mavsdk_server. So that the Android code would get the file descriptor to access USB, and pass it directly tomavsdk_server. Though that requires a change in C++ inmavsdk_server.Would you be willing to look into that?
Hi,sir.In MAVSDK--src--core-serial_connection.cpp,I can't find anything about android.What should I do to make it support Android? And I used mavsdk-server for android-0.4.0 ,I could not successfully find my virtual Drone by udp://:14540,though they are in same network. But my Ubuntu server can find it.I don't know what is wrong with my program.
I found QGroundControl can support Serial in android,I have try it .Dose QGroundControl use mavsdk-server?
In MAVSDK--src--core-serial_connection.cpp,I can't find anything about android.
No, it's just creating a serial link, it doesn't have to know about Android. Currently, it opens a file descriptor from the path you give (e.g. if you pass serial:///dev/ttyACM0, it will open a file descriptor for /dev/ttyACM0). Your tests suggest that we can't create a file descriptor on Android like this.
What I believe we should try is the following:
- Instead of passing a path to
serial_connection.cpp, we want to be able to also pass a file descriptor, when we have it. - From Android (in Java), you could get the file descriptor, e.g. I would look for this.
- From Android, we would pass this file descriptor when running
MavsdkServer.
The first step is to add the possibility to pass a file descriptor to serial_connection.cpp (here). The rest will be done in MAVSDK-Java (actually in the mavsdk_server Android library, here).
Does that make sense?
Dose QGroundControl use mavsdk-server?
No, QGC uses its own MAVLink implementation. Though if you look at the sources, I'm pretty sure QGC does something similar (get the file descriptor and uses it to open the serial connection).
I could not successfully find my virtual Drone by udp://:14540
This has nothing to do with serial, so if you have issues using UDP, it should go in another issue :+1:
Wow,thank you very much to reply me so detailedly.I have read your suggestions carefully.Because I am not familiar with Mavsdk-server,and I can read the full QGroundControl code.I think the best way to realize it maybe is to imitate QGC code. All in all,you saved my time.Thank you again~
I think the best way to realize it maybe is to imitate QGC code.
I personally would look at the Android documentation: just try to get a file descriptor to talk to a USB device. Note: this is done in Java, not in C++ :slightly_smiling_face:. But if you find where this is done in QGC (again, I think it has to be done in Java), then I would be happy to look at that code!
I think the best way to realize it maybe is to imitate QGC code.
I personally would look at the Android documentation: just try to get a file descriptor to talk to a USB device. Note: this is done in Java, not in C++ . But if you find where this is done in QGC (again, I think it has to be done in Java), then I would be happy to look at that code!
I know how QGC open android serial,I am very happy to share this with you .QGC used the old version of https://github.com/mik3y/usb-serial-for-android to open the serial ,The code is in QGCActivity located in android.src.org.mavlink.qgroundcontrol in QGC QT project，QGC call open() in QGCActivity,then use QIODevice to send Mavlink data. Now I find dronekit-android has lots of code I can reuse,though it is very old code .
Thanks! So I think that you want to get a FileDescriptor, just like the public static int getDeviceHandle(int idA) function does. That's for the Android part.
Now in MAVSDK-C++, you want to extend serial_connection.h, which currently takes a path, to take an int fileDescriptor (e.g. in a new constructor). Then in serial_connection.cpp, you want to use the fileDescriptor passed in the constructor instead of initializing it from the path (i.e. instead of this, for Linux, I don't know for Windows). You see, _fd is short for "file descriptor", and it is an int. That's exactly the fileDescriptor you got from Java above.
You need a way to start mavsdk_server with a file descriptor, too. I would suggest ./mavsdk_server fd://<the_int>. So that when we parse the "connection_url" here, if it is fd://<int> (instead of e.g. serial:///dev/ttyACM0), then it creates a SerialConnection with your new constructor instead of doing this.
How does it sound? :blush:
Thanks! So I think that you want to get a
FileDescriptor, just like thepublic static int getDeviceHandle(int idA)function does. That's for the Android part.Now in MAVSDK-C++, you want to extend
serial_connection.h, which currently takes apath, to take anint fileDescriptor(e.g. in a new constructor). Then inserial_connection.cpp, you want to use thefileDescriptorpassed in the constructor instead of initializing it from the path (i.e. instead of this, for Linux, I don't know for Windows). You see,_fdis short for "file descriptor", and it is an int. That's exactly thefileDescriptoryou got from Java above.You need a way to start
mavsdk_serverwith a file descriptor, too. I would suggest./mavsdk_server fd://<the_int>. So that when we parse the "connection_url" here, if it isfd://<int>(instead of e.g.serial:///dev/ttyACM0), then it creates aSerialConnectionwith your new constructor instead of doing this.How does it sound?
Sorry for my late reply.I am not very familiar with c++,but I think your solution is possible.I have decided to use dronekit-android code to realize my app.In dronekit-android ,I can use usb serial to communicate with my drone.You can refer to https://github.com/dronekit/dronekit-android if you want to add some usb serial code.
Hello, I too have been trying to run MavsdkServer using a serial address, and I have come across a similar problem as @lbb1993 did:
serial:///dev/bus/usb/001/002... 2020-07-17 18:04:19.088 26853-26853/com.leopard.droneclient E/Mavsdk: open failed: Permission denied 2020-07-17 18:04:19.088 26853-26853/com.leopard.droneclient E/Mavsdk: Connection failed: Connection error
On checking the USB device permission:
Log.d(TAG, "hasPermission: " + usbManager.hasPermission(usbDevice));
I can see that it returns true.
In Android (using Java) it is quite easy to find the FileDescriptor int:
UsbManager usbManager = (UsbManager) mAppContext.getSystemService(Context.USB_SERVICE);
UsbDevice device = usbManager.getDeviceList().get(deviceName);
UsbDeviceConnection connection = usbManager.openDevice(device);
int fileDescriptor = connection.getFileDescriptor());
Is there any interface provided in the Java library that passes the fileDescriptor to the native code? Or has anyone been able to find a work-around to this?
Thank you in advance.
Awesome! You would need to pass that file descriptor to the C++ code, over JNI, and then pass it to mavsdk_server. Because mavsdk_server doesn't have an interface for receiving a file descriptor yet, you would need to add that.
I wrote more details in this comment: https://github.com/mavlink/MAVSDK-Java/issues/23#issuecomment-663623634.
It would be amazing if you could have a look at that!
Sure. I will try to figure it out. I guess I'll have to make the changes that you've suggested in the C++ code and rebuild it using Android NDK tools. Please let me know if this is the part of the C++ code that has been used to generate the libmavsdk_server.so file that is mentioned over here. Or is it this?
MAVSDK-C++ is built from core. mavsdk_server (and hence libmavsdk_server.so) is built on top of that, in backend. So backend uses core.
This said, before going into cross-compilation and all, I would just edit the core to accept a file descriptor in the serial_connection.cpp (and in the add_any_connection string), as mentioned above. And then build up from there.