How to use ofxLibwebsockets to get video data from a JS/Node server?

[ghost]

Hi, is it possible to use ofxLibwebsockets to get a video stream from a node server? I tried pulling an image but wasn't having luck. Was only receiving 5 bytes back from the node server instead of an image.

[robotconscience]

Hello! Sorry for the delay.

That should be possible, yes. The biggest questions is: are you using a pure node websocket server, or using something like Socket.IO? Socket requires its own library, which goes beyond ofxLibwebsockets, and definitely won't work (check this issue if you'd like to more on that).

If you're just using a raw websocket, you can presumably send anything! Getting it back into openFrameworks may require something like ofxTurboJpeg, as shown in the client blob example.

Can you provide more info on the server you're using? Thanks!

[leefeel]

Hi. I've been exploring the client and server binary examples from ofxLibwebsockets. And I found them very helpfull on how to send image from OF to js, but I haven't found any example where sending and image from javascript through websockets and reading it in OF.

In the example I am developing, I'm trying to send a capture from the webcam, converting it to base 64 with the method toDataURL:

var ws = new WebSocket("ws://192.168.1.130:9093");
ws.binaryType = "arraybuffer";

var ctx = canvas.getContext('2d');
ctx.drawImage(video, 0, 0);

var image = canvas.toDataURL("image/jpeg");
ws.send(image);

On the server side, I'm using the client blob example provided with ofxLibwebsockets. BUt in this case, the image is arriving as a string:

void ofApp::onMessage(ofxLibwebsockets::Event& args) { if (args.isBinary) {

isBinary is false. args.data.size() is 0. The jpeg image, is placed in args.message, so:

buff.clear(); buff.set(args.message); locked = true; needToLoad = true;

But, ofxTuboJpeg probably is not supporting this base64 format and is giving me the next error: Error in tjDecompressHeader2(): Not a JPEG file: starts with 0x64 0x61

So, I'm not sure if this is the good approach, and how to solve it. But I didn't find any working example on the internet on how to send an image from javascript to openframeworks through websockets. I would really appreciate a solution for this. Thanks!

[robotconscience]

Hello! The main issue is that you're converting to base64 (a string). If you want to do that, you'll have to decode the base64 data in OF in the onMessage – as a string.

Or, you can play a bit with the toBlob method of canvas: https://developer.mozilla.org/en-US/docs/Web/API/HTMLCanvasElement/toBlob

I haven't personally used that one, but if you're in Chrome you should be able to use it! I believe from there you can just send the blob directly via websockets; check out my mobile sending example (buried in the binary blob server ex) for doing that with a File: https://github.com/robotconscience/ofxLibwebsockets/blob/master/example_server_blob/bin/data/web/mobile/js/main.js

[leefeel]

Thank you so much @robotconscience for your fast and accurate response.

The method .toBlob was all I needed. It works perfectly. It would be great to add this case to the examples to keep the reference.