DepthCloud streaming format error

[arnasIV]

I can see the image being published at http://localhost:8888/stream?topic=depthcloud_encoded&type=mjpeg Whenever I try to use this url in the depthcloud node:

depthCloud = new ROS3D.DepthCloud({
   url : 'http://localhost:8888/stream?topic=depthcloud_encoded&type=mjpeg',
   streamType : 'mjpeg',
   f : 819.0
 });

I get the following error:

DOMException: Failed to execute 'texImage2D' on 'WebGLRenderingContext': The cross-origin image at 'http://localhost:8888/stream?topic=depthcloud_encoded&type=mjpeg', may not be loaded. at THREE.WebGLRenderer.setTexture (three.js:25507:10) at loadUniformsGeneric (/three.js:24428:12) at setProgram (three.js:23957:4) at THREE.WebGLRenderer.renderBuffer (three.js:22336:17) at renderObjects (fthree.js:23026:12) at THREE.WebGLRenderer.render (three.js:22900:4) at draw (fros3d.js:3841:19)

[sevenbitbyte]

This could possibly be a CORS issue. Have you loading your web application from a local http server or just loading it directly from the file system? If you are using the file system, try hosting it on http://localhost and see if you get a different error.

[arnasIV]

Yes, I believe it is the problem. However using web_video_server I cannot specify the www_file_server param as it was possible using ros_web_video. Do you have any ideas on how I could load my files on the same server as the image stream?

[viktorku]

I had many issues with streaming MJPEG: from playing around and reordering the CORS headers in the web_video_server (try this fork) to trying various ways to specify the attributes in the image on the client.

I wasn't able to make it work so I gave up and used a proxy: https://github.com/legege/node-mjpeg-proxy.

[oka1125]

I found a solution for this issue.

The cause exists in HTTP HEADER from web_video_server stream.

curl -I http://localhost:9999/stream?topic=/depthcloud_encoded&type=mjpeg

HTTP/1.0 200 OK
Connection: close
Server: web_video_server
Cache-Control: no-cache, no-store, must-revalidate, pre-check=0, post-check=0, max-age=0
Pragma: no-cache
Content-type: multipart/x-mixed-replace;boundary=boundarydonotcross
Access-Control-Allow-Origin: *

There are only "Access-Control-Allow-Origin: *", but some other values are needed to allow origin.

After I edit web_video_server package's multipart_stream.cpp as follows (add some 'Access-Control-' tags) and rebuild, DOMException(cross-origin) no longer occured in ros3d.js.

void MultipartStream::sendInitialHeader() {
  async_web_server_cpp::HttpReply::builder(async_web_server_cpp::HttpReply::ok).header("Connection", "close").header(
      "Server", "web_video_server").header("Cache-Control",
                                           "no-cache, no-store, must-revalidate, pre-check=0, post-check=0, max-age=0").header(
      "Pragma", "no-cache").header("Content-type", "multipart/x-mixed-replace;boundary="+boundry_).header(
      "Access-Control-Allow-Origin", "*").header("Access-Control-Allow-Methods", "GET,POST,PUT,DELETE,HEAD,OPTIONS").header(
      "Access-Control-Allow-Headers", "Origin, Authorization, Accept, Content-Type").header("Access-Control-Max-Age", "3600").write(connection_);
  connection_->write("--"+boundry_+"\r\n");
}

void MultipartStream::sendPartHeader(const ros::Time &time, const std::string& type, size_t payload_size) {
  char stamp[20];
  sprintf(stamp, "%.06lf", time.toSec());
  boost::shared_ptr<std::vector<async_web_server_cpp::HttpHeader> > headers(
      new std::vector<async_web_server_cpp::HttpHeader>());
  headers->push_back(async_web_server_cpp::HttpHeader("Content-type", type));
  headers->push_back(async_web_server_cpp::HttpHeader("X-Timestamp", stamp));
  headers->push_back(async_web_server_cpp::HttpHeader("Access-Control-Allow-Origin", "*"));
  headers->push_back(async_web_server_cpp::HttpHeader("Access-Control-Allow-Methods", "GET,POST,PUT,DELETE,HEAD,OPTIONS"));
  headers->push_back(async_web_server_cpp::HttpHeader("Access-Control-Allow-Headers", "Origin, Authorization, Accept, Content-Type"));
  headers->push_back(async_web_server_cpp::HttpHeader("Access-Control-Max-Age", "3600"));
  headers->push_back(
      async_web_server_cpp::HttpHeader("Content-Length", boost::lexical_cast<std::string>(payload_size)));
  connection_->write(async_web_server_cpp::HttpReply::to_buffers(*headers), headers);
}

Then, the HTTP HEADER becomes as follows,

curl -I http://localhost:9999/stream?topic=/depthcloud_encoded&type=mjpeg

HTTP/1.0 200 OK
Connection: close
Server: web_video_server
Cache-Control: no-cache, no-store, must-revalidate, pre-check=0, post-check=0, max-age=0
Pragma: no-cache
Content-type: multipart/x-mixed-replace;boundary=boundarydonotcross
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: GET,POST,PUT,DELETE,HEAD,OPTIONS
Access-Control-Allow-Headers: Origin, Authorization, Accept, Content-Type
Access-Control-Max-Age: 3600

[oka1125]

Fixing DepthCloud class code in fros3d.js is also needed to avoid DOMException (cross origin), since this.video.crossOrigin='Anonymous' was set after 'this.video.loop = true;' and 'this.video.src = this.url'.

class DepthCloud extends THREE$1.Object3D {
   ...
   this.video = document.createElement(this.isMjpeg ? 'img' : 'video');
   this.video.crossOrigin = 'Anonymous';  // INSERT THIS CODE
   ...
}