wrong result at toPImage -> toCv() -> toPImage()

[kougaku]

Hi, thank you for your great contribution. If you don't mind, could you please help me?

It cause wrong result that not became original image when converting toPImage -> toCv() -> toPImage().

In the following code, create a red image (PImage) and convert it Mat by toCv(). After that, convert its Mat to PImage by toPImage(). As the result, the final image becomes cyan image, not original red.

import gab.opencv.*;
import org.opencv.core.*;

OpenCV opencv;
PImage input, output;

void setup() {
  size(800, 300);
  
  opencv = new OpenCV(this, 0, 0);
  input = createImage(400, 300, ARGB);

  // fill red
  for (int y=0; y<input.height; y++) {
    for (int x=0; x<input.width; x++) {
      input.set(x,y,color(255,0,0));
    }
  }
  
  Mat mat = new Mat(input.height, input.width, CvType.CV_8UC4);
  OpenCV.toCv(input, mat);
  
  output = createImage(input.width, input.height, RGB);
  opencv.toPImage(mat, output);
  
  println( "first pixel of input:  " + binary(input.pixels[0]) );
  println( "first pixel of mat:"); 
  println(mat.get(0,0));
  println( "first pixel of output :" + binary(output.pixels[0]) );
}

void draw() {
  image(input, 0, 0);
  image(output, input.width, 0); 
}

It also shows:

first pixel of input:  11111111111111110000000000000000
first pixel of mat:
[0] 255.0
[1] 255.0
[2] 0.0
[3] 0.0
first pixel of output :00000000000000001111111111111111

It seems that toPImage() causing wrong alignment of the pixels.

[kougaku]

PImage has pixel data stored in ARGB order. In OpenCV java, Imgcodecs.imread() provides Mat that has pixels stored in BGR. Would it be better to convert ARGB to BGR (or BGRA) when convert PImage to Mat?

The following is a shortcut solution, but it might not good.

  // PImage -> Mat  
  opencv.toCv(input, mat);
  
  // ARGB -> BGRA
  double[] out = new double [4];
  for (int i=0; i<mat.rows(); i++) {
    for (int j=0; j<mat.cols(); j++) {
      double[] in = mat.get(i, j);
      out[0] = in[3];
      out[1] = in[2];
      out[2] = in[1];
      out[3] = in[0];
      mat.put(i,j,out);
    }
  }

I wanted use cvtColor(), unfortunately, Imgproc does not have COLOR_ARGB2BGRA.

[cansik]

Hi @kougaku, sorry for my late reply I just did not have the time to look into that until now. In general it is recommended to use loadImage() to load an image into opencv. This method does the ARGBtoRGBA conversion when converting from PImageToOpenCV and fixes your issue (as far as I can see).

But if you want to use the toCV() method and handle the conversion your own: ARGB2BGRA is already implemented in opencv-processing, but the function is quite slow (for real-time applications). I have implemented a faster one for the deep-vision-processing library in javacv. I think we could re-use this one for your problem as well:

private static final int[] ARGB2BGRA_MAPPING = new int[] { 0,3, 1,2, 2,1, 3,0 };

public static void ARGBtoBGRAFast(Mat rgba, Mat bgra) {
    mixChannels(rgba, 1, bgra, 1, ARGB2BGRA_MAPPING, 4);
}

Converted into plain opencv it looks like that:

public static void ARGBtoBGRA(Mat rgba, Mat bgra) {
    Core.mixChannels(Arrays.asList(rgba), Arrays.asList(bgra), new MatOfInt(0, 3, 1, 2, 2, 1, 3, 0));
}

Does that help you?

[kougaku]

Thank you for very helpful advice. Sorry for late reply. I tryed it, and it works. I see that the same variable cannot be passed to the input and output of ARGBtoBGRA(), like ARGBtoBGRA(mat, mat). It would be better if it could be.