Creating InputImage from Uint8List

[Jayvd]

Ho do we create an InputImage if all we have Uint8List? How do we create the Plane object needed?

The link below from the Google ML Kit Github page doesn't describe how we can create an Input Image if we only have Uint8List object.

https://github.com/bharat-biradar/Google-Ml-Kit-plugin/tree/master/packages/google_mlkit_commons#creating-an-inputimage

Thank you

[Jebiel]

I've got the exact same question. I'm not able to find good documentation on this.

[DavidGrunheidt]

Ho do we create an InputImage if all we have Uint8List? How do we create the Plane object needed?

The link below from the Google ML Kit Github page doesn't describe how we can create an Input Image if we only have Uint8List object.

https://github.com/bharat-biradar/Google-Ml-Kit-plugin/tree/master/packages/google_mlkit_commons#creating-an-inputimage

Thank you

I've got the exact same question. I'm not able to find good documentation on this.

That's what I did on my app to create the InputImage:

Future<InputImage> createInputImage(CreateInputImageArgs args) async {
  final image = args.image;
  final cameraDescription = args.cameraDescription;

  final bytes = Uint8List.fromList(image.planes.expand((p) => p.bytes).toList());
  final imageSize = Size(image.width.toDouble(), image.height.toDouble());
  final imageRotation = InputImageRotationValue.fromRawValue(cameraDescription.sensorOrientation) ?? InputImageRotation.rotation0deg;
  final inputImageFormat = InputImageFormatValue.fromRawValue(image.format.raw) ?? InputImageFormat.nv21;

  final planeData = image.planes.map((plane) {
    return InputImagePlaneMetadata(bytesPerRow: plane.bytesPerRow, height: plane.height, width: plane.width);
  }).toList();

  final inputImageData = InputImageData(size: imageSize, imageRotation: imageRotation, inputImageFormat: inputImageFormat, planeData: planeData);
  return InputImage.fromBytes(bytes: bytes, inputImageData: inputImageData);
}

If you already have the Uint8List you could proceed after the line where I define the bytes value. I'm also calling that inside a compute

    // Do heavy computations outside the main thread, so we don't freeze the UI
    final inputImage = await compute<CreateInputImageArgs, InputImage>(
      createInputImage,
      CreateInputImageArgs(image: image, cameraDescription: cameraDescription),
    );

[Parfyonator]

@DavidGrunheidt it looks like instance args of class CreateInputImageArgs contains all the necessary information about the image, its format and planes. Is this a class from some package/plugin? I wasn't able to find it at pub.dev

[chris1869]

From the source it seems that the image-format nv21 is hard-coded in the InputImageConverter.java module. So in case you have bytes from another source (e.g., Bitmaps) you might need to convert it from rgb to nv21.

https://github.com/bharat-biradar/Google-Ml-Kit-plugin/blob/master/packages/google_mlkit_commons/android/src/main/java/com/google_mlkit_commons/InputImageConverter.java

This might be more complicated as nv21 is a planar data format and needs some conversion scripts. In my situation, I needed to convert greyscale to yuv. This is quite easy as this corresponds to the Y component of nv21. Once the conversion worked the text recognizer started working reliably.

Uint8List convertGreytoYuv(List<int> grey, int width, int height) {
  int size = width * height;
  List<int> yuvRaw = List.empty(growable: true);
  yuvRaw.addAll(grey);
  yuvRaw.addAll(List.filled(size ~/ 2, 0));
  return Uint8List.fromList(yuvRaw);
}

In the future, it would be great if google_mlkit_commons provides a direct feed through for the data format that is send to the google_mlkit