Color.py example: Color stream from Orbbec Gemini 336L is uniform gray

[theWoogle]

Bug Report: Color stream from Orbbec Gemini 336L is uniform gray

Summary

When capturing RGB frames from an Orbbec Gemini 336L using the color.py example, every color frame appears as a uniform image (all pixels share the same intensity). The intensity changes when the camera is covered, e.g. all pixels turn black. But it does not show a correctly colored image. OrbbecViewer shows a correct image.

Environment

• Operating system: Linux (host machine running the capture scripts)
• Python: Python 3.11.13
• pyorbbecsdk: 2.0.13 (uv pip show pyorbbecsdk)
• Hardware: Orbbec Gemini 336L (single camera connected over USB)

Steps to Reproduce

1. Connect an Orbbec Gemini 336L to the host machine.
2. Run the example: python3 color.py
3. Inspect the image shown by cv2. The image appears as a flat gray box; the image has no color variation.
4. When covering the camera with something, the image turns into a black rectangle

Expected Behavior

• The color_frame should deliver a full RGB image that reflects the scene captured by the Gemini 336L.

Actual Behavior

• Debugging shows that color_image is a numpy array of the expected size with all pixel values being the same, e.g. [226,226,266]

[theWoogle]

The error seems to be that the stride of the array returned by frame.get_data() is 0. A temporary fix is creating a ctype array like this: extract_frame_data(frame.get_data(),np.uint8)

def extract_frame_data(raw_buffer, dtype=np.uint8) -> np.ndarray:
    # Extract the data pointer from __array_interface__
    if not hasattr(raw_buffer, "__array_interface__"):
        raise RuntimeError("Buffer doesn't have __array_interface__")

    array_interface = raw_buffer.__array_interface__
    data_ptr, readonly = array_interface["data"]
    data_shape = array_interface["shape"]

    # Determine the number of elements based on dtype
    dtype_map = {
        np.uint8: (ctypes.c_uint8, 1),
        np.uint16: (ctypes.c_uint16, 2),
        np.float32: (ctypes.c_float, 4),
    }

    if dtype not in dtype_map:
        raise ValueError(
            f"Unsupported dtype: {dtype}. Use np.uint8, np.uint16, or np.float32"
        )

    c_type, bytes_per_element = dtype_map[dtype]
        total_data_size_bytes = data_shape[0]
        num_elements = total_data_size_bytes // bytes_per_element
        ArrayType = c_type * num_elements
        ctypes_array = ArrayType.from_address(data_ptr)
        # Convert ctypes array to numpy array and copy to own the data
        data = np.frombuffer(ctypes_array, dtype=dtype)
        return data.copy()

[WangYCheng23]

Downgrade the NumPy version to 1.2x.*

[theWoogle]

Thank you for the response. Unfortunately that interferes with my other dependecies

[gongye2]

@theWoogle Hi, Thank you for your feedback! Could you please let us know which NumPy version you are using? This will help us investigate the issue more accurately.

[theWoogle]

I have to use 2.3.3 or higher and am currently using 2.3.4

[gongye2]

I have to use 2.3.3 or higher and am currently using 2.3.4

Support for NumPy ≥ 2.0.0 will be included in the next release scheduled for next week.

[theWoogle]

Thank you looking forward to that : )

[liuchangji]

@theWoogle Thanks. Your code works perfectly.

I modified the code like below.

def frame_to_bgr_image(frame: VideoFrame) -> Union[Optional[np.array], Any]:
    width = frame.get_width()
    height = frame.get_height()
    color_format = frame.get_format()

# -----------------------this line---------------------------
   data = extract_frame_data(frame.get_data(),dtype=np.uint8) 

    image = np.zeros((height, width, 3), dtype=np.uint8)
    if color_format == OBFormat.RGB:
        image = np.resize(data, (height, width, 3))
        image = cv2.cvtColor(image, cv2.COLOR_RGB2BGR)
    elif color_format == OBFormat.BGR:
        image = np.resize(data, (height, width, 3))
        image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
    elif color_format == OBFormat.YUYV:
        image = np.resize(data, (height, width, 2))
        image = cv2.cvtColor(image, cv2.COLOR_YUV2BGR_YUYV)
    elif color_format == OBFormat.MJPG:
        image = cv2.imdecode(data, cv2.IMREAD_COLOR)
    elif color_format == OBFormat.I420:
        image = i420_to_bgr(data, width, height)
        return image
    elif color_format == OBFormat.NV12:
        image = nv12_to_bgr(data, width, height)
        return image
    elif color_format == OBFormat.NV21:
        image = nv21_to_bgr(data, width, height)
        return image
    elif color_format == OBFormat.UYVY:
        image = np.resize(data, (height, width, 2))
        image = cv2.cvtColor(image, cv2.COLOR_YUV2BGR_UYVY)
    else:
        print("Unsupported color format: {}".format(color_format))
        return None
    return image