Decimation filter distorting pointcloud with OAK-D

[misofey]

Setting decimation filter to anything other than 1 distorts pointcloud in a very weird way. It appears that all the points are transformed to positive xy coordinates and/or removed. I have plotted a histogram of all point xyz coordinates with and without the decimation filters. Also included are images of the pointclouds visualized. Bit ugly but it illustrates the issue.

Without decimation filter:

With decimation filter:

here is the code that i run:

import numpy as np
import depthai as dai
from pypcd4 import PointCloud
import os
import matplotlib.pyplot as plt


def create_pipeline():
    pipeline = dai.Pipeline()
    monoLeft = pipeline.create(dai.node.MonoCamera)
    monoRight = pipeline.create(dai.node.MonoCamera)
    depth = pipeline.create(dai.node.StereoDepth)
    sync = pipeline.create(dai.node.Sync)
    xOut = pipeline.create(dai.node.XLinkOut)

    # camRgb.setResolution(dai.ColorCameraProperties.SensorResolution.THE_720_P)
    # camRgb.setBoardSocket(dai.CameraBoardSocket.CAM_A)
    # camRgb.setIspScale(1, 3)
    camRgb.setFps(15)

    monoLeft.setResolution(dai.MonoCameraProperties.SensorResolution.THE_800_P)
    # monoLeft.setBoardSocket(dai.CameraBoardSocket.CAM_B)
    monoLeft.setFps(15)

    monoRight.setResolution(dai.MonoCameraProperties.SensorResolution.THE_800_P)
    # monoRight.setBoardSocket(dai.CameraBoardSocket.CAM_C)
    monoRight.setFps(15)

    # depth.setDefaultProfilePreset(dai.node.StereoDepth.PresetMode.HIGH_ACCURdudsACY)
    depth.setLeftRightCheck(True)
    # depth.setSubpixel(True)
    # depth.setSubpixelFractionalBits(3)
    # depth.initialConfig.setConfidenceThreshold(50)
    depth.setPostProcessingHardwareResources(3, 3)



    pp_config = depth.initialConfig.get()
    # pp_config.postProcessing.spatialFilter.enable = True
    # pp_config.postProcessing.speckleFilter.enable = False
    # pp_config.postProcessing.speckleFilter.speckleRange = 50
    pp_config.postProcessing.decimationFilter.decimationFactor = 1
    depth.initialConfig.set(pp_config)

    depth.setDepthAlign(dai.CameraBoardSocket.CAM_B)
    pointcloud = pipeline.create(dai.node.PointCloud)

    monoLeft.out.link(depth.left)
    monoRight.out.link(depth.right)
    depth.depth.link(pointcloud.inputDepth)
    pointcloud.outputPointCloud.link(sync.inputs['pcl'])



    sync.out.link(xOut.input)

    xOut.setStreamName("output")

    return pipeline


pipeline = create_pipeline()
with dai.Device(pipeline) as device:
    q = device.getOutputQueue(name="output", maxSize=4, blocking=False)
    frame_count = 0
    output_dir = "pointclouds/home"
    os.makedirs(output_dir, exist_ok=True)
    
    fig, axs = plt.subplots(3, 1)


    while device.isPipelineRunning():
        inMessage = q.get()

        inPointCloud = inMessage["pcl"]
        inMono = inMessage["monoLeft"]

        if inPointCloud:
            points = inPointCloud.getPoints()
            mask = ~np.all(np.isclose(points[:, :3], 0), axis=1)
            points = points[mask]
            pc = PointCloud.from_xyz_points(points)
            pcd_filename = f"{output_dir}/frame_{frame_count:04d}.pcd"
            if input('asdf') == "a":
                frame_count += 1
                pc.save(pcd_filename)

                axs[0].hist(points[:, 0], bins=100, range=[-3000, 3000])
                axs[1].hist(points[:, 1], bins=100, range=[-3000, 3000])
                axs[2].hist(points[:, 2], bins=100, range=[-3000, 3000])
                
                plt.show()`

[Erol444]

Hi @misofey , We have found the issue, it's a bug in PointCloud node (something with intrinsics/size scaling), we'll push the fix to develop soon. We apologize for the inconvenience. Until then, you could either manually create pointcloud (instead of using on-device node), example here: https://github.com/luxonis/depthai-experiments/tree/master/gen2-pointcloud

[SzabolcsGergely]

@misofey please check out latest develop

[nturro]

Hello,

i encounter the same bug using the luxonis/depthai-ros:humble-latest container

Is there a luxonis/depthai-ros container with this bug fixed ?

[Erol444]

(Continuing conversation on the new issue, closing this one as it has been fixed for >6 months now)