Slow simGetImages function with Scene and DepthPlanar

[DanieleMarchisotti]

Hi,

I am opening this issue about the time required to get images in Cosys-Airsim. The simGetImages function goes very slow when saving Scene and DepthPlanar images.

This is the code I use to get images:

scene_requests = [
    airsim.ImageRequest("0", airsim.ImageType.Scene, False, False), # camera_name, image_type, pixels_as_float, compress
    airsim.ImageRequest("0", airsim.ImageType.DepthPlanar, True, False) # camera_name, image_type, pixels_as_float, compress
]
getimages_time_start = time.time()
scene_responses = self.client.simGetImages(scene_requests)
getimages_time_stop = time.time()
print("Time taken to get images: ", getimages_time_stop - getimages_time_start)
for idx, response in enumerate(scene_responses):
    if response.pixels_as_float:
        filename = self.save_dir + "/depth/" + str(response.time_stamp)
        pfm_time_start = time.time()
        self.depth = airsim.get_pfm_array(response)
        pfm_time_end = time.time()
        print("Time taken to get pfm image: ", pfm_time_end - pfm_time_start)
        airsim.write_pfm(os.path.normpath(filename + '.pfm'), self.depth)
    else: #uncompressed array
        filename = self.save_dir + "/rgb/" + str(response.time_stamp)
        rgb_timestamp = response.time_stamp
        rgb_time_start = time.time()
        img1d = np.fromstring(response.image_data_uint8, dtype=np.uint8) # get numpy array
        rgb_time_end = time.time()
        print("Time taken to get rgb array: ", rgb_time_end - rgb_time_start)
        img_rgb = img1d.reshape(response.height, response.width, 3)
        if self.ue_ver == 5:
            img_rgb = cv2.cvtColor(img_rgb, cv2.COLOR_BGR2RGB)
        cv2.imwrite(os.path.normpath(filename + '.png'), img_rgb) # write to png

and the performances with CityPark environment are the following, 2-3FPS (similar but slightly worse performance are obtained with ElectricDreams environment):

Captured frame:  0
Time taken to get images:  0.4015023708343506
Time taken to get rgb array:  0.0005230903625488281
Time taken to get pfm image:  0.03378462791442871
Time taken to capture data:  0.5081825256347656
Captured frame:  1
Time taken to get images:  0.56178879737854
Time taken to get rgb array:  0.0004322528839111328
Time taken to get pfm image:  0.03719067573547363
Time taken to capture data:  0.6768307685852051
Captured frame:  2
Time taken to get images:  0.2557053565979004
Time taken to get rgb array:  0.0004165172576904297
Time taken to get pfm image:  0.03760504722595215
Time taken to capture data:  0.36663150787353516

while with blocks environment I get the following:

Captured frame:  0
Time taken to get images:  0.04610753059387207
Time taken to get rgb array:  0.001096963882446289
Time taken to capture data:  0.06566596031188965
Captured frame:  1
Time taken to get images:  0.0488286018371582
Time taken to get rgb array:  0.0011472702026367188
Time taken to capture data:  0.0663454532623291
Captured frame:  2
Time taken to get images:  0.04476642608642578
Time taken to get rgb array:  0.0011594295501708984
Time taken to capture data:  0.06387186050415039

which is better (around 20 FPS).

Do you know how could I increase speed of simGetImages? Is it a matter of Cosys-Airsim or UE environments?

I have seen a similar issue also in Airsim https://github.com/microsoft/AirSim/discussions/3566