Modelling fisheye cameras

[tharindurmt]

I am trying to record camera images from four cameras which have a very high field of view of 194 degrees, which AirSim does not support as of yet (if the FoV is more than 180 degrees the images will be flipped vertically. Even if the FoV is set to be just below 180 degree there is no fisheye effect).

AirSim cameras operate a little different to other sensors and as per my understanding, it uses SceneCaptureComponent2D of UE to generate the images using PIPCamera.

So, I was wondering what would be the most suitable way to implement fisheye model. One option would be to change the camera projection model of UE. Is there any other way to achieve this in AirSim side (other than the trivial solution of stitching the output of multiple cameras. This is not an option for me as I need four fisheye cameras with 194 FoV, which means that I will have to create at least 20 cameras with 90 FoV).

Related issues: #2129, #2379

[tharindurmt]

@jonyMarino @rajat2004 Is it possible use PIPCamera with UE LensDistortion plugin? It provides a camera matrix and its parameters can be changed accordingly. I tried adding it to the PIPCamera blueprint as in the following figure (I am completely new to UE). Material and a render target used in this are created following this Global Shader as a Plugin UE tutorial (steps 1 to 3 of Blueprint Setup section). I was under the impression that this transformation will affect any content passed through PIPCamera blueprint. However, this seems to have no effect on captured images. I can't figure out what's wrong with my approach. Could you please help me out?

[petergerten]

@tharindurmt Did you get further with this? I try to get a 220deg fisheye camera to work.

[tharindurmt]

@petergerten Unfortunately, no, I was not able to get this to work. I have a very limited knowledge of UE and I wasn't even sure what i was doing. But from my research, the best bet would be to use post-processing materials. I couldn't figure out a way to integrate the post-processing material to AirSim.

[stale[bot]]

This issue has been automatically marked as stale because it has not had activity from the community in the last year. It will be closed if no further activity occurs within 20 days.

[zsy1987]

@petergerten Unfortunately, no, I was not able to get this to work. I have a very limited knowledge of UE and I wasn't even sure what i was doing.不幸的是，不，我无法让它工作。我对 UE 的了解非常有限，我什至不确定我在做什么。 But from my research, the best bet would be to use post-processing materials. I couldn't figure out a way to integrate the post-processing material to AirSim.但根据我的研究，最好的办法是使用后处理材料。我想不出将后处理材料集成到 AirSim 的方法。

Have you achieved a 200-degree fisheye camera yet？