A Novel Viewport-Adaptive Motion Compensation Technique for Fisheye Video
We propose a novel viewport-adaptive motion compensation technique for fisheye video that reaches average gains of 2.40 dB in terms of PSNR compared to the state of the art in fisheye motion compensation. The former state-of-the-art projection-based approach takes the fisheye projection into account to improve the motion compensation performance by modeling motions in the perspective domain. However, this approach does not take the large fields of view of fisheye lenses into account, which require the consideration of different motion planes in 3D space.
The proposed viewport-adaptive motion compensation technique realizes these motion planes by modeling the motion in different perspective viewports. Thereby, some pixels are mapped to so-called virtual image planes and require special treatment to obtain reliable mappings between the perspective viewports and the original fisheye image. While the state-of-the-art ultra wide-angle compensation is sufficiently accurate, the proposed virtual image plane compensation leads to perfect mappings.
Basic intuition and further information about the proposed viewport-adaptive approach is provided in the following presentation.
This contribution was presented at ICASSP 2021 (6-11 June 2021).
Research at the chair is regularly being published at international conferences and impactful journals. In times of virtual conferences, many organizers have resorted to presentations in the form of pre-recorded videos that can be accessed on-demand by attendees. We use this opportunity to make our research available to a broader public by sharing some of our contributions on our website.