Virtual view quality enhancement using side view information for free viewpoint video

Research output: Book chapter/Published conference paperConference paper

Abstract

With the advancement of displaying technologies, virtual viewpoint video needs to be synthesized from adjacent viewpoints to provide immersive perceptual viewing experience of a scene. View synthesized techniques suffer poor rendering quality due to holes created by occlusion in the warping process. Currently, spatial and temporal correlation techniques are used to improve the quality of the synthesized view. However, spatial correlation e. g. inpainting and inverse mapping (IM) techniques cannot fill holes efficiently due to low spatial correlation in the edge between foreground and background pixels. On the other hand, the temporal correlation among already synthesized frames through learning by Gaussian mixture modelling (GMM) may fill occluded areas efficiently. However, there are no frames for GMM learning when the user switches view instantly. To address the aforementioned issues, in the proposed view synthesis technique, we apply GMM on the adjacent viewpoint videos. Then, we utilize the number of GMM models to refine pixel intensities of the synthesized view by using a weighting factor between pixel intensities in GMM models and warped images. This technique provides a better pixel correspondence, which improves 0.47~0.58dB PSNR compared to the IM technique.
Original languageEnglish
Title of host publication2018 IEEE Visual Communications and Image Processing (VCIP)
PublisherIEEE
Number of pages5
ISBN (Electronic)9781538644584
Publication statusPublished - 25 Apr 2019
EventVisual Communications and Image Processing Conference 2018 - Tempus Hotel , Taichung, Taiwan, Province of China
Duration: 09 Dec 201812 Dec 2018
https://ieeexplore.ieee.org/xpl/mostRecentIssue.jsp?punumber=8694905 (conference proceedings)

Conference

ConferenceVisual Communications and Image Processing Conference 2018
CountryTaiwan, Province of China
CityTaichung
Period09/12/1812/12/18
Internet address

Fingerprint Dive into the research topics of 'Virtual view quality enhancement using side view information for free viewpoint video'. Together they form a unique fingerprint.

  • Cite this