Virtual view quality enhancement using side view temporal modelling information for free viewpoint video

Research output: Book chapter/Published conference paperConference paper

Abstract

Virtual viewpoint video needs to be synthesised from adjacent reference viewpoints to provide immersive perceptual 3D viewing experience of a scene. View synthesised techniques suffer poor rendering quality due to holes created by occlusion in the warping process. Currently, spatial and temporal correlation of texture images and depth maps are exploited to improve the quality of the final synthesised view. Due to the low spatial correlation at the edge between foreground and background pixels, spatial correlation e.g. inpainting and inverse mapping (IM) techniques cannot fill holes effectively. Conversely, a temporal correlation among already synthesised frames through learning by Gaussian mixture modelling (GMM) fill missing pixels in occluded areas efficiently. In this process, there are no frames for GMM learning when the user switches view instantly. To address the above issues, in the proposed view synthesis technique, we apply GMM on the adjacent reference viewpoint texture images and depth maps to generate a most common frame in a scene (McFIS). Then, texture McFIS is warped into the target viewpoint by using depth McFIS and both warped McFISes are merged. Then, we utilize the number of GMM models to refine pixel intensities of the synthesised view by using a weighting factor between the pixel intensities of the merged McFIS and the warped images. This technique provides a better pixel correspondence and improves 0.58∼0.70dB PSNR compared to the IM technique.

Original languageEnglish
Title of host publication2018 International Conference on Digital Image Computing
Subtitle of host publicationTechniques and Applications, DICTA 2018
EditorsMark Pickering, Lihong Zheng, Shaodi You, Ashfaqur Rahman, Manzur Murshed, Md Asikuzzaman, Ambarish Natu, Antonio Robles-Kelly, Manoranjan Paul
PublisherIEEE, Institute of Electrical and Electronics Engineers
ISBN (Electronic)9781538666029
DOIs
Publication statusPublished - 16 Jan 2019
Event2018 International Conference on Digital Image Computing: Techniques and Applications: DICTA 2018 - Canberra Rex Hotel, Canberra, Australia
Duration: 10 Dec 201813 Dec 2018
https://dicta2018.org/ (Conference website)

Publication series

Name2018 International Conference on Digital Image Computing: Techniques and Applications, DICTA 2018

Conference

Conference2018 International Conference on Digital Image Computing: Techniques and Applications
CountryAustralia
CityCanberra
Period10/12/1813/12/18
Internet address

Fingerprint Dive into the research topics of 'Virtual view quality enhancement using side view temporal modelling information for free viewpoint video'. Together they form a unique fingerprint.

  • Cite this

    Rahaman, D. M. M., Paul, M., & Shoumy, N. J. (2019). Virtual view quality enhancement using side view temporal modelling information for free viewpoint video. In M. Pickering, L. Zheng, S. You, A. Rahman, M. Murshed, M. Asikuzzaman, A. Natu, A. Robles-Kelly, & M. Paul (Eds.), 2018 International Conference on Digital Image Computing: Techniques and Applications, DICTA 2018 [8615827] (2018 International Conference on Digital Image Computing: Techniques and Applications, DICTA 2018). IEEE, Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/DICTA.2018.8615827