Virtual view quality enhancement using side view information for free viewpoint video

D.M. Motiur Rahaman, Manoranjan Paul, Nusrat Jahan Shoumy

Research output: Book chapter/Published conference paperConference paperpeer-review


With the advancement of displaying technologies, virtual viewpoint video needs to be synthesized from adjacent viewpoints to provide immersive perceptual viewing experience of a scene. View synthesized techniques suffer poor rendering quality due to holes created by occlusion in the warping process. Currently, spatial and temporal correlation techniques are used to improve the quality of the synthesized view. However, spatial correlation e. g. inpainting and inverse mapping (IM) techniques cannot fill holes efficiently due to low spatial correlation in the edge between foreground and background pixels. On the other hand, the temporal correlation among already synthesized frames through learning by Gaussian mixture modelling (GMM) may fill occluded areas efficiently. However, there are no frames for GMM learning when the user switches view instantly. To address the aforementioned issues, in the proposed view synthesis technique, we apply GMM on the adjacent viewpoint videos. Then, we utilize the number of GMM models to refine pixel intensities of the synthesized view by using a weighting factor between pixel intensities in GMM models and warped images. This technique provides a better pixel correspondence, which improves 0.47~0.58dB PSNR compared to the IM technique.
Original languageEnglish
Title of host publication2018 IEEE Visual Communications and Image Processing (VCIP)
Place of PublicationUnited States
Number of pages5
ISBN (Electronic)9781538644584
ISBN (Print)9781538644591 (Print on demand)
Publication statusPublished - 25 Apr 2019
EventVisual Communications and Image Processing Conference 2018 - Tempus Hotel , Taichung, Taiwan, Province of China
Duration: 09 Dec 201812 Dec 2018 (Conference website) (conference proceedings)


ConferenceVisual Communications and Image Processing Conference 2018
Country/TerritoryTaiwan, Province of China
OtherThe Visual Communications and Image Processing (VCIP) Conference, sponsored by the IEEE Circuits and Systems Society will be held at Tempus Hotel of the beautiful city of Taichung, Taiwan, during December 9-12, 2018.

Since 1986, VCIP has served as a premier forum in SPIE for the exchange of fundamental research results and technological advances in the field of visual communications and image processing. The 2018 edition will be the eighth time that VCIP will be held under the auspices of the IEEE. IEEE VCIP 2018 shall inherit the tradition of previous conferences in providing fertile ground for leading engineers and scientists from around the world to escalate collaboratively the research frontiers within these and related areas.
Internet address


Dive into the research topics of 'Virtual view quality enhancement using side view information for free viewpoint video'. Together they form a unique fingerprint.

Cite this