A novel framework for video summarization based on smooth pursuit information from eye tracker data

Md Musfequs Salehin, Manoranjan Paul

Research output: Book chapter/Published conference paperConference paper

3 Citations (Scopus)

Abstract

Existing methods for video summarization fails to achieve a satisfactory result for a video with camera movement, low contrast, and significant illumination changes. To solve these problems, we propose a novel framework for video summarization based on the smooth pursuit which is the state of eye movement when a user follows a moving object in a video. First, we propose a new method to distinguish smooth pursuit from another type of gaze points, such as fixation and saccade. Later, we assign a probability score to each frame based on the smooth pursuit information. Finally, we select a set of key frames based on the probability score. To evaluate the proposed method, we implement it on Office video dataset that contains videos with camera movement/shaking and illumination changes. Experimental results show the superior performance compared to the single view results of the state-of-the-art GMM based method.
Original languageEnglish
Title of host publicationProceedings of the IEEE International Conference on Multimedia and Expo Workshops (ICMEW) 2017
Place of PublicationUnited States
PublisherIEEE, Institute of Electrical and Electronics Engineers
Pages692-697
Number of pages6
ISBN (Electronic)9781538605608
DOIs
Publication statusPublished - 07 Sep 2017
Event2017 IEEE International Conference on Multimedia and Expo: ICME 2017 - Harbour Grand Kowloon, Hong Kong, Hong Kong
Duration: 10 Jul 201714 Jul 2017
http://www.icme2017.org/ (Conference website)
http://ieeexplore.ieee.org/xpl/mostRecentIssue.jsp?punumber=8014303 (Conference proceedings (ICME 2017))
http://ieeexplore.ieee.org/xpl/mostRecentIssue.jsp?punumber=8014334 (Conference proceedings (ICMEW 2017))

Conference

Conference2017 IEEE International Conference on Multimedia and Expo
Abbreviated titleThe New Media Experience
CountryHong Kong
CityHong Kong
Period10/07/1714/07/17
OtherThe IEEE International Conference on Multimedia & Expo (ICME) has been the flagship multimedia conference sponsored by four IEEE societies since 2000. It serves as a forum to promote the exchange of the latest advances in multimedia technologies, systems, and applications from both the research and development perspectives of the circuits and systems, communications, computer, and signal processing communities. ICME also features an Exposition of multimedia products and prototypes. ICME 2017 is the 18th ICME conference. The main theme of 2017 is "The New Media Experience", enabling next generation 3D/AR/VR experiences and applications, based on which various sessions and events, in particular a Grand Challenge, will be organized. About 400 participants mainly from Asia, Europe and North America will gather in Hong Kong to discuss and progress latest development in multimedia technologies and related fields. This year, the best contributions to the conference will be honoured with the 10k Best Paper Award which promotes research advances in the general Multimedia related areas: Text, Graphics, Vision, Image, Video, Audio, Speech, Sensing data, and their mining, learning, processing, compression, communications, rendering, and associated innovations and applications.
Internet address

Fingerprint

Eye movements
Lighting
Cameras

Cite this

Salehin, M. M., & Paul, M. (2017). A novel framework for video summarization based on smooth pursuit information from eye tracker data. In Proceedings of the IEEE International Conference on Multimedia and Expo Workshops (ICMEW) 2017 (pp. 692-697). [8026294] United States: IEEE, Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/ICMEW.2017.8026294
Salehin, Md Musfequs ; Paul, Manoranjan. / A novel framework for video summarization based on smooth pursuit information from eye tracker data. Proceedings of the IEEE International Conference on Multimedia and Expo Workshops (ICMEW) 2017. United States : IEEE, Institute of Electrical and Electronics Engineers, 2017. pp. 692-697
@inproceedings{8ce9def342044618b439181df59638f2,
title = "A novel framework for video summarization based on smooth pursuit information from eye tracker data",
abstract = "Existing methods for video summarization fails to achieve a satisfactory result for a video with camera movement, low contrast, and significant illumination changes. To solve these problems, we propose a novel framework for video summarization based on the smooth pursuit which is the state of eye movement when a user follows a moving object in a video. First, we propose a new method to distinguish smooth pursuit from another type of gaze points, such as fixation and saccade. Later, we assign a probability score to each frame based on the smooth pursuit information. Finally, we select a set of key frames based on the probability score. To evaluate the proposed method, we implement it on Office video dataset that contains videos with camera movement/shaking and illumination changes. Experimental results show the superior performance compared to the single view results of the state-of-the-art GMM based method.",
keywords = "eye movement, eye tracker, Smooth pursuit, video summarization",
author = "Salehin, {Md Musfequs} and Manoranjan Paul",
year = "2017",
month = "9",
day = "7",
doi = "10.1109/ICMEW.2017.8026294",
language = "English",
pages = "692--697",
booktitle = "Proceedings of the IEEE International Conference on Multimedia and Expo Workshops (ICMEW) 2017",
publisher = "IEEE, Institute of Electrical and Electronics Engineers",
address = "United States",

}

Salehin, MM & Paul, M 2017, A novel framework for video summarization based on smooth pursuit information from eye tracker data. in Proceedings of the IEEE International Conference on Multimedia and Expo Workshops (ICMEW) 2017., 8026294, IEEE, Institute of Electrical and Electronics Engineers, United States, pp. 692-697, 2017 IEEE International Conference on Multimedia and Expo, Hong Kong, Hong Kong, 10/07/17. https://doi.org/10.1109/ICMEW.2017.8026294

A novel framework for video summarization based on smooth pursuit information from eye tracker data. / Salehin, Md Musfequs; Paul, Manoranjan.

Proceedings of the IEEE International Conference on Multimedia and Expo Workshops (ICMEW) 2017. United States : IEEE, Institute of Electrical and Electronics Engineers, 2017. p. 692-697 8026294.

Research output: Book chapter/Published conference paperConference paper

TY - GEN

T1 - A novel framework for video summarization based on smooth pursuit information from eye tracker data

AU - Salehin, Md Musfequs

AU - Paul, Manoranjan

PY - 2017/9/7

Y1 - 2017/9/7

N2 - Existing methods for video summarization fails to achieve a satisfactory result for a video with camera movement, low contrast, and significant illumination changes. To solve these problems, we propose a novel framework for video summarization based on the smooth pursuit which is the state of eye movement when a user follows a moving object in a video. First, we propose a new method to distinguish smooth pursuit from another type of gaze points, such as fixation and saccade. Later, we assign a probability score to each frame based on the smooth pursuit information. Finally, we select a set of key frames based on the probability score. To evaluate the proposed method, we implement it on Office video dataset that contains videos with camera movement/shaking and illumination changes. Experimental results show the superior performance compared to the single view results of the state-of-the-art GMM based method.

AB - Existing methods for video summarization fails to achieve a satisfactory result for a video with camera movement, low contrast, and significant illumination changes. To solve these problems, we propose a novel framework for video summarization based on the smooth pursuit which is the state of eye movement when a user follows a moving object in a video. First, we propose a new method to distinguish smooth pursuit from another type of gaze points, such as fixation and saccade. Later, we assign a probability score to each frame based on the smooth pursuit information. Finally, we select a set of key frames based on the probability score. To evaluate the proposed method, we implement it on Office video dataset that contains videos with camera movement/shaking and illumination changes. Experimental results show the superior performance compared to the single view results of the state-of-the-art GMM based method.

KW - eye movement

KW - eye tracker

KW - Smooth pursuit

KW - video summarization

UR - http://www.scopus.com/inward/record.url?scp=85031696003&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85031696003&partnerID=8YFLogxK

UR - http://www.icme2017.org/wp-content/uploads/2016/11/ICME2017_CFP_final_november.pdf

UR - http://www.icme2017.org/wp-content/uploads/2017/06/technical_program_final.pdf

U2 - 10.1109/ICMEW.2017.8026294

DO - 10.1109/ICMEW.2017.8026294

M3 - Conference paper

AN - SCOPUS:85031696003

SP - 692

EP - 697

BT - Proceedings of the IEEE International Conference on Multimedia and Expo Workshops (ICMEW) 2017

PB - IEEE, Institute of Electrical and Electronics Engineers

CY - United States

ER -

Salehin MM, Paul M. A novel framework for video summarization based on smooth pursuit information from eye tracker data. In Proceedings of the IEEE International Conference on Multimedia and Expo Workshops (ICMEW) 2017. United States: IEEE, Institute of Electrical and Electronics Engineers. 2017. p. 692-697. 8026294 https://doi.org/10.1109/ICMEW.2017.8026294