Detecting human beings accurately in a visual surveillance system is crucial for diverse application areasincluding abnormal event detection, human gait characterization, congestion analysis, person identification, genderclassification and fall detection for elderly people. The first step of the detection process is to detect an object which isin motion. Object detection could be performed using background subtraction, optical flow and spatio-temporalfiltering techniques. Once detected, a moving object could be classified as a human being using shape-based,texture-based or motion-based features. A comprehensive review with comparisons on available techniques fordetecting human beings in surveillance videos is presented in this paper. The characteristics of few benchmarkdatasets as well as the future research directions on human detection have also been discussed.