Robust and efficient foreground analysis in complex surveillance videos

Mixture of Gaussians-based background subtraction (BGS) has been widely used for detecting moving objects in surveillance videos. It is very efficient and can update the background model with slow lighting changes, however, it suffers from a number of limitations in complex surveillance conditions such as quick lighting variations, heavy occlusion, foreground fragments, slow moving or stopped object etc. To address these issues, this paper first focuses on foreground analysis within the mixture of Gaussians BGS framework in long-term scene monitoring to handle (1) quick lighting changes, (2) static objects, (3) foreground fragments, (4) abandoned and removed objects, and (5) camera view changes. Then, we propose a framework with interactive mechanisms between BGS and processing from different high levels (i.e. region, frame, and tracking) to improve the accuracy of moving object detection and tracking to handle (1) objects that stop for a significant period of time and (2) slow-moving objects. The robustness and efficiency of the proposed mechanism are tested in IBM Smart Surveillance Solution on a variety of sequences, including standard datasets. The proposed method is very efficient and handles ten video streams in real-time on a 2GB Pentium IV machine with MMX optimization.