ICONS: Incident Recognition for Surveillance and Security

ICONS is a DTI/EPSRC LINK project that links Queen Mary Vision Lab with several industry partners including Safehouse technologies, BAA plc, BT and Heritage Protection Ltd. 

Human activities captured in video footages are characterised by the spatio-temporal structure of their motion pattern. These structures are intrinsically probabilistic and often ambiguous. In general, they can be treated as temporal trajectories in a high-dimensional feature space representing correlated visual measurements. For example, the spatio-temporal structure of a simple behaviour such as walking towards a telephone box in a public place can be represented by the trajectory of an observation vector given by the mean position and displacement of the human body. The problem becomes hard when human (and other motion object) activities captured in crowded public space need to be interpreted correctly. The aim of the project is to advance techniques for:

Zero Motion Detection

The task is to distinguish background from foreground, even though the scene appearance is changing due to varying illumination and transitory movements, such as human activity or a swaying tree.

By learning pixel variations and adapting to slow changes over time, sudden changes in the scene can be detected. In the movie example below, a box is placed in the room. Temporary changes such as the person walking are ignored, but after the box has been present for a while it is detected as a foreground object.


MPEG demo (285 Kb)

Behaviour Recognition

In dynamic scenes, the task is to separate normal from abnormal behaviour. To do this, pixel phase variations are learned over time for "normal" behaviours. When a variation is observed that does not fit the model of allowable variations, it is flagged as an abnormal occurrence.

In the movie example below, the system has been trained on sequences of the subject entering the room and walking up and down. During testing, the same behaviour is ignored because it is "normal". However when the subject stops moving or jumps, that behaviour is detected as abnormal.


MPEG demo (1.3 Mb)

Current Work

A major component of the project is the detection of changes in the background of scenes in which there is a lot of activity.

Fig 1. Before processing
Fig 2. The result after processing

The first photo (Fig. 1) is a still shot from a video sequence taken by a CCTV camera looking at the forecourt of terminal 2 at Heathrow airport. The image in the camera is constantly changing due to the wind moving trees, clouds moving across the sun and camera shake as well as the constantly moving traffic. The second photo (Fig. 2) is the same shot after processing (in real time) using the algorithms being developed. We have detected that two vehicles have been parked in unexpected places. Notice that the standard background has been successfully eliminated.

Publications

  1. S. Gong and T. Xiang. Recognition of group activities using a dynamic probabilistic network. In Proc. IEEE International Conference on Computer Vision, Nice, France, October 2003.
  2. T. Xiang, S. Gong and D. Parkinson. Outdoor activity recognition using multi-linked temporal processes. In Proc. British Machine Vision Conference, Norwich, UK, September 2003.
  3. A. Graves and S. Gong. Spotting scene change for indexing surveillance video. In Proc. British Machine Vision Conference, Norwich, UK, September 2003.
  4. T. Xiang and S. Gong. Discovering Bayesian causality among visual events in a complex outdoor scene. In Proc. IEEE International Conference on Advanced Video- and Signal-based Surveillance, Miami, USA, July 2003.
  5. S. Gong and T. Xiang. Scene event recognition without tracking. Special issue on visual surveillance of dynamic scenes, Acta Automatica Sinica (Chinese Journal of Automation), Chinese Academy of Sciences, Vol. 29, No. 3, pages 321-331, May 2003.
  6. J. Ng and S. Gong. On the binding mechanism of synchronised visual events. In Proc. IEEE Workshop on Motion and Video Computing, Orlando, USA, December 2002.
  7. S. Gong and H. Buxton. Editorial: Understanding visual behaviour. Image and Vision Computing, Vol. 20, No. 12, pages 825-826, October 2002.
  8. S. Gong, J. Ng and J. Sherrah. On the semantics of visual behaviour, structured events and trajectories of human action. Image and Vision Computing, Vol. 20, No. 12, pages 873-888, October 2002.
  9. T. Xiang, S. Gong and D. Parkinson. Autonomous visual events detection and classification without explicit object-centred segmentation and tracking. In Proc. British Machine Vision Conference, Cardiff, September 2002.
  10. J. Ng and S. Gong. Learning intrinsic video content using Levenshtein distance in graph partitioning. In Proc. European Conference on Computer Vision, Part-IV, pages 670-684, Copenhagen, Denmark, May 2002.
  11. J. Ng and S. Gong. Learning pixel-wise signal energy for understanding semantics. In Proc. British Machine Vision Conference, Manchester, UK, September 2001.
  12. J. Sherrah and S. Gong. Automated detection of localised visual events over varying temporal scales. In Proc. European Workshop on Advanced Video-based Surveillance Systems, Kingston, UK, September 2001.
  13. J. Sherrah and S. Gong. Continuous global evidence-based Bayesian Modality Fusion for simultaneous tracking of multiple objects. In Proc. IEEE International Conference on Computer Vision, Vancouver, Canada, 2001.

Partners

The ICONS project is funded and managed by DTI and EPSRC, and involves the following academic and industrial collaborators:

Links