Vision and Interaction
Introduction
The Vision & Interaction Group combines two subgroups: the Vision group (led by Professor Sean Gong) and the Interaction, media and communication (led by Dr Pat Healey). The vision work in extracting models from image sequences and live video is world-leading and has been applied by major organisations in a range of systems for security and crime prevention. Other vision work has been adopted as part of the international coding standard JPEG, while our work on biometric authentication has resulted in a patent licensed to a US organisation.
The work in interaction is organised around novel methods of communication together with systems, tools and methodologies (e.g. tools for dialogue analysis). The interaction work has been reported in New Scientist and has had an international impact on media and communication with spin-off applications for mobile phones, used in live performance and ongoing collaborations with artists.
International quality researchers in this group include Dr Lourdes Agapito, Dr Nick Bryan-Kinns, Professor Sean Gong, Dr Pengwei Hao, Dr Pat Healey, Professor Peter McOwan, Dr Fabrizio Smeraldi, Dr Tony Stockman and Dr Tony Xiang. Highlights of our research include:
Face and behaviour recognition from video: Professor Sean Gong and Dr Tony Xiang pioneered the extraction of dynamic face models, adaptive colour based object tracking, and deriving object behaviour models from image sequences and live video. This work has been widely applied to vehicle and people detection, object tracking, counting and recognition in public space CCTV, human gesture recognition in visually mediated interaction and abnormal behaviour recognition in visual surveillance. A significant focus of this work is crime prevention - real-time surveillance and biometrics. The algorithms have been further developed and deployed by Safehouse Technologies Ltd in operational systems in Australia (security services and museums), USA (Smiths Detection), Hong Kong (government), and the UK (Home Office). QinetiQ and Ultra Electronics, working with DSTL, have adopted algorithms from this work for developing a next generation MOD operational system. A demonstrator was also developed with BAA T5 for civil airport applications. Intel, Kodak, Microsoft, Samsung and Sun Microsystems have adopted/reverse-engineered these algorithms for their commercial applications.
Non-rigid motion : Dr Lourdes Agapito's work on 3D reconstruction of deformable structure with applications to modelling human facial structures has led to two papers being selected amongst the top five of their conferences (and invited for publication in special issues of journals) and a PhD thesis being short-listed for the 2006 British Machine Vision Association best thesis in Computer Vision prize.
Computational neurobiology: Professor Peter McOwan is concerned with brain function at the algorithmic level. The flow based motion perception algorithm solves a long-standing issue in visual perception, showing that luminance and contrast-defined motions can be detected accurately with a single mechanism, and reducing the complexity of the algorithms that have to be implemented in the brain. This algorithm has been applied to facial expression recognition and facial puppetry.
Human Communication: Dr Pat Healey and Dr Nick Bryan-Kinns' work is based on using language and drawing-based communication tasks. They have identified generic, modality independent, interaction mechanisms that drive the development of human symbol systems. These results have ramifications for theories of human language evolution, cognitive development and tools to support richer forms of human interaction. The groundbreaking chat tool developed by Dr Pat Healey provides the first fine-grained, word level, experimental control of human-human dialogue. This approach (which has demonstrated the limitations of the current leading theory of human dialogue co-ordination) is being adopted by research groups in Gothenburg, Stoneybrook and Muenster. Their work on mutual engagement in musical improvisation has led to the development of novel interactive interfaces for collaborative composition of music. This has been exploited through the development of web-based 'jamming' environments.
Matrix Methods: Factorisations are widely employed to optimise operations involving matrix multiplication. Dr Pengwei Hao's PLUS matrix factorisation has been applied in image coding and other areas, and his technique of integer transform for lossless coding has been included in the international standard for image coding, JPEG 2000.
Industrial collaboration
The Vision and Interaction group has wide-ranging industrial collaborations. Highlights include:
Partners on research projects led by Professor Sean Gong include: MOD (DSTL and SA/SD), Home Office HOSDB, Petards Vision, Safehouse Technologies, ABM UK, British Airport Authorities, BT Labs, and the Heritage Protection.
Professor Sean Gong has many prominent advisory roles, including to the US Office of Naval Research and many UK Government agencies.
Professor Sean Gong and Dr Pengwei Hao have both been invited visiting scientists at Microsoft Research Asia, and Sean at Samsung (2003).
Professor Peter McOwan's work has been exhibited at the Royal Society and widely reported in the national and international press. His work on biometric authentication resulted in a US patent now licensed to Intellectual Ventures Holding 3 LLC.
The group collaborates with artists in ways that are pushing at the boundaries of the discipline. This has led to exhibits at the National Portrait Gallery and the Institute for Contemporary Arts, and to spin-off applications for mobile phones and in a live concert series in Japan.
Academic collaboration
The group is a founding member of the EPSRC Leonardo network in the area of Culture, Creativity and Interaction Design, developing research at the intersection of human-computer interaction, the arts and humanities, and a Leverhulme international research network "Dialogue Matters" with partners in Edinburgh, Glasgow, Gothenburg, Stanford and Stony Brook. This is also the subject of a major new ESRC-funded collaborative project with Kings and Stanford.
Professor Peter McOwan's work on biologically inspired vision is in collaboration with the Department of Psychology at UCL.
Professor Sean Gong collaborates with numerous international and UK institutions through: a Royal Society International Joint Project on Fusion of Multimodal Biometrics for Non-Intrusive Person Identification in CCTV, EPSRC SERVE (Surveillance, Evaluation, Research,Validation, and Exploitation) Network (19 UK universities); and EPSRC ViTAB (Video-based Threat Assessment and Biometrics) Network (4 UK universities, 4 companies, Home Office PITO and HOSDB, and the International Association of Biometrics).