Introduction
With the proliferation of surveillance sensors, there has been an exponential increase in the data generated by these sensors, which is then made available to surveillance personnel. Typically, surveillance data has been used either at or close to the time it was generated; after which it is usually archived, and forgotten (unless retrieved for forensic analysis). With an increase in the number and type of sensors, there comes a threshold after which surveillance personnel will not be in a position to process any additional sensor data in-situ and in real or near-real time; leaving major chunks of data unanalyzed and thus having served no purpose.
How does one automate the analysis of sensor data, in a manner that allows such analyses to scale as more sensor data pours in, and that allows the information contained in the data to be extracted in time? This brief looks at a methodology for implementing an Artificial Intelligence (AI) process for analyzing surveillance sensor data.
Analysing Surveillance Data
As mentioned earlier, traditionally, analyzing surveillance data required humans to sift through the data (visual, video, text, audio) and pick up patterns or pick out outliers in the data stream. Given the vast amount of data generated these days, such a process is no longer practical.
The first steps taken to address this imbalance – between data pouring in and humans viewing, analysing, and acting on the information extracted from the data – were through enabling a level of visual intelligence in surveillance cameras, i.e. video analytics. However, given the nature of threats today and the quantum of data, video analytics alone will not solve the surveillance data deluge problem. What is required is the application of Artificial Intelligence (AI) in analyzing surveillance sensor data.
 |
| System Architecture for Artificial Intelligence in Surveillance |
There are three components to a system architecture of a surveillance system utilizing artificial intelligence:
- Analytics
Video analytics is the best known of sensor analytics, and has been around for some time now; promised a lot, was underwhelming in delivery; but is finally coming of age. Today video analytics is available both on the edge device (camera), working in real-time, and as a back-end service that delivers near real-time response. An August 2009 Tech. Brief article – Video Analytics: The What’s and the How’s – goes into more detail on the subject matter.Analytics engines need to constantly evolve, and waiting for half-yearly or once-a-year software updates to the analytics engine is not the pace of evolution surveillance users are looking at. Instead, the analytics engine is updated on an on-going basis, as the surveillance system uncovers new threat patterns or identifies symptoms of imminent threats. This is done through a combination of data mining and employing artificial intelligence. - Data Mining
Data mining is a computing technique for discerning patterns from data: patterns that are not obvious to a human viewer. Such patterns are teased out using statistical techniques such as regression analysis and Bayesian analysis. Data mining has traditionally been used in scientific research and in the financial sector, but is now spreading to areas such as marketing, retail, fraud detection, and surveillance. The technique consists of a pre-processing activity, when the data is collected, cleaned, and then translated into feature vectors. Following the pre-processing activity, the actual data mining activity consists of four tasks: clustering, classification, regression, and association rule-learning.In the proposed system architecture, all sensor feeds are routed in parallel to the data-warehouse, where a data mining server then mines the constantly updated data for fresh patterns. The following link provides a summary of the five best free and open-source data mining software, for those users interested in further exploring building a data mining application for their surveillance data-set: http://www.junauza.com/2010/11/free-data-mining-software.html. - AI Engine
AI is the branch of computing dedicated to creating intelligence, as defined (in degrees of complexity) by behaviour of living systems, in machines. AI is a term widely known, though more misunderstood than well understood, thanks to its (mis)appropriation in fiction, movies, and songs, in pop culture. Given the highly technical nature of the field and its vastness, it will not be possible to summarise the key developments in AI, here.AI plays a part in the system architecture of the artificial intelligence surveillance system by refining patterns thrown up by the data mining server. A data mining exercise can through up several patterns, not all of which will be valid or not all of which may be practical to apply. The AI engine (typically a neural network) picks up the patterns, and through a process of trial and error, involving inputs from humans and additional field information. The engine then selects the patterns that it thinks are valid, and passes them on to the Command & Control Server, for further action: updating the analytics engine and preparing an SOP for surveillance personnel when faced with such a pattern. There are several open-source AI software available on the Web, for those interested in building a prototype AI engine.
Conclusion
There is no COTS artificial intelligence surveillance system solution available in the market, that will meet the exact requirements of a user. Each requirement will demand a customized solution, and that is the hurdle standing in the way of the widespread deployment of such systems. For users interested in taking the first steps towards building an artificial intelligence surveillance system, implementing a video analytics solution may be a good starting point.
Mistral Solutions’ C4ISR solutions and its MC3S (Mobile Command, Control, & Communications System) solution can integrate third-party applications and systems, and allow customers to build bespoke surveillance systems.
