Video and rich data present massive opportunities and equally significant challenges. A suspicious package can be spotted in real time, a person of interest tracked, a violent incident monitored or even anticipated. After an event, video recordings can be used for prosecutions, investigations, or training, while facial recognition promises to automate screening at airports and similar locations and free up staff for analysis.
One challenge is the sheer volume, the hundreds or even thousands of cameras feeding multiple screens in control rooms and the gigantic quantities of video to be stored. We have had to invent new classifications for data volume – petabytes, exabytes, right up to geopbytes. At the same time, human resources are always limited; how many screens can be watched by one person and for how long? So, can we get silicon to do some of the work?
The technology exists to allow hours of footage to be viewed in minutes to detect key events, while keeping movements at a manageable pace. Rule-based analytics can detect suspicious objects or movements near prohibited areas. But what is on offer now and in the near future? What issues have emerged, be they technical or social?
My process of discovery into this fascinating area has taken me to three companies offering solutions backed by years of experience – Digital Barriers, Accenture and Agent Vi – but there are numerous other suppliers out there.
Analytics and body-worn cameras
Digital Barriers is a company specialising in secure video streaming and analytics. It offers smart software solutions which can run on-site or via the cloud to manage aspects of video capture, analysis and streaming, and has customers in more than 50 countries. The core technologies are resilient live video streaming, live facial recognition, intruder detection using ground sensors and video surveillance cameras and object detection using video. An example is detecting obstructions on railway crossings from fixed cameras.
Richard Revis, Digital Barriers’ director, group product management, talks me through its live-streaming body-worn camera solution. It has developed a number of software applications for video analytics which can be embedded in the camera. One example is facial recognition. The body-worn unit can take still pictures which can be compared to a database to positively identify a person or vehicle; similarly, if a unit is live-streaming, then images could be taken from the stream and compared to a database in real time. Completing the identification process can save up to 20 minutes on the street for both parties. Results are communicated back by the controller or sent directly to the user’s mobile phone. Cameras also work well in moving vehicles, for example in the form of a dash cam, at motorway speeds.
The body-worn camera is linked through 2G, 3G or 4G to the controller. Matching video to the available bandwidth is achieved with a number of techniques. For example, if a facial image is to be sent for identification, then only the face itself is sent. Streamed video can be adapted down to bandwidth as low as 20Kbit/s. Digital Barriers has its own patented TVI codec originally developed for the military which overcomes some of the issues with other protocols which are intended mainly for entertainment. Latency is reduced and lost frames are eliminated. It takes the view that situational awareness is key, so basic information takes precedence over image quality.
The control station software permits hundreds of officers to be managed by one controller, and Digital Barriers provides tools for the analysis of video which support real-time operation. For example, being able to quickly search back and find a face or number plate, if necessary correlating this with other images. Revis says: “It’s about freeing up people by putting them in the right place with the right supporting information. Our customers are bringing us the ideas and problems from which we can develop custom solutions.”
Platforms and artificial intelligence
The platform approach is seen as essential by Accenture’s global managing director – public safety, James Slessor. Platforms will enable different sources of data to be combined to meet the needs of different agencies. “Our focus is to free up the potential of raw data to become usable information in real time. CCTV, body-worn cameras, the internet and sensors are all providing us with inputs that could reduce crime and improve public safety. Another factor is that information may be held by different agencies and we need to find ways to safely share and combine data between them.”
He gives a user example. “Sharing data can help to show up common risks, threats and vulnerabilities and allow joint intervention strategies to be developed, especially to protect more vulnerable groups. Data from police, council, education and health agencies would need to be accessed and used. Resources can then be focused where most needed. This highlights the need for a platform approach to enable sharing, but also the issue of protecting data. Blockchain technology may be used to ensure data integrity.”
Slessor adds: “It needs a scientific approach to analysing the operational needs together with an understanding of where the right data can come from. Our data scientists are working with forces around the world to do just this. The platform is then the glue to make it all work together. Artificial intelligence is also part of the mix, unlocking value in the mountain of data out there to find patterns in video and other data sources to predict where resources will be needed.”
To demonstrate how this works, Slessor says: “In the UK, some police forces are looking at how organised crime groups and gang behaviours can be analysed to identify which are most likely to pull others into criminality.” Looking again at video analytics, he points out that “facial recognition can be enhanced by recognising clothing, backpacks or tattoos. You can’t always see faces.”
He adds: “AI is good at spotting things which may otherwise be missed… in effect seeing the unseeable. Analysts can then be directed to the most interesting areas. AI can be a powerful tool in the virtual world. Chat rooms can be monitored to detect possible predators who will use certain phrases to hook young people in. Gangs may use emojis or other means to communicate over the internet; AI can recognise patterns and alert officers.”
Slessor cites four essential elements in the development of analytics and the use of AI. The first of these is the need for governance over how data is stored, shared, combined and used – “there needs to be an overriding code of ethics in place”. The second is that “the design of any AI system needs to be transparent and auditable; it is highly likely that actions that result from the system will be subject to scrutiny.
“Thirdly, as any system will be designed by humans, there is the possibility of bias. The outputs need to be tracked, recorded and analysed to detect this and correct it if necessary.
“Finally, the impact of the systems on the teams and officers needs to be tracked. People may no longer be needed on highly repetitive tasks; this frees them for higher-value, often more interesting tasks – but concern over AI automation can be a source of stress for some if not proactively addressed.”
Edge or cloud processing?
Zvika Ashani, CTO at Agent Video Intelligence (Agent Vi), says: “Our savVi solution has been in operation for about 10 years now and is an on-premise Windows-based system integrating video feeds from up to 200 cameras per server. It can automatically detect events of interest such as a potential intruder or suspicious object.” SavVi also offers forensic search capabilities, enabling search through large quantities of recorded video to locate and extract events of interest in a few seconds. It provides business intelligence insights regarding traffic volumes and trends, as well as motion patterns.
Ashani adds that it is “in service with public safety agencies and many industrial and public authorities. There are [around] 4,500 sites employing savVi around the world, including colleges, transport systems, critical infrastructure facilities and smart cities.”
In terms of what’s next for analytics, Ashani says: “We have built on savVi to offer our innoVi platform. The big difference is that this can be a cloud-based system. Some processing takes place at the edge or locally, but the cloud gives us the ability to handle large amounts of data and employ powerful analysis tools such as AI. It also gives us the ability to scale for very large deployments.” Ashani notes that some agencies are wary of using the cloud to store and process data, and therefore innoVi will also be offered as an on-premise solution in the future.
Asked if the huge volume of data is a challenge, particularly video, Ashani says: “We are now analysing the video using machine learning. Traditionally a person had to interact with each camera to set up a series of rules for what constitutes an event of interest. With thousands of cameras in the larger deployments, this becomes impractical and only a selection can be programmed in this way. With our solution, and employing machine learning tools, each camera monitors the scene for days or weeks and the system ‘learns’ what normal activity is. In other words, the normal movement of people, day and night lighting, and so on. It can then alert when something unusual happens, such as a group forming at an unusual time or objects left unattended. This structured description of the scene or metadata is several magnitudes smaller than pixel-based video and radically changes how information can be stored. It also provides the layer on which we can build actionable intelligence and can be used to audit particular events.”
Governance and ethical considerations
Professor Monika Buscher is based at Lancaster University looking at ethical issues faced by public agencies in critical communications and disaster risk management, and is the director at the Centre for Mobilities Research. Buscher and her colleagues are developing a resource for ethical innovation in critical comms and disaster risk management at www.isITethical.eu.
On the main ethical issues that need to be considered, she says: “We shouldn’t confuse images and video with reality. The humans at the scene will add context and judgement, which may be essential to understanding what is happening. Secondly, privacy may be an issue; for example, medical staff at an incident may not be comfortable with being on camera. Filming may be distressing to victims or their families, and the faces of innocent passers-by may be captured.
“Principles such as purpose limitation or data minimisation may be at odds with some of the most significant capacities of video data, analytics and artificial intelligence. Non-discrimination is difficult as facial recognition tools [don’t currently work consistently] across different ethnic groups and ages. Finally, images that find their way onto social media can cause problems, as was the case after the Boston bombings when incorrect identification of suspects led to discrimination against individuals not connected with the incident.”
Ethical considerations are not obstacles for innovation, but can drive more innovative design to automatically control the capture and recognition of faces and make it easier for officers to control cameras and data analytics. As trust is increasingly put into such systems, more ambitious, agile and robust innovation is needed to ensure that they are fit for purpose and actively support public agencies in working within legal and ethical limits.
Where does this lead us?
From these discussions, several themes emerge. Building intelligence into the camera itself is an approach taken by Digital Barriers, which has also been providing streaming technologies, and it proposes that these are more suited to public safety than uploading footage at the end of a shift. The company identifies combining data as a further opportunity, either linking to sensors on the officer’s body for safety, or back to databases to provide greater intelligence for a live incident or investigation.
Accenture points to the possibilities in AI and machine learning to greatly increase the extent to which officers and controllers can be freed to focus on the essential elements of their job. Patterns of human behaviour such as gang or internet activity can be tracked and then alerted to officers for action. But this brings with it the need for governance and auditing to ensure successful prosecution and protection for the officers and agencies, and Buscher proposes that ethical considerations need to be built into the design of analytics and AI systems from the start to avoid problems and issues later.
It’s worth noting that the technology is already at the point where it can make a significant difference in real-life applications. For example, a trial that took place in Australia to evaluate the use of image recognition software to identify drivers who were using mobile phones led to the capture of more than 700 motorists.
Should applications be run at the edge or in the cloud? Agent Vi offers both, as some agencies are still unsure whether cloud-based computing can offer them the security that they need.
How far this story will evolve is more likely to be limited by our imagination, rather than technology. Increasing risks and demand coupled with limits on manpower and budgets are creating rich opportunities for innovation; both large corporations and fast-moving SMEs will bring exciting new solutions to the market. We need to watch this closely.
