Maintaining good situational awareness is enormously important for emergency services personnel in the field and for control room operators. Voice, messaging and GPS-based location data have traditionally been the key aids. But new technology is beginning to enhance situational awareness in ways that were unthinkable before now.
Video analytics is the most obvious change, but local sensors, virtual reality (VR), augmented reality (AR), artificial intelligence (AI), machine learning and predictive policing are also starting to contribute. These can provide a much wider and more accurate real-time common operating picture across the response team, enabling commanders to deploy their resources faster, more efficiently and with greater precision.
Ken Rehbehn, founder and principal analyst at CritComm Insights, notes: “We are still in the very early stages of leveraging mobile broadband capabilities in the hands of emergency services. So much of it is an extension of the desktop. Today’s capability provides ways of entering information into electronic forms, or it uses computer aided dispatch (CAD)-based solutions. It solves a big problem by making more information available to the field force, but it is far short of delivering a broad graphic picture of the incident status. Situational awareness is all about understanding what is going on at the scene of an incident. However, the good news is that the foundation is in place.”
Drones are one obvious technology that can provide a ‘bigger picture’ view. Rehbehn cites New York City Fire Department’s use of a tethered drone placed strategically around an active fireground, which can be left unattended to transmit visual images and thermal signatures. “In the future we will see that extend to beyond visual line-of-sight drone capability,” adds Rehbehn. “There have been efforts in Germany to take advantage of this capability using LTE as a way to command the drone vehicles from a control room.”
Rehbehn also foresees the possibility of body-worn cameras transmitting over the LTE network in real time. “The latest Axon Body 3 bodycam is designed to do this. What it points to is the potential of the control room or incident commanders having a view from multiple perspectives.”
Even more radical is the deployment of tactical throwable, 360-degree cameras such as that developed by Buffalo-based Bounce Imaging. “The product is shaped like a ball and fitted with multiple cameras, which can be thrown into a hazardous area such as a collapsed building or hostage situation and flow 360-degree visuals,” says Rehbehn.
Video analytics
Familiar video analytics applications that aid situational awareness include facial-recognition technology, and the ability to spot unusual events or behaviour.
AI is promising further extensions to these capabilities, as Tunde Williams, head of marketing, EMEA at Motorola Solutions, explains. “The police might be interested in a particular vehicle seen in a carpark, but the CCTV camera does not provide a clear front view of the number plate. Using deskewing video technology, you can flatten out the angular image to reveal the number plate.”
Another technique is deblurring, which allows lo-res or blurred images to be sharpened up using AI techniques. Williams also points to the cloud as way to support new apps and database queries.
VR/AR
At CCW 2018, Airbus demonstrated a VR scenario where a firefighter equipped with a 360-degree camera in a metro station streams images back to a supervisor in the command centre, who could be located miles from the scene. The supervisor wears VR goggles, enabling him to ‘step in’ virtually.
“The concept enables you to base your experts anywhere,” says Rahim Zaknoun, head of innovation and developer ecosystem at Secure Land Communications, Airbus. “A supervisor or an expert can guide the field agent. The field agent may be looking one way, but the expert has a 360-degree view and can warn him if there is someone behind him, and so on.”
Rehbehn cites a NIST-funded trial of AR/AI in the USA carried out by the Cosumnes Fire Department in California in conjunction with NASA’s Jet Propulsion Laboratory and others including Qwake Technologies. At the heart of the trial is the Assistant for Understanding Data through Reasoning, Extraction and sYnthesis (AUDREY).
The idea is to upload masses of 360-degree video and sensor data to AUDREY, including temperature readings, thermal imagery, and so on. AUDREY will provide firefighters with real-time data and ‘eyes’ that can see in smoke- and flame-filled environments. Its AI and machine learning software will also have built up a memory bank of previous fires and test burns, which will enable it to guide firefighters by predicting how the fire is likely to develop and warn them if the situation is becoming too dangerous.
Qwake provided its C-Thru AR vision system, which is mounted in the firefighter’s mask and wirelessly connected to commanders. It provides a heads-up display combining thermal imagery and AR to enhance performance in hazardous environments and zero-visibility conditions.
Hybrid solutions
When it comes to location-based data, Jon Cossins, product manager for applications portfolio at Sepura, says: “The location information may come from TETRA, broadband or other sources but it is presented to, and controlled in, a standardised manner by the applications using the data. This greatly improves the situational awareness in the control room regardless of the technology carried by the end-user.”
Sepura’s AppSPACE solution allows users to add apps to their TETRA radios, including downloading them in the field. “When linking the AppSPACE radio apps with backend applications, those backend applications can also be used with LTE/broadband devices,” explains Peter Hudson, head of TETRA products development at Sepura.
Sepura has developed a secure picture messaging application that allows images to be sent with minimal data use, so that both narrowband and broadband devices can receive them at the touch of a button. “Of course, there are a lot of broadband apps that do the same thing,” concedes Cossins, “but not everyone has a broadband device.”
A number of two-way radio manufacturers are using built-in Wi-Fi and Bluetooth to connect with local devices and sensors such as Tasers, firearm holsters, smart protective vests, bioharnesses or a firefighter’s breathing apparatus. Alerts can be triggered if the Taser is drawn, the vest stabbed or oxygen levels in the tank drop to 20 per cent.
Airbus advocates a similar hybrid approach to mission-critical applications. Zaknoun says: “We are developing solutions to help customers, not to replace TETRA/Tetrapol, but to extend operations to LTE and using hybrid networks. For example, we have our Tactilon Dabat hybrid TETRA/LTE device and our Tactilon Agnet solution for smartphones.”
An example of this new capability might be stewards having to evacuate a stadium where the electricity has gone. Staff equipped with the floorplan and exit routes uploaded to the Dabat just need to point the radio and, using indoor geolocation and AR, it will indicate the way to the right exits.
Voice enhancements
It is worth stressing that audio remains a vital application for public safety. Rehbehn notes that both Motorola and Harris provide noise-cancelling microphone capabilities that can be integrated into a firefighter’s face mask.
Voice commands and natural language processing also provide enhancements to traditional audio applications, as Motorola’s Williams outlines. “Police officers need to keep their eye on the subject or incident and not get distracted. They need to be eyes-up, hands-free. So, if they can control their device using voice, that is potentially a game-changer.”
He adds this can be extended to using voice commands over the radio to cognitive computers (bots) to enable database queries. “Police officers spend a lot of time running queries, criminal records searches and so on. The solution uses a combination of natural language processing and AI to run those queries with a voice command. This has significant implications for increasing productivity, as you no longer have to rely on a human dispatcher carrying out those queries on your behalf and relaying information over the radio system. Using voice commands also increases safety, as the officer does not have to look down and enter the information or query,” points out Williams.
Control room trends
One of the biggest trends in control rooms is that companies are now offering a much more integrated approach, according to Alex Richardson, senior research analyst in the Critical Communications Division at IHS Markit.
“They are not marketing CAD and risk management software specifically. Instead they are approaching it from a higher level,” he says. “For example, Motorola Solutions now markets its CommandCentral Aware solution, which is an integration software solution with access to incident details, CAD, records intelligence, geospatial data, real-time alerts from multiple sources and with analytics capabilities built in.
“The CAD system pulls it all together and tells you what is happening. This improves situational awareness at the control centre and helps provide a bigger picture of what is going on. What they are trying to do is get all the information in front of the operator on one screen.”
He observes that in the past control rooms were entirely reactive and dominated by voice. “But now you can be proactive and it is all about preventative policing by having the officers positioned in potential trouble spots or nearby.”
Mark Pearson, key account manager for London Metropolitan Police at control room solutions provider Frequentis, notes that the use of smartphones and social media is one way law enforcement can do ‘more with less’. “These tools can provide frontline officers with real-time access to vast amounts of data, ensuring better intelligence and situational awareness.
“The Frequentis LifeX3020 is an example of a solution that can handle multimedia information that comes in, in any form, and feed it out to the operator as a single communication stack. By adding smart geolocation there is an additional and valuable information layer, for improved contact handling and resource allocation.”
Pearson adds that artificial/augmented intelligence can help to consolidate this information and make an ‘intelligent’ presentation of the overall situation to control room personnel, to aid more efficient decision-making. “Once live images or videos from the public can be analysed in real time, the potential is endless,” he says.
Airbus’s Zaknoun argues that one of AI’s main contributions will be improving workflows. “The AI analyses the intelligence, pulls in lots of historical data and reviews what resources are available. It then makes suggestions to the commander about how to allocate the available people resources to the tasks. It helps the commander decide where to send his police officers and which ones should do what.”
Harnessing social media to aid situational awareness can provoke mixed reactions. Richardson reports being told on visiting a large control centre in Houston, Texas that they were not interested in using social media as an intelligence-gathering tool. “But the Metropolitan Police in London love using [it] to help predict where a large gathering may happen.”
Airbus offers its Fortion MediaMining solution, an AI tool that enables security and police officers to crawl the internet, social media and the darknet to figure out and predict what could happen the next day. It has speech-to-text and translation functions, which helps police prepare better for potential events.
Rehbehn agrees that there is a benefit to be had from monitoring the sentiment of society as expressed in social media, but adds: “The challenge of absorbing this volume of information and making rational operational decisions is large. There are efforts to take advantage of AI machine learning to point to the more important streams that are appearing.”
One company that offers this kind of intelligence gathering and analysis is the rather secretive Palo Alto-based Palantir Technologies, which claims it ‘integrates structured and unstructured data, provides search and discovery capabilities, knowledge management, and secure collaboration’. It is apparently used by US government counter-terrorism analysts and some police departments.
Crowdsourced applications can also help provide a faster response to an incident. The FirstNet-approved PulsePoint app enables CPR-trained bystanders to go to the assistance of a nearby victim if they are in a public place. The app also directs potential rescuers to the exact location of the closest defibrillator. As nearly 1,000 people a day in the USA die of sudden cardiac arrest and there is little chance of successful resuscitation after 10 minutes, this kind of app can really save lives and help out hard-pressed paramedic crews.
Challenges
“New technology and data analytics can play a big role in enhancing situational awareness, predictive policing and helping improve the efficiency of the emergency services. We are now moving from where we are today to this more advanced way of working,” says Airbus’s Zaknoun.
But while there is good reason to be optimistic, Sepura’s Hudson adds a word of caution: “There may be lots of new things coming in the future, but the biggest challenge is to get the public safety end-users to mobilise and use the apps and solutions that are already available and start taking the gains as early as possible.”
Williams points to another challenge. “While this technology is available, police forces are very procedural in the way they work. You cannot just throw technology at them and expect a transformation. There needs to be cultural change and adoption of new ways of working. It is not just a technology issue. It is a human resources and change management challenge as well.”
