This article originally appeared in the March 2021 issue of Security Business magazine. When sharing, don’t forget to mention @SecBusinessMag on Twitter and Security Business magazine on LinkedIn.
Artificial intelligence (AI) performs services, recognizes patterns, learns object and situation relationships, and makes decisions. There is a paradigm shift provided by new, proven technologies, such as 5G, thermally efficient, low-power AI chipsets and AI itself, carrying the global surveillance and IoT sensor markets forward – seemingly without a threshold.
When we ask a voice assistant to search for our favorite law enforcement show, we often get our intended choice first. Vehicle manufacturers are developing advanced driver-assistant systems (ADAS) like Automatic Emergency Braking (AEB) with the goal of creating processes like collision avoidance with both pedestrian and cyclist. The security industry is fortunately tasked with less critical tasks like recognizing objects, a potential threat, and an appropriate deterrence response.
It is also fortunate that our industry is leveraging the availability of more thermally efficient AI chipsets and Edge AI sensors like LiDAR and thermal imaging – all of which are decreasing in price. AI chipsets with improved thermal management, such as the Ambarella CV5 introduced at CES 2021, supports four independent 4K video streams, AI capabilities, low power consumption 5nm node. This would be the answer to an underpowered IP camera processing vehicle Automatic Number Plate Recognition (ANPR) in multiple lanes of traffic producing “choppy” video, as streaming and recognition processes compete for resources.
AI Challenges
Deep Neural Networks (DNN) need an enormous amount of data to learn. Security Solutions themselves are not “intelligent;” they leverage deep learning of a situational awareness report (sitrep) and historical data to deliver the most appropriate action. Data from sensors of multiple formats (visible light, infrared, audio, laser) and complex data from environmental, social media, crime datasets are becoming too massive to process by legacy operating procedures. Fortunately, companies like ESRI offer datasets linking crime, location, and time; in fact, its ArcGIS Insights is now available at no cost through the Disaster Response Program (DRP) to analyze the impact of the COVID-19 pandemic.
Privacy and data retention policies do create challenges for some industry AI solutions to improve – for example, a retail loss prevention DNN might need to “review” days of differential video content, or content in scenes with enough variances in people, crowd movement, products on shelves and endcaps, lighting conditions and types of floor surfaces in order to “recognize” the basic behavior of a person planning, executing, and leaving the scene of a shoplift.
Corporate security and first responders can ingest, analyze and predict potential outcomes and share data as we move forward as an industry to allow AI to eventually perform learned, basic tasks and give us back the manpower we need to make critical decisions.
“Ingest” may be a new term to some in the security industry, but it is well-used in markets relying on “big data” or applied data sciences. For your customers to leverage today’s AI solutions – especially with video surveillance – it will be advantageous to start collecting quality video content within a perimeter for higher assurance of quality alarm processing and response.
While human intelligence continues to lead, Edge AI presents security and safety management with a rebate of time for known processes.
Bringing AI to the Edge in Video Surveillance
In general, for IP video cameras, common video analytic features like object recognition, zone detection, number plate recognition are now available as AI algorithms embedded within the camera itself. It may not be clear how firmware updates affect AI, or if the algorithm is updated with the latest factory-trained model, or if it retains algorithm training for the specific use-case.
Solutions that have been developed all use updatable algorithms, and white and blacklists, such as basal temperature/fever screening and visual weapons detection by Athena Security; energy waveform signature of ballistics by EAGL; and drone RF signature/behavior detection by WhiteFox and Aero Defense.
Many “Edge AI” cameras, however, may also require the user to have a Video Management System to update an algorithm. The user should be cautioned to prioritize surveillance investments in Edge AI devices or more specialized weapons/drone detection software, rather than a potentially more isolated VMS that may become a pay-as-you-go service.
Many IP video camera manufacturers have touted AI features. Here are a few of the more recent developments:
Panasonic’s i-Pro Extreme cameras feature built-in AI for motion detection and analysis for precision object categorization, such as the difference between a human and bicycle at distance. It can detect objects that enter an area restricted by object category – such as a pedestrian-only area or region – as well as crossline or loitering detection. Privacy guard redacts video – specifically faces or bodies. Panasonic also provides an SDK for third-party developers to add advanced features like weapons or fall detection.
Hanwha Techwin’s Wisenet7 chipset delivers in-camera AI features that include: AI-based object classification for detected objects, people, vehicles, license plates and faces; Reduction in false positive alerts, for improved monitoring in operations having many cameras; AI-based object tracking with some PTZ cameras; and an auto-tracking function that tracks vehicles and people – an improvement over frame-based tracking.
Milesight delivers edge-based AI over a variety of cameras. Features include: Pre-trained deep learning model and continuously training of algorithms automatically; and three groups of algorithms, including Video Content Analysis (VCA), such as for human and vehicle detection; real-time people counting based on AI algorithms, with statistic reports for analysis; and AI Face Detection.
The use of Edge AI cameras is significant because they can serve as a “model” for simplified yet effective AI training workflow.
Industrial cameras can capture number plates at high speed and close distance for safety and tracking applications like rail transit and vehicle screening. The IDS NXT camera is an example of improved AI Training and integration with third party systems through web apps (no additional coding needed). The system can create training images and upload them, and users can assign labels (ex. "good" or "bad") so the AI can learn. This, in turn, starts automatic training of the neural network, eventually leading to full deployment.
Other Security Areas Impacted by Edge AI
As reported at CES last year, the use of LiDAR in small 3D cameras can deliver a wireframe “image” that preserves privacy, yet identifies a face – even one partially obscured by a mask or PPE. Furthering on this trend, Intel’s RealSense ID, introduced at CES 2021, combines active depth with a neural network, a dedicated system-on-chip and embedded secure element to encrypt and process user data. Using deep learning, it adapts to users over time as they change physical features, such as facial hair and glasses or appear in different lighting conditions.
It remains to be seen if complex entry screening tasks will eventually be included in the average security solution; however, there are already AI-based solutions in the security/safety market that perform facial recognition and/or face matching where users are wearing masks, hats and the process is consistent across gender, age, and ethnicity. They can also perform elevated temperature screening, leveraging appropriate basal temperature measurement locations and multi-spectral imaging (usually visible light plus thermal imaging); as well as consistent on person, concealed and non-concealed weapons detection, front, side or rear-worn weapon.
Critical infrastructures can be dangerous places, and Edge AI sensors can play a life-saving role. High voltage wires, combustible chemicals, hazardous waste, and other threats can be detected early with thermal imaging and AI training. LiDAR sensors with an AI process can trigger alerts when personnel enter dangerous areas such as tunnels, railway tracks and bridges.
Steve Surfaro is Chairman of the Public Safety Working Group for the Security Industry Association (SIA) and has more than 30 years of security industry experience. He is a subject matter expert in smart cities and buildings, cybersecurity, forensic video, data science, command center design and first responder technologies. Follow him on Twitter, @stevesurf.
