Using PTZ camera as an alarm sensor

Securify AB, Richard Ankarman

Field trial - AI video analysis for perimeter protection


2021.04.09

From January to March 2021, Securify evaluated U-alarm, a software for perimeter protection developed by Ultinous. 

U-alarm is based on one of Ultinous’s in-house developed datasets and AI video analysis and is a further development of the previous software version named U-vap. The data set for U-alarm has been trained with just over 8 million images to classify people in outdoor environments. 

Multi-level system: 

  • Level 1. Radar for detection and tracking of movements

  • Level 2. PTZ camera for image verification of detected movements

  • Level 3. AI video analysis for object classification

The field test began in early January -21 when Securify gained access to a BETA version of the U-alarm. The objective was to test U-alarm in as realistic and challenging circumstances as possible. Not the standard setup used by many to showcase their products, using their back yard or other close perimeters as a scene.
Instead, we decided to perform video analysis on a video stream from a moving PTZ camera, controlled by a radar system. No presets were used, the PTZ camera was dynamically controlled by the radar system to follow detections qualified by the radar system. This meant that the conditions for U-alarm were the most difficult imaginable, a significant difference from analyzing video from fixed surveillance cameras with only a 40-60 meters range. 

The test system was installed in a demanding environment and an extensive area that would be monitored with a lot of activity, rail traffic, service staff, deer, rabbits, buildings, fences, trees, a lot of vegetation, and uneven lighting. The radar system's detection area covered approx. 7 hectares while the alarm zones, mainly covered areas within the fence line, approx. 3 hectares.

During a randomly selected measurement period at the end of January, the radar system qualified a total of 699 detections during 72 hours. So, during this period, the radar system activated the PTZ camera to follow qualified tracks, a total of 699 times. This in turn meant that the Ultinous U-alarm analyzed video from 699 unique events. Of these 699 events, U-alarms classified 65 events as human. Of these 65 events, we registered 5 fails. 

All classifications took place during night hours and with relatively poor lighting. Also, worth noticing, all classifications were done at a distance from 150 to 250 meters from the position of the camera. Some classifications that stood out from the crowd:

  • A train driver inside an arriving train at night was successfully classified

  • Train technician who, for some reason, decided to ride at the front of a moving train

  • A person outside the fence line at a distance of 390 meters (this one during daytime)

Briefly about the system:

Video analysis: Ultinous, U-alarm ver 1.0.1

Camera: Axis, Q6215-E (built-in IR with up to 300 meters range)

Radar sensors: SpotterRF, model C40-EXT, CK20

Radar server: SpotterRF, NetworkedIO

VMS: Milestone Corporate

About U-alarm: 

U-alarm belongs to the software category 'AI video analysis' which with the help of machine learning classify different objects in real-time. U-alarm is in this version trained with more than 8 million images to recognize and classify humans, walking, crawling, well in all kinds of postures. Unlike previous generations of video analysis (more like advanced motion detection), U-alarm does not require any perspective settings and other time-consuming configurations, nor does it require any adjustments related to seasonal changes. Bottom line is, U-alarm is both scalable and reliable and should be a good fit for clients with high-level expectations regarding false alarm rate (FAR) and probability of detection (POD). 

This technology is relatively processor-intensive and for this reason, the analysis takes place on a dedicated server with an Nvidia GPU with 256 CUDA processors. An advantage of server-based analysis is that it is camera-independent and analysis can take place regardless of which camera manufacturer and camera models the customer prefers. The analysis takes place on a separate video stream with lower resolution and frame rate; 800 x 450 px and 2 fps. In the current version, U-alarm has support for the analysis of day & night cameras, both fixed and PTZ cameras. Ultinous plans to introduce support also for the analysis of thermal imaging cameras. 


Features

Machine learning
Serverbased analysis
Analysis on PTZ

Advantage

Recognizes objects
Camera-independent
Covers large area

Features

Non-seasonal dependent
Analysis using existing cameras
Cost-effective surveillance

Conclusions: 

The results from the field test show that the combination of radar, PTZ camera, and AI video system together with good detection ability. In addition, a perimeter system with excellent ability to reduce the number of false alarms. 

The limitation we see in video analysis and the camera's ability to match the radar's surface and distance coverage is the camera's ability to deliver sufficiently good image quality for the video analysis. It is firstly the stage lighting, secondly the camera's capacity to set the right focus, and thirdly the weather that reduces the image quality. 

At distances below 300 meters, however, the restrictions are small. We see a realistic possibility to, with this combination, reduce false alarms by up to 99%.