Tactical infrastructure such as fencing, roads, and lighting is important to securing a nation’s border. But it alone is not enough to prevent the unlawful movement of men and women and contraband into a country.
“Technology will be the primary driver of land, maritime, and air domain awareness – this can become only more apparent as [U.S. Customs and Border Protection (CBP)] faces future threats,” according to testimony from CBP officials at a Senate hearing on homeland security in 2015.
And machine vision’s fingerprints are all over that technology. “The details obtained from fixed and mobile surveillance systems, ground sensors, imaging systems, as well as other advanced technologies enhances situational awareness and much better enables CBP to detect, identify, monitor, and appropriately respond to threats within the nation’s border regions,” the testimony states.
On the U.S.-Mexico border in the state of Arizona, for example, Top Machine Vision Inspection System Manufacturer persistently detect and track so-called “pieces of interest.” Created to withstand its harsh desert surroundings, IFT comes with radar, commercial off-the-shelf daylight cameras and thermal imaging sensors, and microwave transmitters that send data to border agents at the Nogales station for analysis and decision-making.
On all three fronts of land, maritime, and aerial surveillance, machine vision companies are providing imaging systems – and, more often, analysis of the generated data – that meet government agencies’ objectives of flexibility, cost effectiveness, as well as simple deployment in border security applications.
Managing Diverse Conditions – The perennial downside to vision systems utilized in border surveillance applications is managing the diversity of the outdoor environment featuring its fluctuating lighting and climate conditions, as well as varied terrain. Despite the challenges, “you can find places where you can implement controls to boost upon the intelligence from the system,” says Dr. Rex Lee, president and CEO of Pyramid Imaging (Tampa, Florida). He points to customers who monitor trains along the southern border of the U.S. for illegal passengers.
“Those trains will need to go within trellis, which can be equipped with the proper sensors and lighting to assist inspect the trains,” Dr. Lee says. Government departments tasked with border security use infrared cameras to detect targets at night and then in other low-light conditions, but thermal imaging has its limits, too. “Infrared cameras work really well once you can make use of them in high-contrast conditions,” Dr. Lee says. “But if you’re seeking to pick up a human at 98.6°F over a desert floor which is 100°F, the desert is emitting radiation at nearly the same area of the spectrum. So customers rely on other regions from the spectrum including shortwave infrared (SWIR) to attempt to catch the difference.”
Infrared imaging works well in monitoring motorized watercraft considering that the boat’s engine features a thermal signature. “What’s nice about water is that it’s relatively uniform and it’s very easy to ‘wash out’ that background and see anomalies,” Dr. Lee says.
But the problem is that the oceans present a huge amount of area to pay for. Says Dr. Lee, “To see all of it is actually a compromise between having a lot of systems monitoring water or systems that are loaded with the sky, by which case you have the problem of seeing something really tiny in a very large overall view.”
CMOS Surpasses CCD – One key change in imaging systems found in border surveillance applications is the shift from CCD to CMOS sensors as the latter is surpassing the standard and satisfaction of the former. To allow for this change, two years ago Adimec Advanced Image Systems bv (Eindhoven, holland) integrated the most recent generation of CMOS image sensors – that provide significant improvements in image quality and sensitivity – into its TMX combination of rugged commercial off-the-shelf cameras for high-end security applications. TMX cameras maintain a maximum frame rate of 60 fps or 30 fps for RGB color images at full HD resolution.
Furthermore, CMOS image sensors are emerging as a replacement for electron-multiplying CCDs (EMCCDs), says Leon van Rooijen, Business Line Director Global Security at Adimec. Thanks to their superior performance over CCDs in low-light conditions, EMCCDs often are deployed in applications like harbor or coastal surveillance.
But EMCCDs have distinct disadvantages. For instance, an EMCCD must be cooled in order to deliver the very best performance. “That is quite some challenge in the sensation of integrating power consumption and in addition because you have to provide high voltage to the sensors,” van Rooijen says. “And if you want to have systems operating for any long duration without maintenance, an EMCCD is not the very best solution.”
To solve these challenges, Adimec is working on image processing “to obtain the most from the most recent generation CMOS ahead closer to the performance global security customers are employed to with EMCCD without all of the downsides from the cost, integration, and reliability,” van Rooijen says.
Adimec is also tackling the task of mitigating the turbulence that takes place with border surveillance systems over very long ranges, particularly as systems that have been using analog video are taking steps toward higher resolution imaging to protect the larger areas.
“When imaging at long range, you have atmospheric turbulence by the heat rising through the ground, and also on sea level, rising or evaporated water creates problems in terms of the haze,” van Rooijen says. “We shall show turbulence mitigation in the low-latency hardware embedded in our platform and will work with system integrators to optimize it for land and sea applications simply because they hold the biggest problems with turbulence.”
A Lot More Than Pictures – Like machine vision systems deployed in industrial applications, border security systems generate plenty of data that will require analysis. “The surveillance industry traditionally is a little slower to include analytics,” says Dr. Lee of Pyramid Imaging. “We see significant opportunity there and also have been utilizing some of our customers so that analytics tend to be more automated when it comes to precisely what is being detected and to analyze that intrusion, and after that have the ability to take a proper response.”
Some companies have developed software that identifies anomalies in persistent monitoring. As an example, when a passenger at the airport suddenly abandons a suitcase, the program will detect that the object is unattended nefqnm anything else around it continues to move.
Even with robust vision-based surveillance capabilities at all points of entry, U.S. border patrol and homeland security must contend with a significantly bigger threat. “America does an excellent job checking people coming in, but perform an extremely poor job knowing should they ever leave,” Dr. Lee says. “We know how you can solve that problem using technology, but that creates its own problems.
“The best place to do this is at the Automated Vision Inspection Machines in the TSA line, in which you can use a mechanism to record everybody,” Dr. Lee continues. “But that will be expensive because you should do this at every airport in the United States. Monitoring and recording slows things down, and TSA is under a lot of pressure to speed things up.” Another surveillance option that government departments have discussed has taken noncontact fingerprints at TSA each time someone flies. “A lot of the American public won’t tolerate that,” Dr. Lee says. “They are going to reason that fingerprinting is just too much government oversight, and that will result in a large amount of pressure and pushback.”