Aerial surveillance can reach backwards in time, by the expedient of indiscriminately recording everything that is going on in a particular neighborhood, and then looking for useful patterns in the resulting footage. This technique, called wide-area motion imagery (Wami), has been around since 2006. But improvements in both the recording equipment used and the means by which the images are analysed are making it more and more valuable.

Wami was first employed by American forces in Iraq to track down those placing roadside bombs. When such a bomb went off, it was possible to run the relevant footage in reverse and trace the events that led up to the explosion. That often allowed the bombers to be identified and dealt with…Wami began with an aircraft-borne system called Constant Hawk, which was developed by Lawrence Livermore National Laboratory, in California. Constant Hawk’s success in Iraq begat more powerful versions. Gorgon Stare, carried by drone, was designed by the armed forces themselves…

But there is a problem. Explosions are easy to see. For many tasks, however, an awful lot of staring at screens looking for things that are out-of-the-ordinary is involved. People are bad at this…So AI is here to help…. Chips called graphic-processing units, borrowed from the video-game industry, are helping. So is machine learning, the basis of much modern artificial intelligence. .

l3Harris, a company in Florida, sells Wami sensors for use as automatic sentries. Their software monitors the coming and going of vehicles and pedestrians into and out of so-called watch boxes. These are protected areas surrounded by virtual trip wires, the triggering of which will cause a vehicle or individual of interest to be tracked…This approach can detect immediate threats. It can also, working over a longer period, carry out “pattern of life” analysis by building up a picture of what normal daily traffic looks like in an area. That permits the identification of anomalies which might signal hostile agents whose movements would otherwise be masked by the hurly-burly around them.

The sensors themselves are getting better, too….The latest version includes a so-called hyperspectral sensor, which sees simultaneously across many different wavelengths, including infrared and ultraviolet. It is thus able to distinguish things which the naked eye cannot, such as the difference between camouflage and vegetation. This approach’s real power, however, lies in software which automatically passes data between sensors…Future multi-sensor pods may include other instruments, such as signals-intelligence receivers. These are bits of equipment which can detect radio-frequency communicators like mobile phones and walkie-talkies, enabling particular devices to be identified and located. That would permit the individual carrying the phone, and also those he or she came into contact with, to be tracked and photographed. 

So far, the costs and complexity of Wami have kept it as a predominantly military technology. But that is starting to change. Smaller and more affordable versions are now within the reach of police, fire services and other non-military users…The most famous examples were in Baltimore, where the local cops experimented with the idea twice—first in 2016 and then in 2020. The second time around they made the mistake of monitoring a political protest as well as looking for crimes such as vehicle theft. 

Excerpts from Aerial Surveillance: The Spies in the Sky that See Backwards in Time, Economist, May 7, 2022