Tag Archives: Surveillance & Reconnaissance (ISR)

The Chinese Balloon and the American Psyche

The U.S. blacklisted six Chinese companies on February 10, 2023  that it said were involved in Beijing’s surveillance-balloon program, in a move taken in retaliation for the suspected spy balloon that traversed the U.S. The companies blacklisted are Beijing Nanjiang Aerospace Technology; China Electronics Technology Group Corporation 48th Research Institute; Dongguan Lingkong Remote Sensing Technology; Eagles Men Aviation Science & Technology Group; Guangzhou Tian-Hai-Xiang Aviation Technology; and Shanxi Eagles Men Aviation Science & Technology Group.

The Commerce Department’s Bureau of Industry and Security said the Chinese entities were added over their support for the People’s Liberation Army’s aerospace programs, including airships and balloons and related materials and components.  “The PLA is utilizing High Altitude Balloons (HAB) for intelligence and reconnaissance activities,” it said….  While many national-security analysts have been sounding the alarm about China’s surveillance practices in recent years, the balloon offered the American public a visible picture of the Chinese threat as it crossed much of the nation.

The newly formed House Select Committee on the Strategic Competition Between the U.S. and the Chinese Communist Party is likely to press for tougher U.S. measures to slow China’s advance, said Emily Benson, a senior fellow at the Center for Strategic and International Studies. “This is a really concrete example of an issue that Washington has so far not succeeded in penetrating the American public’s mind,” Ms. Benson said. “This could be kind of a pivotal moment for the American psyche to really start to realize that China is important and that this is a policy area they should be tuning in to.”

Excerpts from Ian Talley and Vivian Salama, U.S. Blacklists Chinese Companies It Links to Balloon Program, WSJ, Feb. 11, 2023

Everybody and their Watch Box: State Surveillance

Aerial surveillance can reach backwards in time, by the expedient of indiscriminately recording everything that is going on in a particular neighborhood, and then looking for useful patterns in the resulting footage. This technique, called wide-area motion imagery (Wami), has been around since 2006. But improvements in both the recording equipment used and the means by which the images are analysed are making it more and more valuable.

Wami was first employed by American forces in Iraq to track down those placing roadside bombs. When such a bomb went off, it was possible to run the relevant footage in reverse and trace the events that led up to the explosion. That often allowed the bombers to be identified and dealt with…Wami began with an aircraft-borne system called Constant Hawk, which was developed by Lawrence Livermore National Laboratory, in California. Constant Hawk’s success in Iraq begat more powerful versions. Gorgon Stare, carried by drone, was designed by the armed forces themselves…

But there is a problem. Explosions are easy to see. For many tasks, however, an awful lot of staring at screens looking for things that are out-of-the-ordinary is involved. People are bad at this…So AI is here to help…. Chips called graphic-processing units, borrowed from the video-game industry, are helping. So is machine learning, the basis of much modern artificial intelligence. .

l3Harris, a company in Florida, sells Wami sensors for use as automatic sentries. Their software monitors the coming and going of vehicles and pedestrians into and out of so-called watch boxes. These are protected areas surrounded by virtual trip wires, the triggering of which will cause a vehicle or individual of interest to be tracked…This approach can detect immediate threats. It can also, working over a longer period, carry out “pattern of life” analysis by building up a picture of what normal daily traffic looks like in an area. That permits the identification of anomalies which might signal hostile agents whose movements would otherwise be masked by the hurly-burly around them.

The sensors themselves are getting better, too….The latest version includes a so-called hyperspectral sensor, which sees simultaneously across many different wavelengths, including infrared and ultraviolet. It is thus able to distinguish things which the naked eye cannot, such as the difference between camouflage and vegetation. This approach’s real power, however, lies in software which automatically passes data between sensors…Future multi-sensor pods may include other instruments, such as signals-intelligence receivers. These are bits of equipment which can detect radio-frequency communicators like mobile phones and walkie-talkies, enabling particular devices to be identified and located. That would permit the individual carrying the phone, and also those he or she came into contact with, to be tracked and photographed. 

So far, the costs and complexity of Wami have kept it as a predominantly military technology. But that is starting to change. Smaller and more affordable versions are now within the reach of police, fire services and other non-military users…The most famous examples were in Baltimore, where the local cops experimented with the idea twice—first in 2016 and then in 2020. The second time around they made the mistake of monitoring a political protest as well as looking for crimes such as vehicle theft. 

Excerpts from Aerial Surveillance: The Spies in the Sky that See Backwards in Time, Economist, May 7, 2022

How to Spy on Your Own Country for $1.25 per day

San Francisco-based Premise Data Corp. pays users, many of them in the developing world, to complete basic tasks for small payments. Typical assignments involve snapping photos, filling out surveys or doing other basic data collection or observational reporting such as counting ATMs or reporting on the price of consumer goods like food.

About half of the company’s clients are private businesses seeking commercial information, Premise says. That can involve assignments like gathering market information on the footprint of competitors, scouting locations and other basic, public observational tasks. Premise in recent years has also started working with the U.S. military and foreign governments, marketing the capability of its flexible, global, gig-based workforce to do basic reconnaissance and gauge public opinion.

Premise is one of a growing number of companies that straddle the divide between consumer services and government surveillance and rely on the proliferation of mobile phones as a way to turn billions of devices into sensors that gather open-source information useful to government security services around the world.

Premise launched in 2013,, As of 2019, the company’s marketing materials said it has 600,000 contributors operating in 43 countries, including global hot spots such as Iraq, Afghanistan, Syria and Yemen. According to federal spending records, Premise has received at least $5 million since 2017 on military projects—including from contracts with the Air Force and the Army and as a subcontractor to other defense entities. In one pitch on its technology, prepared in 2019 for Combined Joint Special Operations Task Force-Afghanistan, Premise proposed three potential uses that could be carried out in a way that is “responsive to commander’s information requirements”: gauge the effectiveness of U.S. information operations; scout and map out key social structures such as mosques, banks and internet cafes; and covertly monitor cell-tower and Wi-Fi signals in a 100-square-kilometer area. The presentation said tasks needed to be designed to “safeguard true intent”—meaning contributors wouldn’t necessarily be aware they were participating in a government operation…

 Another Premise document says the company can design “proxy activities” such as counting bus stops, electricity lines or ATMs to provide incentives for contributors to move around as background data is gathered. Data from Wi-Fi networks, cell towers and mobile devices can be valuable to the military for situational awareness, target tracking and other intelligence purposes. There is also tracking potential in having a distributed network of phones acting as sensors, and knowing the signal strength of nearby cell towers and Wi-Fi access points can be useful when trying to jam communications during military operations. Nearby wireless-network names can also help identify where a device is, even if the GPS is off, communications experts say.

Mr. Blackman said gathering open-source data of that nature doesn’t constitute intelligence work. “Such data is available to anyone who has a cellphone,” he said. “It is not unique or secret.” Premise submitted a document last July to the British government describing its capabilities, saying it can capture more than 100 types of metadata from its contributors’ phones and provide them to paying customers—including the phone’s location, type, battery level and installed apps. 

Users of the Premise app aren’t told which entity has contracted with the company for the information they are tasked with gathering. The company’s privacy policy discloses that some clients may be governments and that it may collect certain types of data from the phone, according to a spokesman…Currently the app assigns about five tasks a day to its users in Afghanistan, according to interviews with users there, including taking photos of ATMs, money-exchange shops, supermarkets and hospitals. One user in Afghanistan said he and others there are typically paid 20 Afghani per task, or about 25 cents—income for phone and internet services. A few months ago, some of the tasks on the site struck him as potentially concerning. He said the app posted several tasks of identifying and photographing Shiite mosques in a part of western Kabul populated largely by members of the ethnic Hazara Shiite minority. The neighborhood was attacked several times by Islamic State over the past five years…. Because of the nature and location of the tasks in a hot spot for terrorism, the user said he thought those tasks could involve spying and didn’t take them on.

Excerpt from Byron Tau, App Users Unwittingly Collect Intelligence, WSJ,  June 25, 2010

Dodging the Camera: How to Beat the Surveillance State in its Own Game

Powered by advances in artificial intelligence (AI), face-recognition systems are spreading like knotweed. Facebook, a social network, uses the technology to label people in uploaded photographs. Modern smartphones can be unlocked with it… America’s Department of Homeland Security reckons face recognition will scrutinise 97% of outbound airline passengers by 2023. Networks of face-recognition cameras are part of the police state China has built in Xinjiang, in the country’s far west. And a number of British police forces have tested the technology as a tool of mass surveillance in trials designed to spot criminals on the street.  A backlash, though, is brewing.

Refuseniks can also take matters into their own hands by trying to hide their faces from the cameras or, as has happened recently during protests in Hong Kong, by pointing hand-held lasers at cctv cameras. to dazzle them. Meanwhile, a small but growing group of privacy campaigners and academics are looking at ways to subvert the underlying technology directly…

Laser Pointers Used to Blind CCTV cameras during the Hong Kong Protests 2019

In 2010… an American researcher and artist named Adam Harvey created “cv [computer vision] Dazzle”, a style of make-up designed to fool face recognisers. It uses bright colours, high contrast, graded shading and asymmetric stylings to confound an algorithm’s assumptions about what a face looks like. To a human being, the result is still clearly a face. But a computer—or, at least, the specific algorithm Mr Harvey was aiming at—is baffled….

Modern Make-Up to Hide from CCTV cameras

HyperFace is a newer project of Mr Harvey’s. Where cv Dazzle aims to alter faces, HyperFace aims to hide them among dozens of fakes. It uses blocky, semi-abstract and comparatively innocent-looking patterns that are designed to appeal as strongly as possible to face classifiers. The idea is to disguise the real thing among a sea of false positives. Clothes with the pattern, which features lines and sets of dark spots vaguely reminiscent of mouths and pairs of eyes are available…

Hyperface Clothing for Camouflage

 Even in China, says Mr Harvey, only a fraction of cctv cameras collect pictures sharp enough for face recognition to work. Low-tech approaches can help, too. “Even small things like wearing turtlenecks, wearing sunglasses, looking at your phone [and therefore not at the cameras]—together these have some protective effect”. 

Excerpts from As face-recognition technology spreads, so do ideas for subverting it: Fooling Big Brother,  Economist, Aug. 17, 2019

Nuclear Weapons Proliferation: the Race

As nuclear blasts go, North Korea’s first test in 2006 was small. The detonation of an underground device produced an explosive force well below one kiloton (less than a tenth of the size of the bomb dropped on Hiroshima in 1945). Even so, the vibrations it caused were recorded half a world away in the centre of Africa. Advances in the sensitivity of seismic sensors and monitoring software are now good enough to distinguish between a distant nuclear detonation and, say, a building being demolished with conventional explosives, says Lassina Zerbo, head of the Preparatory Commission for the Comprehensive Test-Ban-Treaty Organisation (CTBTO), the international organisation that seeks to enforce the agreement ratified, so far, by 163 nations.

The CTBTO operates 170 seismic stations worldwide, 11 underwater hydroacoustic centres detecting sound waves in the oceans, 60 listening stations for atmospheric infrasound (low-frequency acoustic waves that can travel long distances) and 96 labs and radionuclide-sampling facilities. More sensors are being installed. Crucially, however, the optimal number for global coverage was recently reached. It is now impossible, reckons Dr Zerbo, to test even a small nuclear weapon in secret anywhere on Earth. And on top of that, the United States Air Force runs a detection network that includes satellites that can spot nuclear-weapons tests.

It is better, though, to discover a secret weapons programme before testing. Once a country has a nuclear bomb or two, there is not much other governments can do to stop it from making more, says Ilan Goldenberg, a former head of the Iran team at the Pentagon. Plenty of states want such capabilities. The Defence Science Board, an advisory body to the Pentagon, concluded in a report last year that the number of countries that might seek nuclear weapons is higher now than at any time since the cold war. Those states include Saudi Arabia and other Sunni-Arab rivals of Iran, which in July, after long and tortuous negotiations, signed a nuclear deal with America and other nations to restrict its nuclear activities, and to allow enhanced monitoring and inspection of its facilities.

As the technologies to unearth work on clandestine nuclear weapons become more diverse and more powerful, however, the odds of being detected are improving. Innovation is benefiting detection capabilities, says Ramesh Thakur, a former UN assistant secretary-general. The products under development range from spy software that sifts through electronic communications and financial transactions to new scanners that can detect even heavily shielded nuclear material….

Software used for this type of analysis include i2 Analyst’s Notebook from IBM, Palantir from a Californian firm of the same name, and ORA, which was developed with Pentagon funds at Carnegie Mellon University in Pennsylvania. ORA has crunched data on more than 30,000 nuclear experts’ work and institutional affiliations, research collaborations and academic publications, says Kathleen Carley, who leads the ORA work at Carnegie Mellon. Changes, such as a halt in publishing, can tell stories: scientists recruited into a weapons programme typically cannot publish freely. Greater insights appear when classified or publicly unavailable information is sifted too. Credit-card transactions can reveal that, say, a disproportionate number of doctors specialising in radiation poisoning are moving to the same area.

The software uses combinatorial mathematics, the analysis of combinations of discrete items, to score individuals on criteria including “centrality” (a person’s importance), “between-ness” (their access to others), and “degree” (the number of people they interact with). Network members with high between-ness and low degree tend to be central figures: they have access to lots of people, but like many senior figures may not interact with that many. Their removal messes things up for everybody. Five or more Iranian nuclear scientists assassinated in recent years—by Israel’s Mossad, some suspect—were no doubt chosen with help from such software, says Thomas Reed, a former secretary of the United States Air Force and co-author of “The Nuclear Express”, a history of proliferation.

Importantly, the software can also evaluate objects that might play a role in a nuclear programme. This is easier than it sounds, says a former analyst (who asked not to be named) at the Pentagon’s Central Command in Tampa, Florida. Ingredients for homemade conventional bombs and even biological weapons are available from many sources, but building nukes requires rare kit. The software can reveal a manageable number of “chokepoints” to monitor closely, he says. These include links, for instance, between the few firms that produce special ceramic composites for centrifuges and the handful of companies that process the material.

A number of countries, including Japan and Russia, use network analysis. Japan’s intelligence apparatus does so with help from the Ministry of Economy, Trade and Industry, which assists in deciding which “dual use” items that might have both peaceful and military purposes should not be exported. Such work is tricky, says a member of the advisory board (who also asked not to be named) to the security council of the Russian Federation, a body chaired by Vladimir Putin. Individual items might seem innocent enough, he says, and things can be mislabelled.

Data sources are diverse, so the work takes time. Intelligence often coalesces after a ship has left port, so foreign authorities are sometimes asked to board and search, says Rose Gottemoeller, undersecretary for arms control at America’s State Department. The speed of analysis is increasing, however. Software that converts phone conversations into computer-readable text has been “extremely helpful”, says John Carlson, a former head of the Australian foreign ministry’s Safeguards and Non-Proliferation Office.

Would-be nuclear states can also reduce their networks. North Korea helped to keep its centrifuge facility secret by using mostly black-market or domestically manufactured components. Iran is also indigenising its nuclear programme, which undermines what network analysis can reveal, says Alexander Montgomery, a political scientist at Reed College in Portland, Oregon. Iran mines uranium domestically and has produced centrifuge rotors with carbon fibre, instead of importing special maraging steel which is usually required.

A big computer system to make sense of all this would help, says Miriam John, vice-chairman of the Pentagon’s Threat Reduction Advisory Committee. Which is why the Pentagon is building one, called Constellation. Dr John describes it as a “fusion engine” that merges all sorts of data. For instance, computers can comb through years of satellite photos and infra-red readings of buildings to detect changes that might reveal nuclear facilities. Constellation aims to increase the value of such nuggets of information by joining them with myriad other findings. For example, the whereabouts of nuclear engineers who have stopped teaching before retirement age become more interesting if those people now happen to live within commuting distance of a suspect building.

Yet photographs and temperature readings taken from satellites, even in low Earth orbit, only reveal so much. With help from North Korea, Syria disguised construction of a nuclear reactor by assembling it inside a building in which the floor had been lowered. From the outside the roof line appeared to be too low to house such a facility. To sidestep the need for a cooling tower, water pipes ran underground to a reservoir near a river. The concealment was so good the site was discovered not with remote sensing but only thanks to human intelligence, says Dr Tobey, the former National Security Council official. (Israel bombed the building in 2007 before it could be completed.)

Some chemical emissions, such as traces of hydrofluoric acid and fluorine, can escape from even well-built enrichment facilities and, with certain sensors, have been detectable from space for about a decade, says Mr Carlson, the Australian expert. But detecting signs of enrichment via radiation emissions requires using different sorts of devices and getting much closer to suspected sources.

The “beauty” of neutrons and alpha, beta and gamma radiation, is that the energy levels involved also reveal if the source is fit for a weapon, says Kai Vetter, a physicist at the University of California, Berkeley. But air absorbs enough radiation from uranium and plutonium bomb fuel to render today’s detectors mostly useless unless they are placed just a few dozen metres away. (Radiological material for a “dirty bomb” made with conventional explosives is detectable much farther away.) Lead shielding makes detection even harder. Not one of the more than 20 confirmed cases of trafficking in bomb-usable uranium or plutonium has been discovered by a detector’s alarm, says Elena Sokova, head of the Vienna Centre for Disarmament and Non-Proliferation, a think-tank.

Ground-based detectors are becoming more sensitive….. Detectors still need to be close to whatever it is they are monitoring, which mostly restricts their use to transport nodes, such as ports and borders. The range the detectors operate over might stretch to about 100 metres in a decade or so, but this depends on uncertain advances in “active interrogation”—the bombardment of an object with high-energy neutrons or protons to produce other particles which are easier to pick up. One problem is that such detectors might harm stowaways hiding in cargo.

That risk has now been solved, claims Decision Sciences, a Californian company spun out of the Los Alamos National Laboratory in America. It uses 16,000 aluminium tubes containing a secret gas to record the trajectory of muons. These are charged particles created naturally in the atmosphere and which pass harmlessly through people and anything else in their path. However, materials deflect their path in different ways. By measuring their change in trajectory, a computer can identify, in just 90 seconds, plutonium and uranium as well as “drugs, tobacco, explosives, alcohol, people, fill in the blank”, says Jay Cohen, the company’s chief operating officer and a former chief of research for the United States Navy. The ability to unearth common contraband will make the machine’s $5m price tag more palatable for border officials. A prototype is being tested in Freeport, Bahamas.

Other groups are also working on muon detectors, some using technology developed for particle physics experiments at the Large Hadron Collider in Switzerland. Another approach involves detecting neutrinos, which are produced by the sun and nuclear reactors, and seeing how they interact with other forms of matter. The NNSA and other organisations are backing the construction of a prototype device called WATCHMAN in an old salt mine (to shield it from cosmic rays and other interference) in Painesville, Ohio. It will be used to detect neutrinos from limited plutonium production at a nuclear power station 13km away. Such a system might have a 1,000km range, eventually. But even that means it would require a friendly neighbour to house such a facility on the borders of a country being monitored.

Once nuclear facilities have been discovered, declared or made available for inspection as part of a deal, like that signed with Iran, the job of checking what is going on falls to experts from the UN’s International Atomic Energy Agency (IAEA). The equipment available to them is improving, too. The Canadian Nuclear Safety Commission has built a prototype hand-held spectrometer for determining if traces of uranium collected on a cotton swab and blasted with a laser emit a spectral signature that reveals enrichment beyond that allowed for generating electricity. Within three years it will provide an unprecedented ability to assess enrichment without shipping samples back to a lab, says Raoul Awad, director-general of security and safeguards at the commission.

Laser scanning can also reveal other signs of enrichment. A decade ago inspectors began scanning intricate centrifuge piping with surveying lasers. A change between visits can reveal any reconfiguration of the sort necessary for the higher levels of enrichment needed for bombmaking. Secret underground facilities might also be found by wheeling around new versions of ground-penetrating radar.

The remote monitoring of sites made available to inspectors is also getting better. Cameras used to record on videotape, which was prone to breaking—sometimes after less than three months’ use, says Julian Whichello, a former head of the IAEA’s surveillance unit. Today’s digital cameras last longer and they can be programmed to take additional pictures if any movement is detected or certain equipment is touched. Images are encrypted and stamped with sequential codes. If technicians at a monitored facility delete any pictures, the trickery will be noticed by software and the inspectors informed.

Such technology, however, only goes so far. The IAEA cannot inspect computers and countries can veto the use of some equipment. It does seem that inspectors sent to Iran will get access to Parchin, a site near Tehran where intelligence agencies say tests related to nuclear-weapons making took place. (Iran denies it has a military programme.) But even the best tech wizardry can only reveal so much when buildings have been demolished and earth moved, as in Parchin.

Could nuclear weapons be built in secret today? …. A senior American State Department counter-proliferation official (whose asked to remain anonymous), however, says that it is not impossible…Companies, including a General Electric consortium, are making progress enriching uranium with lasers . If this becomes practical, some worry that it might be possible to make the fuel for a nuclear bomb in smaller facilities with less fancy kit than centrifuges

Monitoring nuclear weapons: The nuke detectives, Economist Technology Quarterly, Sept. 5, 2015, at 10

Your Biosignature and the Military

Human-Centered Intelligence, Surveillance & Reconnaissance (ISR) Leveraged Science & Technology (S&T) Program

The overall RHX (Human Effectiveness Directorate, Anticipate & Influence Behavior Division of the Air Force Research Laboratory) research objective is to develop human-centered S&T that enables the Air Force to more effectively execute the ISR mission…   Current ISR systems are ideal for identifying and tracking entities such as aircraft and vehicles but are less capable of identifying and tracking the human. This research will develop technologies to enable the Air Force to identify, locate and track humans of interest within the operational environment….The scope of human-centered ISR research spans the complete range of human performance starting at the individual molecular, cellular, genomic level and progressing to complex human-to-human and human-to-machine interactions. Human-centered ISR reaches across multiple domains (air, space, cyber) and has broad application to other DoD organizations and the Intelligence Community (IC).  Human-centered ISR research encompasses three major research areas: (1) human signatures, (2) human trust and interaction and (3) human analyst augmentation. The human signatures research develops technologies to sense and exploit human bio-signatures at both the molecular level and macro (anthropometric) level. The human trust and interaction research develops technologies to improve human-to-human interactions as well as human-to-machine interactions. The human analyst augmentation research develops technologies to enhance analyst performance and to test the efficacy of newly developed technologies within a simulated operational environment.

OBJECTIVE 1: Human Signatures

The objective of the Human Signatures Program is to develop technologies to discover, characterize and transition biological-based signatures (biosignatures) to enable effective human and environmental threat detection, identification and exploitation, and operator performance assessment across a variety of Air Force mission areas. Human signatures research seeks to identify and characterize unique biosignatures that can be exploited to identify, locate and track specific individuals or groups of people possessing certain characteristics of operational interest. Bioignatures range from the micro-level (molecular, cellular, genomic) up to whole body physiological signatures based on anthropometric and biomechanical properties and characteristics.

Exploitation of biosignatures also requires development of (1) sensors designed to detect and collect biosignatures; (2) analytics and informatics to process, analyze, fuse and utilize biosignature sensor data; (3) end user systems that integrate biosignatures into the layered sensor network and provide analysis, visualization, and prediction tools to exploit biosignature data.

OBJECTIVE 2: Human Trust and Interaction

The Human Trust and Interaction Program conducts research examining human-to-human interactions and human-to-machine interactions with the focus on developing technological solutions to enhance ISR capabilities and human performance assessments. Research is divided into two major areas: (1) human insight and trust and (2) human language technologies.  The objectives of the Human Interaction and Trust Program are broken down into three subareas. These are: (1) Trust and Suspicion; (2) Trust in Automation; and (3) Social Signature Exploitation. Trust and Suspicion research focuses on the recognition of suspicious activities in the cyberspace realm. The needs include the full gamut of open source data including social media to the more traditional intelligence sources. Trust in Automation is driven by human-machine teams and how humans relate to technology. A key need in this area is the establishment of trust between human operators and the machines/software they are teamed with to complete their mission. Finally, the Social Signature Exploitation theme focuses on recognizing behavior indicators that are based on social and cultural factors to assess and predict military relevant events. The need includes the use of open and closed data resources to assist decision making on the use of force or non-physical actions.

Excerpt  from Human-Centered Intelligence, Surveillance & Reconnaissance (ISR) Leveraged Science & Technology (S&T) Program, Solicitation Number: BAA-HPW-RHX-2014-0001, Agency: Department of the Air Force, Office: Air Force Materiel Command, Location: AFRL/RQK – WPAFB, available online