Tag Archives: open-source intelligence (OSINT)

Surveillance by the Masses for the Masses

New sensors, from dashboard cameras to satellites that can see across the electromagnetic spectrum, are examining the planet and its people as never before. The information they collect is becoming cheaper. Satellite images cost several thousand dollars 20 years ago, today they are often provided free and are of incomparably higher quality….

Human Rights Watch has analysed satellite imagery to document ethnic cleansing in Myanmar. Nanosatellites tag the automatic identification system of vessels that are fishing illegally. Amateur sleuths have helped Europol, the European Union’s policing agency, investigate child sexual exploitation by identifying geographical clues in the background of photographs. Even hedge funds routinely track the movements of company executives in private jets, monitored by a web of amateurs around the world, to predict mergers and acquisitions. OSINT (open-source intelligence) thus bolsters civil society, strengthens law enforcement and makes markets more efficient. It can also humble some of the world’s most powerful countries.

In the face of vehement denials from the Kremlin, Bellingcat, an investigative group, meticulously demonstrated Russia’s role in the downing of Malaysian Airlines Flight mh17 over Ukraine in 2014, using little more than a handful of photographs, satellite images and elementary geometry. It went on to identify the Russian agents who attempted to assassinate Sergei Skripal, a former Russian spy, in England in 2018. Amateur analysts and journalists used OSINT to piece together the full extent of Uyghur internment camps in Xinjiang. In recent weeks researchers poring over satellite imagery have spotted China constructing hundreds of nuclear-missile silos in the desert.

Such an emancipation of information promises to have profound effects. The decentralised and egalitarian nature of OSINT erodes the power of traditional arbiters of truth and falsehood, in particular governments and their spies and soldiers. The likelihood that the truth will be uncovered raises the cost of wrongdoing for governments. Although osint might not prevent Russia from invading Ukraine or China from building its gulag, it exposes the flimsiness of their lies

Liberal democracies will also be kept more honest. Citizens will no longer have to take their governments on trust. News outlets will have new ways of holding them to account. Today’s open sources and methods would have shone a brighter light on the Bush administration’s accusation in 2003 that Iraq was developing chemical, biological and nuclear weapons. That would have subjected America’s invasion of the country to greater scrutiny. It might even have prevented it.,,

The greatest worry is that the explosion of data behind open-source investigations also threatens individual privacy. The data generated by phones and sold by brokers let Bellingcat identify the Russian spies who last year poisoned Alexei Navalny, an opposition leader. Similar data were exploited to pick out a senior Catholic priest in America, who resigned last month after his location was linked to his use of Grindr, a gay dating app.

Excerpts from The people’s panopticon: The promise of open-source intelligence, Economist, Aug. 7, 2021

Surveillance: Private Web Spiders

With so many cheap or free tools out there, it is easy for anyone to set up their own NSA-esque operations and collect data. Though breaching systems and taking data without authorisation is against the law, it is possible to do a decent amount of surveillance entirely legally using open-source intelligence (OSINT) tools…. Daniel Cuthbert, chief operating officer of security consultancy Sensepost, has been happily using OSINT tool Maltego (its open-source version is charmingly called Poortego) [pdf] to track a number of people online.

Over a few days this summer, he was “stalking” a Twitter user who appeared to be working at the Central Intelligence Agency. Maltego allowed him to collect all social media messages sent out into the internet ether in the area around the CIA’s base in Langley, Virginia. He then picked up on the location of further tweets from the same user, which appeared to show her travelling between her own home and a friend or partner’s house. Not long after Cuthbert started mapping her influence, her account disappeared.

But Cuthbert has been retrieving far more illuminating data by running social network accounts related to Islamic State through Maltego. By simply adding names to the OSINT software and asking it to find links between accounts using commands known as “transforms”, Maltego draws up real-time maps showing how users are related to each other and then uncovers links between their followers. It is possible to gauge their level of influence and which accounts are bots rather than real people. Where GPS data is available, location can be ascertained too, though it is rare to find accounts leaking this – only about 2% of tweets have the feature enabled, says Cuthbert.

He has been trying, with mixed results thanks to Twitter’s deletion of accounts spreading Isis propaganda, to determine how tech savvy its members are and how they operate online. Over the past month, Cuthbert has looked at links between a number of pro-Isis users, including one with the handle @AbuHussain104, who has only tweeted 28 times, yet has more than 1,300 followers already. The prominent pro-sharia law Islamic activist Anjem Choudary has been a keen retweeter of Hussain’s words.  The London-based professional hacker has noted the group’s ability to attract followers online; his research shows how a handful of Isis-affiliated accounts have myriad links and wide influence.

Cuthbert is now on the lookout for slipups that reveal the true identity or location of the tweeter. “This is a concern for high-ranking Isis leaders, so much so, they issued a guide on using social media,” he notes, referring to reports of an as-yet unconfirmed document.,,,

Metagoofil, which runs on Linux or Mac machines, is an ideal software for uncovering data businesses have mistakenly leaked onto the internet. Running this free tool in a Linux distribution, hackers can command it to hunt for files related to a particular domain, specifying how many Google searches to look through and how many documents to download. It will then extract whatever metadata the user is looking for and store it all in a file for perusal later on.

For those who want instant visual results, the Shodan search tool is a remarkable piece of work. Simple searches can reveal miraculous details. For instance, type “IP camera” into the search bar and more than 1.3m internet-connected IP cameras show up from across the world. Add “country:gb” and you’ll be shown more than 54,000 based in Great Britain. You could specify a manufacturer too, such as Samsung. That provides just 13 results. From there, it’s a matter of clicking on the IP addresses to see which ones allow you to view live footage either with or without a password (if you guess the password, even if it’s a default one such as “admin”, it will mean you are likely to have broken the Computer Misuse Act).  Either way, it is very easy to find poorly secured cameras – many have a username of “admin” and no password whatsoever, according to previous research. It is that straightforward: no coding skills required….

“The tools are mostly for reconnaissance,” says Christian Martorella, creator of Metagoofil and theHarvester, another OSINT software that pentesters – or “ethical hackers” – use to map their clients’ internet footprint. “This helps the pentester to have as much information as possible about the targets and plan the attacks. This phase is very important but … pentesters usually overlook this phase or dedicate little time, while attackers seem to spend more time in this phase.”

Privacy-conscious folk can also benefit from OSINT. While looking into how his internet service provider [ISP] was interfering with his internet connection, in a method similar to that used by Verizon for its controversial “permacookie” tracking software, researcher Lee Brotherston last month used Shodan to find servers that intercepted his traffic. The wide range of Perftech servers he found were based across the world, and though his ISP was simply using a “man-in-the-middle” technique to add a warning banner to a website he visited, … But what if the ISP was coerced by a government and dropped malware onto people’s machines as they tried to access websites? The much-maligned surveillance tool FinSpy is used for just for that purpose: it is placed into the data centres of ISPs and intercepts traffic to force surreptitious downloads of surveillance software. Instead of dropping banners, as Brotherston’s ISP did, it injects malicious JavaScript.  “When you hear about repressive governments that start installing malware on activists’ machines and then arresting them… it’s the same technique. They’re injecting data into a webpage,” says Brotherston, a Canada-based Brit. “If you’re injecting this, you may have a valid business case for doing, it but someone could break in and start dropping malware on people’s machines.”

A number of developers, inspired by the success of Shodan creator John Matherly, have drawn up search sites for hackable systems. Perhaps the most useful for security professionals, whether of the blackhat or whitehat variety, is the Kickstarter-funded PunkSPIDER, a web app vulnerability search engine, which issues an alert as soon as the visitor arrives: “Please do not use this site for malicious purposes … use it wisely or we’ll have to take it away”. It’s remarkably simple. Type or paste in a URL and it will reveal what vulnerabilities have been documented for the related site.

Such is the openness of the web, and such is the carelessness of so many web denizens, any determined citizen can gather up reams of sensitive information on others and collect enough data to create a decent picture of who they are, where they are and what they are doing. The tools are now accessible for the typical web user.

Excerpts fromTom, Fox-Brewster, Tracking Isis, stalking the CIA: how anyone can be big brother online, Guardian, Nov. 12, 2014