Weapons Of Mass Distraction – Sock Puppets & Persona Management

South African applied research firm Thinkst delivered a talk at Hack In The Box two months ago on a topic that has been on your mind, whether you knew it or not. The practice of sock puppets participating in online conversations is endemic, skewing the thinking and behavior of all of us.

This topic first became an issue of pubic discussion right after Anonymous broke into HBGary Federal in February of 2011. This small, floundering contractor had attempted to poach counterinsurgency work that was currently held by Northrop Grumman. Failing in that, they were shopping their skills to whoever might be interested, and they sought to impress Bank of America, who were facing an angry, energized hive. CEO Aaron Barr publicly claimed to have identified leaders of Anonymous, which promptly resulted in a group of people he’d never heard of wiping his company right off the face of the internet.

The phrase “persona management” was found in HBGary’s email spool and in other places, triggering an obsession with the concept among a subset of online activists. While there are no doubt infiltrators at work, Thinkst’s recent research indicates we ought to be paying more attention to pervasive whispering than looking for Jason Bourne like spies in our midst.

Real World Vulnerabilities

Hate Crymes in CyberspaceThinkst conducted real world experiments in many different venues. All offered some opening for an actor willing to manipulate discussions, with some providing truly striking vulnerabilities given their potential to influence public debate.

  • Mailing Lists
  • Online Polls
  • Twitter
  • Reddit
  • Disqus
  • Wall Street Journal’s popular now panel
  • New York Times most emailed stories

Mailing lists proved trivially easy to manipulate. Respond on topics you want people to read, make other threads busy if there is a topic you want to be ignored. Online polls are famously easy to blow up, which 4chan demonstrated in 2009 when site founder Christopher ‘moot’ Poole won Time Magazine’s Most Influential Person.

Twitter has seized a large portion of the work that used to be done with RSS, and it brings humans into direct involvement. This leads to people sampling feeds when they are free rather than reading to catch up. Automation can drive content just by referencing it more often.

The Wall Street Journal’s popular now panel was weighted such that mailing a story had some effect, but their findings on the New York Times most emailed section were striking.

“Based on quick calculations on how much machine time we were spending on Amazon Web Services, the cost to register 30,000 accounts was about 12 cents; the cost to share 30,000 stories was about 18 cents.”

“To trivially manipulate the NYT front page? That was priceless,” he said.

Digital News Asia has a much more lengthy writeup on Thinkst’s presentation; it’s well worth reading if you are curious.

The Ugly Underside Of Comments

Professionals bending public opinion are one thing, but the endemic indecency, particularly that which is directed at women, is just as pernicious, but more resilient to treatment since it seems to be mostly organic, rather than automated. All it takes is one obsessive male to strike terror into the heart of an attractive executive as Gregory King did to then Google exec Marissa Mayer.

Twitter recently received another in a string of black eyes for its poor safety record with the loss of Zelda Williams, driven from the service by the ugliness that flowed after her father’s suicide. Twitter has reversed its trend in problem reporting, replacing a burdensome long form with a simple popup, but for dedicated harassers and those unfortunate enough to acquire a persistent one this changes little.

While Mayer and Williams received assistance, for the rest of us dealing with such unwanted attention is essentially hopeless. Police where the harasser lives will direct victims to their local law enforcement, who will patiently wait for an opportunity to dismiss concerns by saying it’s a civil issue. A society wide cleanup in this area would likely involve a repeal of Section 230 of the Communications Decency Act of 1996.

A ubiquitous voice in this area is University of Maryland professor Danielle Citron, whose recently published Hate Crimes In Cyberspace offers a set of prescriptions for these troubles.

Real World Solutions

It would be unfair to leave you with the impression that every time you go online you’re walking in a realm of shadows where every opinion is suspect, and every person is just a ploy. Nudges, a behavioral economics project of Richard H. Thaler and Cass R. Sunstein, has been involved, as the name suggests, in ‘nudging’ people.

The Nudges site itself hasn’t stirred in years, thanks in part to Sunstein’s role at the White House, but @Nudgeblog, ran by John Balz, and a cluster of related accounts such as @ideas42 keep up a steady flow of suggestions and links to academic work in the field. The same tactics that can confuse and disperse citizens inclined to public debate are being employed to educate and motivate.

Nudge is a small ray of sunshine and there are others, but you’ll have to look carefully to find them.

Collaring The NSA, But Reducing Free Expression?

The U.S. Congress will be compelled at some point to collar the NSA and put an end to the surveillance dragnet they allowed to grow without any meaningful oversight. This is not just unconstitutional; it has become a barrier to sales from American companies affecting everything from flip phones to fighter jets. When they finally take action it will likely be a broad effort and online indecency may be addressed both for the direct hazard that it is, as well as a means to tamp down some forms of speech used to incite hatred and violence. Citron’s arguments for treating such speech as a civil rights affront, views that are widely held by others considering such problems, will work against the position free speech absolutists will take.

Will Congress address the influence methods which Thinkst demonstrated? This seems unlikely, as they are all means of influencing both policy and elections. They require some money and skill to implement, and that favors incumbent politicians.

When William Gibson coined the word cyberspace to describe online interaction over thirty years ago, he envisioned a world where those who ‘jacked in’ could face harm or even death if their online presence were attacked. Our bodies are safe in cyberspace, at least for the moment, but our minds are certainly in need of some protection. Thinkst’s research demonstrates just how easy it is to manipulate perceptions.

Images from Harvard University Press and Shutterstock.