West Virginia man mesmerized by extremists on YouTube

MARTINSBURG, West Virginia - Caleb Cain pulled a Glock pistol from his waistband, took out the magazine and casually tossed both onto the kitchen counter.

“I bought it the day after I got death threats,” he said.

The threats, Cain explained, came from right-wing trolls in response to a video he had posted on YouTube a few days earlier. In the video, he told the story of how, as a liberal college dropout struggling to find his place in the world, he had gotten sucked into a vortex of far-right politics on YouTube.

“I fell down the alt-right rabbit hole,” he said in the video.

Cain, 26, recently swore off the alt-right nearly five years after discovering it, and has become a vocal critic of it. He is scarred by his experience of being radicalized by what he calls a “decentralized cult” of far-right YouTube personalities, who convinced him that Western civilization was under threat from Muslim immigrants and cultural Marxists, that innate IQ differences explained racial disparities and that feminism was a dangerous ideology.

“I just kept falling deeper and deeper into this, and it appealed to me because it made me feel a sense of belonging,” he said. “I was brainwashed.”

Over years of reporting on internet culture, I've heard countless versions of Cain's story: An aimless young man - usually white, frequently interested in video games - visits YouTube looking for direction or distraction and is seduced by a community of far-right creators.

Some young men discover far-right videos by accident, while others seek them out. Some travel all the way to neo-Nazism, while others stop at milder forms of bigotry.

The common thread in many of these stories is YouTube and its recommendation algorithm, the software that determines which videos appear on users' home pages and in the “Up Next” sidebar next to a video that is playing. The algorithm is responsible for more than 70% of all time spent on the site.

The radicalization of young men is driven by a complex stew of emotional, economic and political elements, many having nothing to do with social media. But critics and independent researchers say YouTube has inadvertently created a dangerous on-ramp to extremism by combining two things: a business model that rewards provocative videos with exposure and advertising dollars, and an algorithm that guides users down personalized paths meant to keep them glued to their screens.

“There's a spectrum on YouTube between the calm section - the Walter Cronkite, Carl Sagan part - and Crazytown, where the extreme stuff is,” said Tristan Harris, a former design ethicist at Google, YouTube's parent company. “If I'm YouTube and I want you to watch more, I'm always going to steer you toward Crazytown.”

In recent years, social media platforms have grappled with the growth of extremism on their services. Many platforms have barred a handful of far-right influencers and conspiracy theorists, including Alex Jones of Infowars, and tech companies have taken steps to limit the spread of political misinformation.

YouTube, whose rules prohibit hate speech and harassment, took a more laissez-faire approach to enforcement for years. Last week, the company announced that it was updating its policy to ban videos espousing neo-Nazism, white supremacy and other bigoted views. The company also said it was changing its recommendation algorithm to reduce the spread of misinformation and conspiracy theories.

With 2 billion monthly active users uploading more than 500 hours of video every minute, YouTube's traffic is estimated to be the second highest of any website, behind only Google.com. According to the Pew Research Center, 94% of Americans ages 18 to 24 use YouTube, a higher percentage than for any other online service.

Like many Silicon Valley companies, YouTube is outwardly liberal in its corporate politics. It sponsors floats at LGBT pride parades and celebrates diverse creators, and its chief executive endorsed Hillary Clinton in the 2016 presidential election. President Donald Trump and other conservatives have claimed that YouTube and other social media networks are biased against right-wing views, and have used takedowns like those announced by YouTube on Wednesday as evidence for those claims.

In reality, YouTube has been a godsend for hyper-partisans on all sides. It has allowed them to bypass traditional gatekeepers and broadcast their views to mainstream audiences, and has helped once-obscure commentators build lucrative media businesses.

It has also been a useful recruiting tool for far-right extremist groups. Bellingcat, an investigative news site, analyzed messages from far-right chat rooms and found that YouTube was cited as the most frequent cause of members' “red-pilling” - an internet slang term for converting to far-right beliefs. A European research group, VOX-Pol, conducted a separate analysis of nearly 30,000 Twitter accounts affiliated with the alt-right. It found that the accounts linked to YouTube more often than to any other site.

“YouTube has been able to fly under the radar because until recently, no one thought of it as a place where radicalization is happening,” said Becca Lewis, who studies online extremism for the nonprofit Data & Society. “But it's where young people are getting their information and entertainment, and it's a space where creators are broadcasting political content that, at times, is overtly white supremacist.”

To back up his recollections, Cain downloaded and sent me his entire YouTube history, a log of more than 12,000 videos and more than 2,500 search queries dating to 2015.

These interviews and data points form a picture of a disillusioned young man, an internet-savvy group of right-wing reactionaries and a powerful algorithm that learns to connect the two. It suggests that YouTube may have played a role in steering Cain, and other young men like him, toward the far-right fringes.

It also suggests that, in time, YouTube is capable of steering them in very different directions.

An online community

From an early age, Cain was fascinated by internet culture. As a teenager, he browsed 4Chan, the lawless message board. He played online games with his friends and devoured videos of intellectuals debating charged topics like the existence of God.

The internet was an escape. Cain grew up in postindustrial Appalachia and was raised by his conservative Christian grandparents. He was smart, but shy and socially awkward, and he carved out an identity during high school as a countercultural punk. He went to community college, but dropped out after three semesters.

Broke and depressed, he resolved to get his act together. He began looking for help in the same place he looked for everything: YouTube.

One day in late 2014, YouTube recommended a self-help video by Stefan Molyneux, a Canadian talk show host and self-styled philosopher.

Like Cain, Molyneux had a difficult childhood, and he talked about overcoming hardships through self-improvement. He seemed smart and passionate, and he wrestled with big questions like free will, along with practical advice on topics like dating and job interviews.

Molyneux, who calls himself an “anarcho-capitalist,” also had a political agenda. He was a men's rights advocate who said feminism was a form of socialism and that progressive gender politics were holding young men back. He offered conservative commentary on pop culture and current events, explaining why Disney's “Frozen” was an allegory about female vanity, or why the fatal shooting of an unarmed black teenager by a white police officer was proof of the dangers of “rap culture.”

Cain was a liberal who cared about social justice, worried about wealth inequality and believed in climate change. But he found Molyneux's diatribes fascinating, even when they disagreed.

“He was willing to address young men's issues directly, in a way I'd never heard before,” Cain said.

In 2015 and 2016, as Cain dived deeper into his YouTube recommendations, he discovered a universe of right-wing creators. Over time, he watched dozens of clips by Steven Crowder, a conservative comedian, and Paul Joseph Watson, a prominent right-wing conspiracy theorist who was barred by Facebook this year. These people weren't all shouty demagogues. They were entertainers, building their audience with satirical skits, debates and interviews with like-minded creators. Some of them were part of the alt-right, a loose cohort of pro-Trump activists who sandwiched white nationalism between layers of internet sarcasm.

The software develops

If alienation was one ingredient in Cain's radicalization, and persuasive partisans like Molyneux were another, the third was a series of product decisions YouTube made starting in 2012.

In March that year, YouTube's engineers made an update to the site's recommendations algorithm. For years, the algorithm had been programmed to maximize views, by showing users videos they were likely to click on. But creators had learned to game the system, inflating their views by posting videos with exaggerated titles or choosing salacious thumbnail images.

In response, YouTube's executives announced that the recommendation algorithm would give more weight to watch time, rather than views. That way, creators would be encouraged to make videos that users would finish, users would be more satisfied and YouTube would be able to show them more ads.

The bet paid off. Within weeks of the algorithm change, the company reported that overall watch time was growing, even as the number of views shrank. According to a 2017 report, YouTube's watch time grew 50% a year for three consecutive years.

A month after its algorithm tweak, YouTube changed its rules to allow all video creators to run ads with their videos and earn a portion of their revenue. Previously, only popular channels that had been vetted by YouTube were able to run ads.

Neither change was intended to benefit the far right, and YouTube's algorithm had no inherent preference for extreme political content. It treated a white nationalist monologue no differently from an Ariana Grande cover or a cake icing tutorial.

But the far right was well positioned to capitalize on the changes. Many right-wing creators already made long video essays, or posted video versions of their podcasts. Their inflammatory messages were more engaging than milder fare. And now that they could earn money from their videos, they had a financial incentive to churn out as much material as possible.

In 2015, a research team from Google Brain, Google's much-lauded artificial intelligence division, began rebuilding YouTube's recommendation system around neural networks, a type of AI that mimics the human brain. In a 2017 interview with the Verge, a YouTube executive said the new algorithm was capable of drawing users deeper into the platform by figuring out “adjacent relationships” between videos that a human would never identify.

The new algorithm worked well, but it wasn't perfect. One problem, according to several current and former YouTube employees, was that the AI tended to pigeonhole users into specific niches, recommending videos that were similar to ones they had already watched. Eventually, users got bored.

Google Brain's researchers wondered if they could keep YouTube users engaged for longer by steering them into different parts of YouTube, rather than feeding their existing interests. And they began testing a new algorithm that incorporated a different type of AI, called reinforcement learning.

The new AI, known as Reinforce, was a kind of long-term addiction machine. It was designed to maximize users' engagement over time by predicting which recommendations would expand their tastes and get them to watch not just one more video but many more.

Reinforce was a huge success. In a talk at an AI conference in February, Minmin Chen, a Google Brain researcher, said it was YouTube's most successful launch in two years. Sitewide views increased nearly 1%, she said - a gain that, at YouTube's scale, could amount to millions more hours of daily watch time and millions more dollars in advertising revenue per year. She added that the new algorithm was already starting to alter users' behavior.

“We can really lead the users toward a different state, versus recommending content that is familiar,” Chen said.

In interviews, YouTube officials denied that the recommendation algorithm steered users to more extreme content. The company's internal testing, they said, has found just the opposite - that users who watch one extreme video are, on average, recommended videos that reflect more moderate views. They declined to share this data or give any specific examples of users who were shown more moderate videos after watching more extreme videos.

The officials stressed, however, that YouTube realized it had a responsibility to combat misinformation and extreme content.

‘A punk-rock moment'

By the night of Nov. 8, 2016, Cain's transformation was complete.

He spent much of the night watching clips of Clinton's supporters crying after the election was called in Trump's favor. His YouTube viewing history shows that at 1:41 a.m., just before bed, he turned on a live stream hosted by Crowder, with the title “TRUMP WINS!”

“It felt like a punk-rock moment, almost like being in high school again,” Cain said.

That year, Cain's YouTube consumption had skyrocketed. He got a job packing boxes at a furniture warehouse, where he would listen to podcasts and watch videos by his favorite YouTube creators all day.

After the election, he began exploring a part of YouTube with a darker, more radical group of creators.

These people posted videos with titles like “‘Refugee' Invasion Is European Suicide.” Cain never bought into the far right's most extreme views, like Holocaust denial or the need for a white ethnostate, he said.

The left strikes back

In 2018, nearly four years after Cain had begun watching right-wing YouTube videos, a new kind of video began appearing in his recommendations.

These videos were made by left-wing creators, but they mimicked the aesthetics of right-wing YouTube, down to the combative titles and the mocking use of words like “triggered” and “snowflake.”

Cain found videos by Natalie Wynn, a former academic philosopher who goes by the name ContraPoints. She explained why Western culture wasn't under attack from immigrants, or why race was a social construct.

Unlike most progressives Cain had seen take on the right, Wynn was funny and engaging. She spoke the native language of YouTube, and didn't get outraged by far-right ideas. Instead, she made them seem shallow and unsophisticated.

Natalie “made a bridge over to my side,” Cain said, “and it was interesting and compelling enough that I walked across it.”