There is no magic algorithm that can moderate live streams and fix racism
Australia’s Prime Minister, Scott Morrison, has called for the G20 to regulate social media companies in the wake of the terrorist massacre in Christchurch last Friday. The knee-jerk reaction to the mass shootings proposes policy responses that won’t work, and fails to address the real challenges of combating the spread of hatred in our society.
The Christchurch massacre was livestreamed on Facebook, and quickly copied and shared on many other sites by thousands of people around the world. Mainstream media coverage has played a big role in increasing demand for the video. As news of the shootings spread, the story was frequently reported with excerpts that inevitably drove people to search for and share copies of the entire video.
Now our political leaders, eager to avoid responsibility for stoking the fires of hate, racism, and Islamophobia in our country, have sought to pin the blame squarely on social media companies. They want tough new laws that impose penalties on platforms like Facebook, YouTube, and Twitter for allowing people to stream or post horrific content. But this is not well thought out policy.
Over the last few years, we have seen just how important livestreaming is to document important events. When Philando Castile, a 32-year-old black man, was shot after a routine traffic stop in Minnesota, the entire event was streamed live to the world on Facebook. Castile’s girlfriend, Diamond Reynolds, relied on Facebook to make us witness the shocking killing. This video, like far too many others, has changed the conversation by raising awareness of the brutal treatment of African Americans by police.
No magic algorithm exists that can distinguish a violent massacre from videos of police brutality.
There is no practical way to stop people from uploading live videos of harmful content. No magic algorithm exists that can distinguish a violent massacre from videos of police brutality. Like with all social media content, it is not possible to moderate videos in advance without making the system almost useless and inevitably censoring important information. Even if Facebook shuts down its live streaming feature, prepared terrorist will find ways to stream videos to other sites, from which they would surely spread just as quickly.
New laws won’t stop the spread of violent videos. The best we can hope for is that social media companies respond quickly to complaints. Facebook’s team of moderators have been working to combat the spread of the video, but the sheer number of people sharing it poses a major challenge. In the first two days, Facebook reported that it had removed posts uploading or linking to the video 1.5 million times. It was eventually able to use audio and video matching technology to prevent people from uploading copies of the video, but this technology is not perfect. It can be defeated by people committed to sharing the video by altering it in ways that confuse Facebook’s systems. Human reviewers will always be an important part of evaluating social media content, and human reviews take time.
Politicians and the media must take responsibility for creating hatred
What we need is a frank conversation about how hatred circulates and grows in our society. For decades racism and Islamophobia has been stoked by political leaders trying to win elections. Our mainstream media provides airtime to far right commentators who direct attention to extremist online influencers. These online influencers exploit the algorithms of social media platforms to gain traffic and advertising revenue and help to radicalise white supremacists.
Social media platforms do have a role to play in combating hate. We should expect major platforms like YouTube, Facebook, Twitter, and Reddit to work harder to stop everyday, ordinary racism, sexism, and Islamophobia from spreading. We should expect them to stop amplifying hatred and driving traffic and advertising funding to groups spreading hateful messages. At the same time, we need to pressure platforms to ensure that they are not stifling content from marginalised groups who are trying to document the hatred and abuse that they face on a daily basis. And we should expect them to improve due process and transparency in all of these processes, because their actions have real impact on our everyday lives. These are the conversations we need to start, but they are complex issues, and there is no simple legislative fix.
Really addressing the flow of hatred in our society will require politicians to take seriously their responsibility for stoking hatred for cynical political gain. It will require journalists and news organisations to do better in reporting on tragedies in ways that do not drive traffic to the communities of hate that radicalise people into white supremacist ideology. And it will require social media companies to engage thoughtfully with their role in amplifying and financially supporting extremist groups and come up with new strategies to effectively fight radicalisation and the normalisation of hate.
This article first appeared in the Australian Financial Review.
