How AI Is Moderating Online Content

How AI Is Moderating Online Content

AI can help flag harmful or offensive content faster and more effectively!

Whether it be by posting a photo on Instagram or writing a blog post, we’re all adding more information to the internet. With over 4.62 billion people using social media, there are bound to be some bad eggs creating harmful or deceitful content. To make sure that users are exposed to as little bad content as possible, websites practice content moderation. 

Content moderation is the process of regulating and monitoring user-generated content based on a set of pre-existing rules and guidelines. Since it is near impossible for human moderators to ever catch up to the huge amount of content being posted every second on social media, some websites are now using artificial intelligence (AI) to moderate posts. Let’s dive deeper into how AI content moderators work and whether they are actually effective in making the internet safer. 

What is AI moderation?

AI moderators are fed user-generated content to train them on what is acceptable and what’s not on a particular platform. The AI will learn to pick up patterns and filter out illegal, dangerous or sexually explicit content. Automating the content moderation process with AI allows a large amount of user-generated content to be checked in real-time. Platforms can thus be proactive in taking down any suspicious content that might be harmful to their users. Using AI moderators not only reduces the workload on human moderators but also reduces the chances of them being exposed to offensive content which can adversely affect their mental health. 

Two layers of content moderation

There are two broad layers of moderating content—surface-level moderation and context-based moderation. In the first layer, image, text and video content are checked through natural language processing to recognize damaging elements. If there is text inside an image, it can be further checked using object character recognition, a technology that converts words inside images into a format that the AI can understand. An example of an AI content moderator is Checkstep, which compares user-generated content against a company’s terms and conditions to check for violations. 

The second layer requires the AI to understand the nuance in which a particular statement is being made. While this form of AI moderation is still in its nascent stages, tech company Spectrum Labs has created the AI moderator Contextual AI, which gives us some clues on what context-based moderation could look like. Contextual AI takes into account current and historical responses to a user’s content to understand the context in which it is being created and understood. 

Is AI alone enough to monitor content?

Although AI moderation can help social media giants monitor content faster, it alone isn’t entirely effective. Since AI understands malign content based on the data it has been exposed to, it wouldn’t be able to flag harmful ideas that it doesn’t recognize. Moreover, even though there are attempts to contextualize content with technologies, it still has a long way to go before achieving accurate comprehension. At present, even if made in a completely harmless context, the use of some words, like “stupid”, might still end up giving the content a high toxicity score.

The issues with AI moderators have been well documented in the media. Recently, Daniel Motaung, a former content moderator at Facebook, shared that even though the social media giant claims that its AI has 95% accuracy in detecting hate speech, there are times when it removes content that doesn’t violate content policies. The same issue has been reported with other platforms too. For instance, in 2017 YouTube took down videos of journalists reporting on extremist content because its AI moderator didn’t know the difference between reportage and content inciting extremist behavior. This lack of detection and differentiation capabilities can wrongfully punish users, some of whom might be full-time content creators, and take down their content. 

Finally, much like other kinds of AI, AI moderators are also susceptible to bias based on the data that they have been trained on. Much like the issues around Google’s allegedly sentient AI, LaMDA, the kind of content being fed to the AI moderator also raises serious ethical questions. Ultimately, even though AI can detect problematic text and imagery, human moderators, who are better equipped to read between the lines, are still essential in content moderation. 

Also read:

Header image courtesy of Freepik


Share on facebook
Share on twitter
Share on linkedin
Share on email


Cultos Global Lands Major Investment to Advance Web3 Brand Engagement Solutions

Cultos Global Lands Major Investment to Advance Web3 Brand Engagement Solutions

Dubai-based Web3 company, Cultos Global, has recently acquired substantial investment in its latest funding round. The specific amount remains undisclosed. This venture specializes in altering the dynamics of brand-customer relationships by launching a sophisticated, integrated platform that intertwines digital marketing and customer rewards, converting conventional consumers into active nano-influencers.

Navigating the Growing Crisis of Space Debris

Orbital Fallout: Navigating the Growing Crisis of Space Debris

Ever since we stepped into the space age back in the 1950s, we’ve been busily sending rockets and satellites up, up and away, painting our mark way beyond our planetary borders. But here’s the thing—our ventures have left the cosmos littered with about 2,000 operational satellites and another 3,000 that are just space junk now.

Pryon Secures US$100 Million in Series B to Advance AI in Knowledge Management

Pryon Secures US$100 Million in Series B to Advance AI in Knowledge Management

Pryon Inc., a North Carolina-based company specializing in integrating artificial intelligence (AI) with knowledge management, has completed a Series B investment round, raising US$100 million. The funding was led by Thomas Tull’s US Innovative Technology Fund (USIT), with contributions from both new and existing investors, including Aperture Venture Capital,

Amazon Launches Upgraded AI Enhanced Fire TV Sticks

Amazon Launches Upgraded AI Enhanced Fire TV Sticks

Amazon has rolled out a series of updates to its Fire TV offerings. The new features include an improved conversational voice search powered by generative AI and Fire TV Ambient Experience advancements. Among the latest hardware releases are the Fire TV Stick 4K Max and the Fire TV Stick 4K, incorporating the enhanced Fire TV Ambient Experience.

Interactive Learning with Augmented Reality

Interactive Learning with Augmented Reality: Applications, Benefits and Challenges

Ever wondered what it would be like if your textbooks could talk, if the illustrations in your lessons could come to life, or if you could step into history rather than just read about it? Welcome to the world of augmented reality (AR) in education! Like a magic wand, AR can turn the abstract into tangible and the mundane into extraordinary, unfolding boundless educational possibilities.