Key Highlights:
- AI-Powered Hate-Speech Detection: Shhor AI is an innovative content moderation API designed by Aindriya Barua, a queer engineer, to combat hate speech online, especially in Indian contexts. It focuses on Hinglish (Hindi + English) and targets various forms of hate, including gendered hate, racism, casteism, and queerphobia.
- Community-Driven Moderation: Barua’s solution addresses gaps in centralized moderation by relying on crowd-sourced datasets and regional knowledge. The AI bot has been integrated into Reddit, where it flags hate speech, issues public warnings, and bans repeat offenders while allowing human moderators to review disputed cases.
- Award-Winning Impact: Shhor AI was awarded the ‘AI for Social Good’ prize at the Global AI Summit 2024 for its ability to protect marginalized communities. It promotes decentralization in content moderation, encouraging platforms to adopt region-specific tools to create a safer internet.
With hate speech on the rise, especially targeting vulnerable communities, innovative solutions are crucial. One such effort comes from Aindriya Barua, a queer artist and AI engineer, who has created Shhor AI, a powerful content moderation tool designed to detect and combat hate speech online, with a special focus on the Indian context. This groundbreaking API identifies hate speech and ensures the internet becomes a safer place for marginalized communities by addressing the specific nuances of local languages and cultures.
The Story Behind Shhor AI
Aindriya Barua, who identifies as non-binary and prefers they/them pronouns, describes their journey of growing up in rural Tripura as challenging. “Growing up, I struggled to fit in because of my marginalized identity, but I couldn’t fully understand why I was treated differently,” shares Barua. Their life experiences, combined with the discrimination they faced as a queer artist, were the driving forces behind Shhor AI. “When your lived experiences are rooted in marginalization, your art naturally becomes political, you don’t have to choose it,” Barua says, highlighting how activism and art intertwined for them.
Barua’s experiences with online hate and abuse made them realize that discrimination extends beyond real life into the virtual world. This realization sparked the idea for Shhor AI, a solution aimed at protecting users from online harassment. Barua’s AI journey began during their college years, when they studied Natural Language Processing (NLP), a branch of AI that processes and interprets human language.
Tackling Hate Speech with Shhor AI
Shhor AI is not limited to a specific platform but can be integrated into different online spaces to detect hate speech. The tool recently made headlines by winning the Just AI Award 2024 in the ‘AI for Social Good’ category at the Global AI Summit in Hyderabad. Recognizing the gaps in traditional moderation systems, Barua built a custom dataset by crowdsourcing comments and messages from the community to train the model. “We need moderation tools that are aware of local contexts, both historical and present-day,” explains Barua, emphasizing that centralized platforms often fail to grasp the cultural nuances required to detect regional hate speech effectively.
Shhor AI was first deployed on Reddit, where it functions as a bot that monitors specific subreddits in real-time. This bot issues warnings, tracks repeat offenders, and bans users after three warnings. Users who feel they were wrongly banned can appeal to a human moderator, ensuring fairness. This dynamic approach, where AI and human moderators work together, ensures effective and unbiased content moderation. “The Reddit API gave us the flexibility to automate moderation while allowing human input where necessary,” Barua notes.
Addressing the Challenges of Indian Hate Speech
The complexity of Indian languages and Hinglish (Hindi + English) presented a unique challenge for Barua. “There is no fixed spelling for words in Hindi when typed in English, so there cannot be an exact list of slurs,” Barua explains. Shhor AI focuses on detecting hate speech in Hinglish, a space that most global content moderation tools overlook. This makes the tool particularly effective for Indian social media, where users express themselves through regional languages typed using English keyboards.
Barua emphasizes the importance of localization in content moderation: “You can report a comment in a regional language, but platforms like Instagram won’t remove it because it doesn’t violate their global standards.” Shhor AI fills this gap by taking a decentralized approach, encouraging platforms to use third-party moderation tools tailored to regional and cultural contexts.
The Impact of Shhor AI on Online Communities
Shhor AI targets eight categories of hate speech: gendered hate, racism, political hate, communalism, queerphobia, casteism, ableism, and general hate. According to Barua, the most frequent targets of online hate are gender and sexual minorities. “Forty percent of hate speech is directed at gender and sexual minorities,” they report, followed by activists and political figures at 23%. The tool also analyzed election-related hate speech and found a significant intersection between communal and sexist hate, especially during election seasons.
In addition to Reddit, Shhor AI aims to expand to other platforms and moderate images, videos, and short-form content like YouTube Shorts and Instagram Reels. By identifying and flagging hateful content in multiple formats, Shhor AI ensures that online spaces remain safer for everyone, especially those belonging to marginalized communities.
A Decentralized Future for Content Moderation
Barua believes that the future of content moderation lies in decentralization, much like Meta’s use of third-party fact-checkers. “We need platforms to open their systems to external tools like Shhor AI, which are tailored to local contexts,” they suggest. This flexibility is key to addressing the growing problem of online hate, which has become more prevalent with the rise of social media.
Shhor AI: Building a Safer Internet, One Platform at a Time
The ultimate goal of Shhor AI is to build an internet where everyone, especially those from marginalized communities, can feel safe and express themselves without fear of harassment. As Barua puts it, “The goal is to create an internet that’s safe for everyone, especially those from marginalized communities.”
With hate speech on social media increasing by 45% last year, as reported by the National Crime Records Bureau (NCRB), tools like Shhor AI are not just timely; they are essential. By bridging the gap between AI and cultural sensitivity, Shhor AI offers a lifeline to online communities seeking safety and acceptance in an increasingly connected world.
References: