Technology + Humanities, Douyin's anti-cyberbully management capabilities upgraded

Editor | Li Zongren

"Cyber ​​violence" has a long history. In the PC era more than a decade ago, the Internet had not yet penetrated into every aspect of people's lives like it does today, but the topic of cyber violence had already attracted people's attention and vigilance. With the advancement of technology and the advent of the mobile Internet era, the number of "Internet users" has risen sharply in recent years, and the impact of cyber violence has become increasingly larger, and it has even gradually become a black industry.

Preventing cyber violence is imperative.

While mobile Internet has brought convenience to people's lives, it has also created a breeding ground for cyber violence. Many public online places such as social platforms and short video platforms have become hotbeds for cyber violence. Comments and private messages have also become the main means of cyber violence. As the saying goes, "the greater the ability, the greater the responsibility." In addition to public appeals, major online platforms should also assume the responsibility of helping users prevent cyber violence.

For example, during the recent hot Winter Olympics, many people used the comments and private messaging functions of online platforms to slander, insult and attack Winter Olympic athletes, and spread false rumors, causing great harm to the Winter Olympics viewing and community atmosphere. Not long ago, the short video platform Douyin banned or blocked 331 accounts that cyberbullied Winter Olympic athletes in response to the situation during the Winter Olympics.

In recent years, Douyin, one of the largest short video platforms in China, seems to have a very deep understanding of the saying "the greater the ability, the greater the responsibility": in the past year, Douyin's open courses, folk art, and rural-related video content have grown rapidly; it has also vigorously promoted adaptation to the elderly, with a dedicated hotline for the elderly; in addition, it has also made the most attentive youth mode. Leifeng.com (public account: Leifeng.com) has also made a horizontal comparison of the youth modes of video platforms on the market, and the results show that Douyin's products are the best overall.

Another area that Douyin has been focusing on for a long time is "preventing cyberbullying."

Just like the updated Douyin brand slogan "Record a beautiful life" in 2018, "everyone is both a content creator and a content consumer" on Douyin. Every day, along with the massive content creation and consumption, there are also a large number of comments and private messages. Douyin generates 800 million comments every day, of which "swearing and personal attacks" account for 32.79% of the processed videos.

If such a large amount of negative comments are only dealt with after the fact, on the one hand, the platform will face tremendous governance pressure. On the other hand, for users, online violence has already occurred and the trauma they have suffered will be difficult to heal.

Therefore, Douyin updated a series of functions such as user risk reminders and negative content filtering in January this year. Through 8 major functions, it became the first platform in China to provide cyberbullying prevention services in various scenarios. Starting from the two dimensions of comments and private messages, from active management of senders to self-management of receivers, from early comment sending warnings, to mid-term private message hierarchical management, and then to the platform assistant providing "mood care" in the later stage, the entire process of preventing cyberbullying has been opened up.

Leifeng.com also conducted a comprehensive evaluation of Tik Tok’s new anti-cyberbullying feature.

Leifeng.com's evaluation of Douyin's new anti-cyberbullying function focuses on two aspects. On the one hand, it evaluates the new permissions and functions for creators to see whether the new functions can directly bring a better experience to creators. On the other hand, from the perspective of humanistic care, it explores how Douyin tries to fundamentally reduce cyberbullying.

1. New permissions and features for creators

The most basic private message recipient limitation function in TikTok was launched as early as December 2017. Users can choose from three states: everyone/mutual friends/closed. It is the most thorough and basic private message filtering.

However, for "big Vs", traffic owners, commercial accounts and other users who need to deal with a large number of strangers and one-way followers, they cannot restrict the recipients of private messages in a one-size-fits-all manner, so these special groups often encounter problems such as advertising, verbal abuse, and harassment.

The "user risk reminder and negative content filtering" function launched at the end of January has greatly optimized the above-mentioned problems.

User risk reminders are the system guiding users to manage, identify messages from strangers or followers, and provide a pop-up window for "blacklisting/reporting/removing followers". Negative content filtering uses algorithms to automatically process, identify insulting/harassing messages, and automatically fold them into stranger spam messages.

According to official data from Douyin, after the function was launched, risk warnings were issued for an average of 250,000 conversations per day, nearly 90,000 messages were listed as spam per day, and more than 70,000 users benefited per day.

As for comments, Douyin has launched the option of "Who can comment". The default option is "Everyone", but there are four other options: "Everyone", "Only fans", "Only friends who follow each other", and "Only myself". This option can be selected for both new and old videos. It is simple, rough and effective.

2. Humanistic care for content consumers

The creators and consumers in TikTok are two sides of the same coin.

Many of the online bullies are from the vulnerable groups offline. They may bully others online because of psychological problems such as depression. Of course, this is not to excuse the bullies, but we need to pay attention to the problems faced by some groups of bullies and guide them back on track. If the anti-bullying function for creators is a "symptomatic treatment", then Douyin's guidance and assistance to users who violate the rules is an attempt to "cure the root cause".

After the upgrade of anti-cyberbullying measures, Douyin will now perform model matching and verification on comments. After posting illegal or low-quality comments, users will immediately receive a pop-up warning saying "this comment may have a negative impact on others", giving users a chance to change their minds.

Of course, you can skip the pop-up window and continue to post comments, but they will not be seen by others. At this time, the comments will enter the normal review logic, and real illegal comments will still be processed.

Douyin has added a platform assistant called "Mood Warmer" to its new anti-cyberbullying system. If a user repeatedly violates the rules by posting private messages or comments, the algorithm can automatically match and trigger the "Mood Warmer", using message conversations to try to relieve the user's depression, and even guide the user to seek manual help or offline medical treatment.

To be more specific, users will receive text messages like "Hello, we have noticed the content you recently posted on the site and are worried about your mental state. Although life is not perfect, we can still heal ourselves. If you want to talk to someone, you can choose to switch to a manual assistant during the XX time period. The assistant is willing to listen to your voice." In addition, users can also reduce negative content recommendations in recommendation, search, comment and other scenarios.

In fact, this is not the first time that Douyin has used algorithms to help vulnerable users. In 2021, Douyin also provided assistance to thousands of users with suicidal tendencies. In this regard, as a leading domestic platform, Douyin did not simply use technology and algorithms to deal roughly with cyber-bullying people, but on the basis of technology, it added more humanistic care and tried to solve the problem from the root. This is an approach worth learning for all large online platforms.

In the era of mobile Internet, cyber violence has become more frequent, and the cost of committing violence against others online has become lower and lower. These violent incidents are indelible pain for the victims. In some occasional public opinion incidents, countless perpetrators, without knowing the inside story, will be led by emotions and participate in large-scale cyber violence.

Over the years, there have been many calls to stop cyber violence, but with little success. In other words, it is unrealistic at this stage to simply hope that netizens can improve their self-awareness and quality to eliminate cyber violence. Therefore, whether it is TikTok or other online platforms, they should shoulder the responsibility of preventing cyber violence.

The anti-cyberbullying system upgraded by Douyin is a very good benchmark case. In terms of technology, it has achieved better protection measures for both video creators and ordinary users. In terms of humanistic care, it takes into account the psychological and behavioral possibilities of the perpetrators at a deeper level, thereby guiding more people to eliminate hostility and jointly build a good online social environment.

By using technology, starting with comments and private messages, we can supervise and guide users who have not yet entered the cyber violence stage, and prevent the occurrence of cyber violence incidents. This can better alleviate the governance pressure of the platform and protect victims from cyber violence. In addition, serious handling of accounts that have already committed cyber violence and establishing a good network ecological environment are what all platforms need to do in the future.

Douyin’s upgraded anti-cyberbullying system, on the one hand, reflects its technological capabilities and sense of responsibility as a leading short video platform; on the other hand, it also sets a good example for other platforms to use technology to manage cyberbullying in the future.