779 lines
24 KiB
Rust
779 lines
24 KiB
Rust
//! Automated moderation for call messages.
|
|
//!
|
|
//! # Rules
|
|
//!
|
|
//! Payphone censors, allows, or blocks messages based on a predefined set of rule categories:
|
|
//!
|
|
//! | Nº | Message Type | Examples | Action |
|
|
//! |:--:|-----------------------------------------|-------------------------------------------------------|-----------------------------|
|
|
//! | 1 | Hate speech and slurs | "nigga", "faggot", "retard", "dumbfuck", racial slurs | Block immediately + penalty |
|
|
//! | 2 | Threats, self harm | "kill yourself", "die" | Block immediately + penalty |
|
|
//! | 3 | Targeted harassment | "kill @user", "die @user" | Block immediately + penalty |
|
|
//! | 4 | Explicit sexual language | "fuck you", "dick", "pussy" | Block immediately + penalty |
|
|
//! | 5 | Suggestive language | "slap your ass", "sexy" | Block immediately + penalty |
|
|
//! | 6 | Advertising | "discord.gg/payphone" | Block immediately + penalty |
|
|
//! | 7 | Sensitive topics and triggering content | "blood", "knife", "suicide method" | Block immediately + penalty |
|
|
//! | 8 | Non-english chats | | Soft censor + warn |
|
|
//! | 9 | E-dating | "let me hit" | Block immediately + penalty |
|
|
//! | 10 | Mild swear words | "damn", "shit", "ass" | Soft censor + warn |
|
|
//! | 11 | Casual insults and teasing | "stupid", "dumb" | Soft censor + warn |
|
|
//! | 12 | Links | "https://google.com/" | Soft censor + warn |
|
|
//! | 13 | Spam and flooding | | Soft censor + warn |
|
|
//! | 14 | Normal conversations | "hey", "lol", "cool" | Allowed freely |
|
|
//!
|
|
//! # Rule-breaking Detection
|
|
//!
|
|
//! Rule-breaking detection is implemented individually per rule via the [`Rule`] trait. This trait
|
|
//! defines a set of methods for defining and detecting rules, together with a set of utilities
|
|
//! used by these detectors to increase their effectiveness.
|
|
//!
|
|
//! Some rules, like the advertising and the no links ones, implement link searching via
|
|
//! [`linkify`], a crate to precisely identify and extracts links and another types of PII from
|
|
//! strings.
|
|
//!
|
|
//! The rest of the rules mostly implement the same normalization algorithm, which normalizes text
|
|
//! into a consistent text and whitespaces format that can be easily parsed searching for blocked
|
|
//! keywords.
|
|
//!
|
|
//! The normalization process is made of the following steps:
|
|
//!
|
|
//! 1. Replace common leet code with their textual representation (e.g. 3 -> E, 4 -> A).
|
|
//! 2. Pass the message through [`unidecode`], which normalizes all messages into their English
|
|
//! ASCII representation. This also unaccents all text, so `café` becomes `cafe`.
|
|
//! 3. Remove non-alphanumeric characters remaining.
|
|
//! 4. Pass the message through each of the rules implementations' detection algorithms.
|
|
//!
|
|
//! Each rule can have its own list of blocked keywords, which you can find in text files under
|
|
//! `src/groups/calls/lists/*.txt`. Each new line in those files represents a different keyword or
|
|
//! phrase that is blocked, except for comments which are preceeded with `//` at the start of the
|
|
//! line.
|
|
//!
|
|
//! Once that file is parsed, keywords are converted into a regex pattern that can be used to
|
|
//! accurately detect blocked keywords. The regex is generated as follows:
|
|
//!
|
|
//! 1. For each keyword, remove all whitespaces and split it into characters.
|
|
//! 2. To each character, append a `+\s*`, except for the last character which only gets a `+`
|
|
//! appended.
|
|
//! 3. After joining those characters and their suffixes together, put them between `(\b` and
|
|
//! `\b)`.
|
|
//! 4. Chain all parenthesized sub-regexes with `|`.
|
|
//!
|
|
//! This generates a quite effective regex, which for `hello` and `good bye` would look as follows:
|
|
//!
|
|
//! ```regex
|
|
//! (\bh+\s*e+\s*l+\s*l+\s*o+\b)|(\bg+\s*o+\s*o+\s*d+\s*\b\s*b+\s*y+\s*e+\b)
|
|
//! ```
|
|
//!
|
|
//! That regex can be matched against the previously normalized message to find all instances of
|
|
//! censored keywords and phrases.
|
|
//!
|
|
//! Note that in some cases, people may attempt to *remove* letters or write variations of the word
|
|
//! to bypass the censorship. In these cases, the lists should try to contain as many alternatives
|
|
//! as possible. E.g. for "shit", the list could also include "sht".
|
|
//!
|
|
//! This algorithm effectively blocks a lot of attempts to bypass censorship. For example, for the
|
|
//! word "shit", a malicious user could attempt to say:
|
|
//!
|
|
//! - "shit"
|
|
//! - "sh7"
|
|
//! - "sh1t"
|
|
//! - "sshhiitt"
|
|
//! - "s'h.ii i1 i1i!1.t t?"
|
|
//! - "Sh.í1.7"
|
|
//!
|
|
//! All of which would be detected.
|
|
|
|
use std::{collections::HashMap, fmt::Debug, sync::Arc};
|
|
|
|
use itertools::Itertools;
|
|
use linkify::LinkFinder;
|
|
use once_cell::sync::Lazy;
|
|
use regex::Regex;
|
|
use twilight_model::channel::Message;
|
|
use unicode_segmentation::UnicodeSegmentation;
|
|
use unidecode::unidecode;
|
|
|
|
use crate::discord::sanitizing::SanitizeDiscordMarkdown;
|
|
|
|
/// Escapes a call message by converting all mentions into user-readable ones, breaking @everyone
|
|
/// and @here pings, and censoring by rules.
|
|
///
|
|
/// Arguments:
|
|
/// * `event`: The incoming message event to escape.
|
|
///
|
|
/// Returns:
|
|
/// [`(String, Vec<Arc<dyn Rule>>)`] - The escaped and censored message, and a vector of the rules
|
|
/// broken.
|
|
pub fn escape_message(event: &Message) -> (String, Vec<Arc<dyn Rule>>) {
|
|
let mut message = event.content.to_string();
|
|
|
|
for mention in &event.mentions {
|
|
message = message.replace(
|
|
&format!("<@{}>", mention.id),
|
|
&format!("**@{}**", mention.name.sanitize(true)),
|
|
);
|
|
}
|
|
|
|
for mention in &event.mention_roles {
|
|
message = message.replace(&format!("<@&{mention}>"), "**@Role**");
|
|
}
|
|
|
|
for mention in &event.mention_channels {
|
|
message = message.replace(
|
|
&format!("<#{}>", mention.id),
|
|
&format!("**#{}**", mention.name.sanitize(true)),
|
|
);
|
|
}
|
|
|
|
let (censored, rules) = moderate_message(&message);
|
|
|
|
log::debug!("Escaped and normalized message: {censored:?}");
|
|
log::debug!("Rules broken: {rules:?}");
|
|
|
|
(censored, rules)
|
|
}
|
|
|
|
/// A normalized message and its mappings to the original message's individual grapheme clusters.
|
|
#[derive(Debug)]
|
|
pub struct NormalizedMessage {
|
|
/// The normalized message content.
|
|
message: String,
|
|
|
|
/// Mappings from normalized grapheme indices to the original message's grapheme indices
|
|
/// (indices are grapheme indices). Each item in the vector corresponds to its character in the
|
|
/// normalized string by index.
|
|
///
|
|
/// For example:
|
|
///
|
|
/// h3lll0 normalized to hello would have a mapping of `vec![0, 1, 2, 3, 5]`. The extra L was
|
|
/// removed in the normalization so it doesn't appear in the indexes list.
|
|
mappings: Vec<usize>,
|
|
}
|
|
|
|
/// Generates a normalized message out of the original message string.
|
|
///
|
|
/// TODO: Replace * in the message with wildcards to match censored words.
|
|
///
|
|
/// Arguments:
|
|
/// * `message`: The original message to normalize.
|
|
///
|
|
/// Returns:
|
|
/// [`NormalizedMessage`] - The normalized message and its mappings to the original string.
|
|
pub fn normalize(message: String) -> NormalizedMessage {
|
|
let replacements: HashMap<char, char> = [
|
|
('$', 's'),
|
|
('@', 'a'),
|
|
('0', 'o'),
|
|
('1', 'i'),
|
|
('2', 'z'),
|
|
('3', 'e'),
|
|
('4', 'a'),
|
|
('5', 's'),
|
|
('6', 'b'),
|
|
('7', 't'),
|
|
('8', 'g'),
|
|
('9', 'q'),
|
|
]
|
|
.iter()
|
|
.cloned()
|
|
.collect();
|
|
|
|
let mut normalized = String::with_capacity(message.len());
|
|
let mut mappings = Vec::new();
|
|
|
|
for (i, mut ch) in message
|
|
.grapheme_indices(true)
|
|
.map(|(i, ch)| (i, ch.to_string()))
|
|
{
|
|
if let Some(&replacement) = replacements.get(&ch.chars().next().unwrap()) {
|
|
ch = replacement.to_string();
|
|
}
|
|
|
|
// Each character is unidecoded individually to be able to keep track of what went where.
|
|
let unidecoded = unidecode(&ch);
|
|
|
|
for char in unidecoded.chars() {
|
|
if char.is_alphanumeric() {
|
|
normalized.push(char);
|
|
} else {
|
|
normalized.push(' ');
|
|
}
|
|
|
|
mappings.push(i);
|
|
}
|
|
}
|
|
|
|
NormalizedMessage {
|
|
message: normalized.to_ascii_lowercase(),
|
|
mappings,
|
|
}
|
|
}
|
|
|
|
/// Generates the regex described in the [`self`] module.
|
|
///
|
|
/// 1. Splits the phrase into individual characters.
|
|
/// 2. Joins them with `+\s*`, which matches the previous character any amount of time and any
|
|
/// whitespaces after it.
|
|
/// 3. Appends `+` to the last character, which matches it any amount of times.
|
|
/// 4. Wraps the whole regex in `\b` word boundaries.
|
|
///
|
|
/// For example:
|
|
/// "hello" -> `\bh+\s*e+\s*l+\s*l+\s*o+\b`
|
|
///
|
|
/// Arguments:
|
|
/// * `phrase` - The phrase to generate the regex for.
|
|
///
|
|
/// Returns:
|
|
/// [`String`] - The generated regex for the phrase. It does not return a [`Regex`] instance so
|
|
/// that it can be easily merged with other words in [`regex_from_phrases_list`].
|
|
fn regex_from_phrase(phrase: &str) -> String {
|
|
let mut pattern = String::new();
|
|
|
|
for (i, ch) in phrase.replace(" ", "").chars().enumerate() {
|
|
if i == phrase.len() - 1 {
|
|
pattern.push_str(&format!("{ch}+"));
|
|
} else {
|
|
pattern.push_str(&format!("{ch}+\\s*"));
|
|
}
|
|
}
|
|
|
|
let pattern = format!("\\b{pattern}\\b");
|
|
|
|
pattern
|
|
}
|
|
|
|
/// Merges multiple [`regex_from_phrase`]-generated regexes into a single regex.
|
|
///
|
|
/// Arguments:
|
|
/// * `phrases` - A slice of phrases to generate the regex for.
|
|
///
|
|
/// Returns:
|
|
/// [`Regex`] - The merged regex for the phrases.
|
|
fn regex_from_phrases_list(phrases: &[String]) -> Regex {
|
|
let mut pattern = String::new();
|
|
|
|
for phrase in phrases {
|
|
if !pattern.is_empty() {
|
|
pattern.push('|');
|
|
}
|
|
|
|
pattern.push_str(&format!("({})", regex_from_phrase(phrase)));
|
|
}
|
|
|
|
Regex::new(&pattern).expect("Failed to compile regex")
|
|
}
|
|
|
|
/// The results of the analysis of a rule, including all detections (on the original message first,
|
|
/// on the normalized message later) and the associated rule.
|
|
type RuleAnalysis = ((Vec<String>, Vec<String>), Arc<dyn Rule>);
|
|
|
|
/// Scans all rules on the message and returns the results for each rule which gave detections.
|
|
///
|
|
/// Arguments:
|
|
/// * `original` - The original message without normalization.
|
|
/// * `normalized` - The normalized version of the message.
|
|
///
|
|
/// Returns:
|
|
/// [`Vec<RuleAnalysis>`] - The results of the analysis, including all detections (on the original
|
|
/// message first, on the normalized message later) and their associated rules.
|
|
pub fn analyze_message(original: &str, normalized: &str) -> Vec<RuleAnalysis> {
|
|
let mut results = Vec::new();
|
|
|
|
for rule in RULES.iter() {
|
|
let (original_detections, normalized_detections) = rule.detect(original, normalized);
|
|
if !normalized_detections.is_empty() || !original_detections.is_empty() {
|
|
results.push(((original_detections, normalized_detections), rule.clone()));
|
|
}
|
|
}
|
|
|
|
results
|
|
}
|
|
|
|
/// Replaces all the found rule breakings in a message with `[censored]`.
|
|
///
|
|
/// Arguments:
|
|
/// * `original` - The original message without normalization.
|
|
/// * `normalized` - The normalized version of the message.
|
|
/// * `analysis` - The results of the analysis, including all detections and their associated
|
|
/// rules. This is generated by [`analyze_message`].
|
|
///
|
|
/// Returns:
|
|
/// [`String`] - The censored version of the original message.
|
|
pub fn censor_message(
|
|
original: &str,
|
|
normalized: NormalizedMessage,
|
|
analysis: &[RuleAnalysis],
|
|
) -> String {
|
|
let mut censored = original.to_string();
|
|
|
|
// Collect all ranges to censor
|
|
let mut ranges = Vec::new();
|
|
|
|
for ((original_findings, normalized_findings), _) in analysis {
|
|
for finding in normalized_findings {
|
|
for (index, _) in normalized.message.match_indices(finding) {
|
|
let start_index = normalized.mappings.get(index).cloned().unwrap_or(0);
|
|
let end_index = normalized
|
|
.mappings
|
|
.get(index + finding.len())
|
|
.cloned()
|
|
.unwrap_or(original.len());
|
|
|
|
ranges.push((start_index, end_index));
|
|
}
|
|
}
|
|
|
|
for finding in original_findings {
|
|
for (index, _) in original.match_indices(finding) {
|
|
let start_index = index;
|
|
let end_index = index + finding.len();
|
|
|
|
ranges.push((start_index, end_index));
|
|
}
|
|
}
|
|
}
|
|
|
|
// Sort ranges in reverse order to avoid shifting indices
|
|
ranges.sort_by(|a, b| b.0.cmp(&a.0));
|
|
|
|
for (start_index, end_index) in ranges {
|
|
if start_index < end_index && end_index <= censored.len() {
|
|
censored.replace_range(start_index..end_index, "[censored]");
|
|
}
|
|
}
|
|
|
|
censored
|
|
}
|
|
|
|
/// Normalizes, analyzes, and censors the original message, returning all rules broken in the
|
|
/// message.
|
|
///
|
|
/// Arguments:
|
|
/// * `original` - The original message.
|
|
///
|
|
/// Returns:
|
|
/// [`(String, Vec<Arc<dyn Rule>>)`] - The censored message and the rules that were broken.
|
|
pub fn moderate_message(original: &str) -> (String, Vec<Arc<dyn Rule>>) {
|
|
let normalized = normalize(original.to_string());
|
|
|
|
let analysis = analyze_message(original, &normalized.message);
|
|
let censored = censor_message(original, normalized, &analysis);
|
|
|
|
(censored, analysis.iter().map(|(_, r)| r.clone()).collect())
|
|
}
|
|
|
|
/// The action to be taken when a rule is broken.
|
|
#[allow(dead_code)]
|
|
#[derive(Debug, PartialEq, Eq, Clone, Copy)]
|
|
pub enum Action {
|
|
/// The message is not allowed to go through, the user must be penalized.
|
|
BlockAndPenalty,
|
|
|
|
/// The message's blocked phrases are censored with `[censored]` but the message goes through.
|
|
/// If the user repeats this multiple times, they may be warned about this.
|
|
///
|
|
/// TODO: Collect information about this type of action in the DB to be able to warn the user.
|
|
SoftCensorAndWarn,
|
|
}
|
|
|
|
/// Trait implemented by all rules.
|
|
#[allow(dead_code)]
|
|
pub trait Rule: Send + Sync + Debug {
|
|
/// The rule's human-readable name.
|
|
fn name(&self) -> &'static str;
|
|
|
|
/// The action that must be taken when the rule is broken.
|
|
fn action(&self) -> &'static Action;
|
|
|
|
/// Detects rule violations in the given message.
|
|
///
|
|
/// It takes both the original and normalized version of the message because some rules may
|
|
/// need the unchanged version to detect strict patterns (e.g. URLs) while some others will
|
|
/// need the normalized version to detect banned keywords and phrases.
|
|
///
|
|
/// Arguments:
|
|
/// * `original` - The original message.
|
|
/// * `normalized` - The normalized version of the message.
|
|
///
|
|
/// Returns:
|
|
/// [`(Vec<String>, Vec<String>)`] - The list of detected rule violations in the original
|
|
/// (first vector) and normalized (second vector) message.
|
|
fn detect(&self, original: &str, normalized: &str) -> (Vec<String>, Vec<String>);
|
|
}
|
|
|
|
/// Parses a file containing phrases to be detected by rules.
|
|
///
|
|
/// Arguments:
|
|
/// * `file_content` - The content of the file to parse.
|
|
///
|
|
/// Returns:
|
|
/// [`(Vec<String>, Vec<String>)`] - The list of phrases found in the file.
|
|
fn parse_phrases_file(file_content: &str) -> (Vec<String>, Vec<String>) {
|
|
let mut banned = vec![];
|
|
let mut exceptions = vec![];
|
|
|
|
for line in file_content.lines() {
|
|
let trimmed = line.trim().to_ascii_lowercase();
|
|
|
|
if !trimmed.is_empty() && !trimmed.starts_with("//") {
|
|
if let Some((phrase, phrase_exceptions)) = trimmed.split_once("/") {
|
|
banned.push(phrase.trim().to_string());
|
|
exceptions.extend(phrase_exceptions.split(',').map(|s| s.trim().to_string()));
|
|
} else {
|
|
banned.push(trimmed.to_string());
|
|
}
|
|
}
|
|
}
|
|
|
|
(banned, exceptions)
|
|
}
|
|
|
|
fn find_with_exceptions(message: &str, pattern: &Regex, exceptions: &[String]) -> Vec<String> {
|
|
pattern
|
|
.find_iter(message)
|
|
.filter_map(|m| {
|
|
let string = message[m.start()..m.end()].to_string();
|
|
|
|
if exceptions.contains(
|
|
&string
|
|
.split_ascii_whitespace()
|
|
.join(" ")
|
|
.to_ascii_lowercase(),
|
|
) {
|
|
None
|
|
} else {
|
|
Some(string)
|
|
}
|
|
})
|
|
.collect()
|
|
}
|
|
|
|
#[derive(Debug)]
|
|
struct HateSpeechAndSlursRule {
|
|
pattern: Regex,
|
|
exceptions: Vec<String>,
|
|
}
|
|
|
|
impl HateSpeechAndSlursRule {
|
|
fn new() -> Self {
|
|
let (phrases, exceptions) =
|
|
parse_phrases_file(include_str!("lists/hate-speech-and-slurs.txt"));
|
|
|
|
let regex = regex_from_phrases_list(&phrases);
|
|
|
|
Self {
|
|
pattern: regex,
|
|
exceptions,
|
|
}
|
|
}
|
|
}
|
|
|
|
impl Rule for HateSpeechAndSlursRule {
|
|
fn name(&self) -> &'static str {
|
|
"Hate Speech and Slurs"
|
|
}
|
|
|
|
fn action(&self) -> &'static Action {
|
|
&Action::BlockAndPenalty
|
|
}
|
|
|
|
fn detect(&self, _original: &str, normalized: &str) -> (Vec<String>, Vec<String>) {
|
|
(
|
|
vec![],
|
|
find_with_exceptions(normalized, &self.pattern, &self.exceptions),
|
|
)
|
|
}
|
|
}
|
|
|
|
#[derive(Debug)]
|
|
struct ThreatsAndSelfHarmRule {
|
|
pattern: Regex,
|
|
exceptions: Vec<String>,
|
|
}
|
|
|
|
impl ThreatsAndSelfHarmRule {
|
|
fn new() -> Self {
|
|
let (phrases, exceptions) =
|
|
parse_phrases_file(include_str!("lists/threats-and-self-harm.txt"));
|
|
|
|
let regex = regex_from_phrases_list(&phrases);
|
|
|
|
Self {
|
|
pattern: regex,
|
|
exceptions,
|
|
}
|
|
}
|
|
}
|
|
|
|
impl Rule for ThreatsAndSelfHarmRule {
|
|
fn name(&self) -> &'static str {
|
|
"Threats and Self Harm"
|
|
}
|
|
|
|
fn action(&self) -> &'static Action {
|
|
&Action::BlockAndPenalty
|
|
}
|
|
|
|
fn detect(&self, _original: &str, normalized: &str) -> (Vec<String>, Vec<String>) {
|
|
(
|
|
vec![],
|
|
find_with_exceptions(normalized, &self.pattern, &self.exceptions),
|
|
)
|
|
}
|
|
}
|
|
|
|
#[derive(Debug)]
|
|
struct ExplicitSexualLanguageRule {
|
|
pattern: Regex,
|
|
exceptions: Vec<String>,
|
|
}
|
|
|
|
impl ExplicitSexualLanguageRule {
|
|
fn new() -> Self {
|
|
let (phrases, exceptions) =
|
|
parse_phrases_file(include_str!("lists/explicit-sexual-language.txt"));
|
|
|
|
let regex = regex_from_phrases_list(&phrases);
|
|
|
|
Self {
|
|
pattern: regex,
|
|
exceptions,
|
|
}
|
|
}
|
|
}
|
|
|
|
impl Rule for ExplicitSexualLanguageRule {
|
|
fn action(&self) -> &'static Action {
|
|
&Action::BlockAndPenalty
|
|
}
|
|
|
|
fn name(&self) -> &'static str {
|
|
"Explicit Sexual Language"
|
|
}
|
|
|
|
fn detect(&self, _original: &str, normalized: &str) -> (Vec<String>, Vec<String>) {
|
|
(
|
|
vec![],
|
|
find_with_exceptions(normalized, &self.pattern, &self.exceptions),
|
|
)
|
|
}
|
|
}
|
|
|
|
#[derive(Debug)]
|
|
struct SuggestiveLanguageRule {
|
|
pattern: Regex,
|
|
exceptions: Vec<String>,
|
|
}
|
|
|
|
impl SuggestiveLanguageRule {
|
|
fn new() -> Self {
|
|
let (phrases, exceptions) =
|
|
parse_phrases_file(include_str!("lists/suggestive-language.txt"));
|
|
|
|
let regex = regex_from_phrases_list(&phrases);
|
|
|
|
Self {
|
|
pattern: regex,
|
|
exceptions,
|
|
}
|
|
}
|
|
}
|
|
|
|
impl Rule for SuggestiveLanguageRule {
|
|
fn action(&self) -> &'static Action {
|
|
&Action::BlockAndPenalty
|
|
}
|
|
|
|
fn name(&self) -> &'static str {
|
|
"Suggestive Language"
|
|
}
|
|
|
|
fn detect(&self, _original: &str, normalized: &str) -> (Vec<String>, Vec<String>) {
|
|
(
|
|
vec![],
|
|
find_with_exceptions(normalized, &self.pattern, &self.exceptions),
|
|
)
|
|
}
|
|
}
|
|
|
|
#[derive(Debug)]
|
|
struct AdvertisingRule;
|
|
|
|
impl Rule for AdvertisingRule {
|
|
fn name(&self) -> &'static str {
|
|
"Advertising"
|
|
}
|
|
|
|
fn action(&self) -> &'static Action {
|
|
&Action::BlockAndPenalty
|
|
}
|
|
|
|
fn detect(&self, original: &str, _normalized: &str) -> (Vec<String>, Vec<String>) {
|
|
let mut detections = Vec::new();
|
|
|
|
for link in LinkFinder::new()
|
|
.url_must_have_scheme(false)
|
|
.links(original)
|
|
{
|
|
let link_str = link.as_str();
|
|
|
|
if link_str.contains("discord.gg") || link_str.contains("discord.com") {
|
|
detections.push(link_str.to_string());
|
|
}
|
|
}
|
|
|
|
(detections, vec![])
|
|
}
|
|
}
|
|
|
|
#[derive(Debug)]
|
|
struct SensitiveTopicsAndTriggeringContentRule {
|
|
pattern: Regex,
|
|
exceptions: Vec<String>,
|
|
}
|
|
|
|
impl SensitiveTopicsAndTriggeringContentRule {
|
|
fn new() -> Self {
|
|
let (phrases, exceptions) = parse_phrases_file(include_str!(
|
|
"lists/sensitive-topics-and-triggering-content.txt"
|
|
));
|
|
|
|
let regex = regex_from_phrases_list(&phrases);
|
|
|
|
Self {
|
|
pattern: regex,
|
|
exceptions,
|
|
}
|
|
}
|
|
}
|
|
|
|
impl Rule for SensitiveTopicsAndTriggeringContentRule {
|
|
fn name(&self) -> &'static str {
|
|
"Sensitive Topics and Triggering Content"
|
|
}
|
|
|
|
fn action(&self) -> &'static Action {
|
|
&Action::SoftCensorAndWarn
|
|
}
|
|
|
|
fn detect(&self, _original: &str, normalized: &str) -> (Vec<String>, Vec<String>) {
|
|
(
|
|
vec![],
|
|
find_with_exceptions(normalized, &self.pattern, &self.exceptions),
|
|
)
|
|
}
|
|
}
|
|
|
|
#[derive(Debug)]
|
|
pub struct MildSwearWordsRule {
|
|
pattern: Regex,
|
|
exceptions: Vec<String>,
|
|
}
|
|
|
|
impl MildSwearWordsRule {
|
|
fn new() -> Self {
|
|
let (phrases, exceptions) = parse_phrases_file(include_str!("lists/mild-swear-words.txt"));
|
|
|
|
let regex = regex_from_phrases_list(&phrases);
|
|
|
|
Self {
|
|
pattern: regex,
|
|
exceptions,
|
|
}
|
|
}
|
|
}
|
|
|
|
impl Rule for MildSwearWordsRule {
|
|
fn name(&self) -> &'static str {
|
|
"Mild Swear Words"
|
|
}
|
|
|
|
fn action(&self) -> &'static Action {
|
|
&Action::SoftCensorAndWarn
|
|
}
|
|
|
|
fn detect(&self, _original: &str, normalized: &str) -> (Vec<String>, Vec<String>) {
|
|
(
|
|
vec![],
|
|
find_with_exceptions(normalized, &self.pattern, &self.exceptions),
|
|
)
|
|
}
|
|
}
|
|
|
|
#[derive(Debug)]
|
|
pub struct CasualInsultsAndTeasing {
|
|
pattern: Regex,
|
|
exceptions: Vec<String>,
|
|
}
|
|
|
|
impl CasualInsultsAndTeasing {
|
|
fn new() -> Self {
|
|
let (phrases, exceptions) =
|
|
parse_phrases_file(include_str!("lists/casual-insults-and-teasing.txt"));
|
|
|
|
let regex = regex_from_phrases_list(&phrases);
|
|
|
|
Self {
|
|
pattern: regex,
|
|
exceptions,
|
|
}
|
|
}
|
|
}
|
|
|
|
impl Rule for CasualInsultsAndTeasing {
|
|
fn name(&self) -> &'static str {
|
|
"Casual Insults and Teasing"
|
|
}
|
|
|
|
fn action(&self) -> &'static Action {
|
|
&Action::SoftCensorAndWarn
|
|
}
|
|
|
|
fn detect(&self, _original: &str, normalized: &str) -> (Vec<String>, Vec<String>) {
|
|
(
|
|
vec![],
|
|
find_with_exceptions(normalized, &self.pattern, &self.exceptions),
|
|
)
|
|
}
|
|
}
|
|
#[derive(Debug)]
|
|
struct LinksRule;
|
|
|
|
impl Rule for LinksRule {
|
|
fn name(&self) -> &'static str {
|
|
"Links"
|
|
}
|
|
|
|
fn action(&self) -> &'static Action {
|
|
&Action::SoftCensorAndWarn
|
|
}
|
|
|
|
fn detect(&self, original: &str, _normalized: &str) -> (Vec<String>, Vec<String>) {
|
|
let mut detections = Vec::new();
|
|
|
|
for link in LinkFinder::new()
|
|
.url_must_have_scheme(false)
|
|
.links(original)
|
|
{
|
|
detections.push(link.as_str().to_string());
|
|
}
|
|
|
|
(detections, vec![])
|
|
}
|
|
}
|
|
|
|
/// The list of moderation rules.
|
|
pub static RULES: Lazy<Vec<Arc<dyn Rule>>> = Lazy::new(|| {
|
|
vec![
|
|
Arc::new(HateSpeechAndSlursRule::new()),
|
|
Arc::new(ThreatsAndSelfHarmRule::new()),
|
|
Arc::new(ExplicitSexualLanguageRule::new()),
|
|
Arc::new(SuggestiveLanguageRule::new()),
|
|
Arc::new(AdvertisingRule),
|
|
Arc::new(SensitiveTopicsAndTriggeringContentRule::new()),
|
|
Arc::new(MildSwearWordsRule::new()),
|
|
Arc::new(CasualInsultsAndTeasing::new()),
|
|
Arc::new(LinksRule),
|
|
]
|
|
});
|