/
Discord

Scalable Trust & Safety Tools for Discord

Project Overview

Discord, a leading communication platform with millions of active communities, needed robust, user-friendly internal tools to support its Trust & Safety (T&S) team. With moderation demands increasing, their existing admin panel was expensive to maintain and lacked the scalability to handle rapidly evolving user needs. Additionally, Discord sought to enhance its ML training workflows for identifying nefarious content and consolidate scattered data sources to enable swift operator action.

Over the course of multiple development cycles, Paramint collaborated with Discord’s T&S leaders to build over a dozen specialized tools—ranging from channel moderation and user management dashboards to subscription oversight and machine learning training interfaces. Powered by React, Retool, Snowflake, HTML/CSS, and Discord’s internal APIs, these solutions gave the T&S team comprehensive, real-time control over user safety, content oversight, and back-office admin operations.

Key Challenges

  1. Complex, High-Volume Moderation Needs
    • T&S staff must ban, suspend, or otherwise moderate at scale, often responding to hundreds of reports simultaneously across numerous servers.
    • Existing tools lacked cohesive integration with user data and channel metrics, forcing manual cross-referencing and repeated steps.
  2. Fragmented Data & Legacy Admin Panel
    • Discord’s previous admin console was expensive to maintain and not easily extensible for new features.
    • Data resided in Snowflake and various internal APIs, but there was no single, intuitive interface for operators to run bulk actions or retrieve historical reports.
  3. Machine Learning Training
    • As Discord expanded its automated detection of policy-violating content, the T&S team needed a robust platform to label, approve, and feed “nefarious content” examples to ML pipelines.
    • Manual processes slowed model training and risked inconsistencies in labeling criteria.
  4. Subscription & Membership Management
    • Additional revenue-related tools required oversight of user subscriptions and community membership tiers.
    • Needed a secure, streamlined method for T&S or admin operators to manage these functionalities without cluttering the moderation workflow.

Solution & Implementation

  1. Modular Internal Tools via Retool + React
    • Created multiple Retool applications embedded with React components for more complex logic.
    • Integrated seamlessly with Discord’s internal APIs for user data, channel details, and server metadata.
    • Clear UI for T&S operators to perform bulk bans, suspensions, or content takedowns with minimal clicks.
  2. Rebuilt Admin Panel
    • Replaced the legacy admin console with a modern React/Retool-based solution that drastically cut maintenance overhead.
    • Unified various data queries (Snowflake, internal microservices) into a single interface, giving operators real-time user status, violation history, and membership details.
  3. ML Training & Data Labeling Tools
    • Developed a dedicated dashboard for machine learning content classification tasks—allowing T&S staff to review and label suspicious text, images, or user behavior logs.
    • Automatic sync with the ML pipeline so labeled data immediately fed training sets, accelerating iteration cycles for model improvements.
  4. Snowflake Integration & Bulk Actions
    • Leveraged Snowflake for advanced queries on user behavior, historical violation logs, and subscription stats.
    • Bulk Action features let T&S quickly address widespread compliance issues, e.g., removing thousands of spam accounts or adjusting membership roles en masse.
  5. Subscription & Membership Oversight
    • Built an operator-friendly interface to manage paid subscription tiers, track membership usage, and handle escalations or refunds.
    • Ensured role-based permissions so that financial and T&S tasks remained appropriately siloed.

Outcomes & Impact

  1. Enhanced Moderation Efficiency
    • T&S operators now handle bulk user actions (ban, suspend, escalate) through a single platform—reducing manual overhead by an estimated 40%.
    • Real-time data syncing from Snowflake and internal APIs provides immediate feedback on user status and server analytics.
  2. Significant Cost & Maintenance Savings
    • Consolidating separate legacy tools into a Retool + React architecture lowered long-term support costs.
    • Discord’s internal teams can now easily add or modify T&S workflows without full-scale platform rebuilds.
  3. Faster Machine Learning Improvements
    • By providing a streamlined labeling environment, the data science team receives higher-quality training data on a continuous basis.
    • Model accuracy for detecting harmful content has risen, as more curated examples are fed into the pipeline quickly.
  4. Unified Subscription Management
    • Operators can handle membership queries, disputes, or role upgrades from the same console used for moderation—improving staff productivity and user satisfaction.
    • Cross-referencing financial data with user behaviors helps T&S identify fraudulent or abuse-related subscription patterns.
  5. Scalability & Flexibility
    • The modular nature of Retool applications and custom React components ensures Discord can rapidly spin up new T&S features as platform policies or user patterns evolve.
    • The robust data integrations future-proof the system against growing user numbers and expanding data sets.

Having a single, modern interface for so many complex tasks has been a game-changer for our Trust & Safety team. Paramint’s approach to modular tool-building was critical in helping us streamline everything from user bans to advanced ML training workflows. They made the upgrade from our outdated admin panel seamless, and now we have a cost-effective, scalable platform for supporting our community.

Product Manager, Trust & Safety at Discord

Conclusion

Paramint’s end-to-end solution for Discord’s T&S operations demonstrates the power of combining Retool, React, Snowflake, and robust internal API integrations into a coherent ecosystem. By modernizing the admin panel, enabling bulk moderation, and creating a dedicated ML labeling pipeline, Discord is now equipped to efficiently protect its massive user base, accelerate machine learning iterations, and adapt as new features and threats emerge.

Ready to partner with Paramint for 
your next digital initiative?