Quality Control
4 min
The Quality Control (QC) module is designed for platforms that rely on multiple moderators, enabling the continuous assessment and improvement of your team’s performance. The module ensures constant feedback, tracks performance, and is critical for maintaining consistent, high-quality moderation decisions at scale.
Key features
The QC process revolves around three core components:
- Sampling Strategy: A method to pull flags for review based on predefined criteria, ensuring the audit process is unbiased and representative.
- Audit by QC Experts: Moderators' original decisions are anonymously reviewed by specialized QC experts to determine accuracy.
- Quality Control Score: The outcome of each audit session produces a measurable score, used for performance tracking, moderator coaching, and identifying training needs.
Sampling strategy
The sampling strategy defines the criteria for which flags are pulled for review.
Sampling Strategy Setting
- Sampling Rule Configuration: QC Admins define rules for each queue, specifying parameters like the timeframe, content type distribution (e.g., 40% approved, 60% rejected), and which moderators' decisions should be sampled. Default rules are provided for immediate functionality.
- Sampling Rule Preview: A feature that allows QC Admins to validate their configuration by viewing the expected outcome (number of flags, content type distribution, moderators involved) before applying the rule.
- QC Queue: A dedicated subset of a moderation queue that contains sampled flags. There is a 1-to-1 relationship between a moderation queue and its corresponding QC queue.
Process flow
- Setup: QC Admins define and finalize the sampling strategy for a specific queue.
- Session Start: A QC Expert starts a QC session. The system fetches flags based on the defined sampling rule and pulls them into the QC Queue. QC Experts can pull flags into the queue at any time for flexibility. When a piece of content has several reports against it, all of its reports are pulled together by content ID in a single action, so the content is reviewed once rather than one report at a time.
- Quality Control Review: QC Experts review the cases in the QC Queue. Crucially, the original moderator’s decision is hidden during this review to maintain anonymity and prevent bias. The system also prevents multiple QC Experts from reviewing the same flag simultaneously.
- Scoring and Reporting: A Score is generated based on the matches and mismatches between the expert's decision and the original moderator's decision. This score is updated dynamically and displayed in real time for performance tracking. Reports are generated for management to evaluate overall policy effectiveness and identify training needs.
Moderator Score Display
User roles and permissions
- QC Expert: Start QC sessions. Review sampled flags and provide feedback. Cannot configure or update sampling rules.
- QC Admin: Configure and update sampling rules. Start QC sessions. Access all QC queues.
QC Permissions