--- title: Quality Control slug: quality-control docTags: createdAt: 2025-02-28T09:14:12.922Z --- The *Quality Control (QC) *module is designed for platforms that rely on multiple moderators, enabling the continuous assessment and improvement of your team’s performance. The module ensures constant feedback, tracks performance, and is critical for maintaining consistent, high-quality moderation decisions at scale. ### Key features The QC process revolves around three core components: - **Sampling Strategy**: A method to pull flags for review based on predefined criteria, ensuring the audit process is unbiased and representative. - **Audit by QC Experts**: Moderators' original decisions are anonymously reviewed by specialized QC experts to determine accuracy. - **Quality Control Reports**: Precision, recall and accuracy for every queue, with a breakdown of errors by type, policy and moderator, used for performance tracking, moderator coaching, and identifying training needs. ### Sampling strategy The sampling strategy defines the criteria for which flags are pulled for review. ::Image[]{src="https://api.archbee.com/api/optimize/ijRmXIIrObJhIfPMa8rmJ/UvYEgPx9aACqsMrGOG8tf_image.png" size="48" width="678" height="490" position="center" caption="Sampling Strategy Setting" darkWidth="678" darkHeight="490" showCaption="true"} - **Sampling Rule Configuration:** QC Admins define rules for each queue, specifying parameters like the timeframe, content type distribution (e.g., 40% approved, 60% rejected), and which moderators' decisions should be sampled. Default rules are provided for immediate functionality. ::Image[]{src="https://api.archbee.com/api/optimize/ijRmXIIrObJhIfPMa8rmJ/nrg0voxMi3IopdYwUd3ls_image.png" size="58" width="1054" height="842" position="center" darkWidth="1054" darkHeight="842" showCaption="false"} - **Sampling Rule Preview:** A feature that allows QC Admins to validate their configuration by viewing the expected outcome (number of flags, content type distribution, moderators involved) before applying the rule. ::Image[]{src="https://api.archbee.com/api/optimize/ijRmXIIrObJhIfPMa8rmJ/SKrR-V_Mdk-3Ewo0KfN8u_image.png" size="68" width="1054" height="194" position="center" showCaption="false" darkWidth="1054" darkHeight="194"} - **Sampling queue:** A dedicated subset of a moderation queue that contains sampled cases. There is a 1-to-1 relationship between a moderation queue and its sampling queue. Each sampling queue appears as a card under **Quality Control > Sampling Queues**, showing how many of the pulled cases have been reviewed and the queues, rules, teams and users it samples. QC Admins edit or delete a sampling strategy from the card's **…** menu; deleting it stops new sampling but keeps previously reviewed cases and their results. ### Process flow 1. **Setup:** QC Admins define and finalize the sampling strategy for a specific queue. 2. **Session Start:** A QC Expert clicks **Pull cases** to fetch cases based on the defined sampling rule into the sampling queue (up to 100 per pull; a new pull is available once the current one is reviewed), then **Start QC** (or **Resume QC**) to review them. When a piece of content has several reports against it, all of its reports are pulled together by content ID in a single action, so the content is reviewed once rather than one report at a time. 3. **Quality Control Review:** QC Experts review the cases in the sampling queue. Crucially, the original moderator's decision is hidden during this review to maintain anonymity and prevent bias. The system also prevents multiple QC Experts from reviewing the same case simultaneously. 4. **Reporting**: Each QC verdict is compared with the original decision as a match or a mismatch. Results update in real time in **Quality Control Reports**, so management can evaluate policy effectiveness and identify training needs. A mismatch does not change the case itself. ### Quality Control Reports **Quality Control > Quality Control Reports** shows how each queue performs against the QC verdict. Filter the whole page by **date** (the QC review date), **Team** and **Moderator name** to focus on one team or one moderator. ::Image[]{src="https://api.archbee.com/api/optimize/ijRmXIIrObJhIfPMa8rmJ/3Gqsh7C8m8kH7rsI5-cKE_screenshot-2026-10-19-at-203448.png" size="60" isUploading="false" width="1279" height="787" darkWidth="1279" darkHeight="787" showCaption="false"} **Scores.** Each queue reports three scores side by side, under a line stating how many cases were reviewed out of the total handled: | Score | What it tells you | | ------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Precision** | Of the reviewed cases where a policy was applied, the share QC agreed were violations. Low precision means safe content is being restricted. | | **Recall** | Of the reviewed cases QC found in violation, the share that had a policy applied. Low recall means harmful content is being left up. It is estimated from the sample. | | **Accuracy** | The share of reviewed cases where QC reached the same outcome as the original decision (same policy applied, or same decision). | Precision and recall count a case where both sides applied a policy as correct even if they chose different policies, since the violation was real. This is why accuracy can read lower than both: the gap is the volume of policy mismatches. **Error analysis.** Every mismatch is classed as one of four error types, shown in an overall chart and broken down in **Policy results** and **Moderator results** (lowest accuracy first): | Error type | Definition | | ---------------------- | ------------------------------------------------------------------------------------------------- | | **False positive** | A policy was applied when it should not have been. It restricts or removes safe content. | | **False negative** | A policy should have been applied and was not. It fails to protect users from violating content. | | **Policy mismatch** | QC agreed a policy violation occurred, but applied a different policy from the original decision. | | **Different decision** | QC selected a different decision from the original one. | **Results.** Each queue's results page lists every reviewed case, matched or not, with the **Result** (Match, Mismatch or Pending), the **Moderator verdict**, the **QC verdict** and the **Error type** at the front. Search by report or content ID, filter by Result, Error type, Content type, Source or QC expert, and use **Reorder columns** to arrange the remaining columns (saved in your browser). **Export CSV** exports the filtered cases with both verdicts. Opening a case shows the moderator decision and the QC verdict side by side in the Moderation Console. ::Image[]{src="https://api.archbee.com/api/optimize/ijRmXIIrObJhIfPMa8rmJ/38HhLgHJcY9FBDWwUBkFF_screenshot-2026-10-19-at-203559.png" size="60" isUploading="false" width="1270" height="930" darkWidth="1270" darkHeight="930" showCaption="false"} ### User roles and permissions - **QC Expert**: Start QC sessions. Review sampled flags and provide feedback. Cannot configure or update sampling rules. - **QC Admin**: Configure and update sampling rules. Start QC sessions. Access all QC queues. ::Image[]{src="https://api.archbee.com/api/optimize/ijRmXIIrObJhIfPMa8rmJ/BL9urOlx77LGgcR3oEVfQ_image.png" size="44" width="1264" height="680" position="center" caption="QC Permissions" showCaption="true" darkWidth="1264" darkHeight="680"}