Quality Review Queue and Cleaning Log

The Quality Review queue helps researchers triage response-level quality flags, mark rows include or exclude, preserve review tags, and export a cleaning log. It is designed for export-first analysis: the raw response remains intact, while review decisions travel into CSV/XLSX exports, replication packages, analysis templates, and API pulls.

• All Responses: every loaded response after the active data-source and field filters.
• Flagged Responses: rows with one or more automated quality flags.
• Unreviewed: rows with no manual include/exclude decision and no effective source-default decision.
• Included: rows marked include manually or by an active source default.
• Excluded: rows marked exclude manually or by an active source default.

Use the queue filter to narrow the response table, then review the quality columns beside the answers, conditions, source fields, and metadata.

1. Step 1: Open Export. Go to the survey's Export tab after responses or pilot submissions exist.

   Step 1: Open Export

2. Step 2: Open Quality Review. Expand the Quality Review panel and choose Flagged Responses.

   Step 2: Open Quality Review

3. Step 3: Read the flag details. Compare the flag with the respondent's answers, source, timing, attention checks, and Flow path.

   Step 3: Read Flag Details

4. Step 4: Mark include or exclude. Use include when the row should remain in analysis, exclude when the protocol says it should be removed, and clear review if the decision should return to unreviewed.

   Step 4: Mark Review Decision

5. Step 5: Export after review. Download responses or push a replication package so quality flags, review decisions, tags, and cleaning actions are preserved.

   Step 5: Export Reviewed Data

• Quality Score: a heuristic 0 to 100 score based on the number and severity of flags.
• Quality Flags: labels such as speeding, failed check, impossible combination, repeated text, bot score, and excessive missingness.
• Quality Flag Details: human-readable reasons for the flags.
• Quality Review: the include, exclude, or unreviewed decision used for cleaning.
• Quality Review Source: manual, source default, or none.
• Quality Tags: researcher or default tags such as pilot, duplicate, failed-check, or panel-default.
• Quality Rule Matches: author-defined rules that matched the response.
• Quality Cleaning Action: the include, review, or exclude action used by exported analysis templates.

Cleaning Log.csv is included in replication packages and can also be rebuilt from response exports. It records response id, submitted time, submission source, quality score, flags, rule matches, review decision, review source, tags, notes, and cleaning action.

• Use it to document why rows were excluded.
• Archive it with the codebook, survey instrument, and raw response export.
• Use it as the bridge between raw data and the analysis-ready dataset.
• Share it with collaborators or reviewers when exclusions affect the final sample.

Quality review metadata is export-first. Replication packages include response CSVs, Cleaning Log.csv, codebook, metadata, and R/Python starter scripts. The scripts load the cleaning log and create analysis-ready data frames that exclude rows with Quality Cleaning Action set to exclude.

API pulls include quality assessment and cleaning-log objects with each response, so scripted workflows can reproduce the same review state used in manual exports.

• Author-Defined Response Quality Rules
• Maintaining Survey Response Quality
• Review and Clean Survey Responses
• Pull Survey Responses With the API