Review and Clean Survey Responses

Response review is where researchers identify test submissions, incomplete cases, failed checks, duplicate entries, and unusual open-text responses. Keep the raw export intact, use the quality review queue for include/exclude decisions, and preserve the cleaning log with the analysis package.

Start from Export. Compare responses against the survey structure, Flow paths, quota rules, source labels, and any quality rules that were part of the study design. For the focused queue workflow, see Quality Review Queue and Cleaning Log.

1. Step 1: Export a backup. Download the current data from Export Research Survey Responses before cleaning.

   Step 1: Export a backup

2. Step 2: Identify test responses. Compare timestamps, draft/public source fields, known pilot submissions, and notes from Preview and Pilot Test a Survey.

   Step 2: Identify test responses

3. Step 3: Check quality fields. Review attention checks, validation fields, quality flags, rule matches, completion timing, duplicate identifiers, and open text that suggests a failed task.

   Step 3: Check quality fields

4. Step 4: Check experimental fields. Confirm condition assignments, quota-relevant variables, randomized order fields, and treatment exposure fields are present.

   Step 4: Check experimental fields

5. Step 5: Apply planned exclusions. Use criteria from your pre-analysis plan or codebook and mark rows include or exclude in the quality queue instead of editing answers.

   Step 5: Apply planned exclusions

6. Step 6: Export the cleaning log. Preserve response ids, flags, rule matches, review decisions, tags, notes, and cleaning actions with the raw export and replication package.

   Step 6: Save a cleaning log

• Dataset version: export date, survey version, and whether the file came from manual Export, a replication package, or the API.
• Rows removed: test responses, duplicates, incomplete cases, and the response ids affected.
• Check outcomes: failed attention, comprehension, manipulation, or validation checks, plus whether those failures are exclusions or diagnostic flags.
• Rule matches: author-defined correct answers, impossible combinations, minimum dwell-time rules, and source defaults that matched each response.
• Routing status: ineligible, quota-full, screened-out, or incomplete paths.
• Any personally identifying fields handled under privacy rules.
• Fielding changes: mid-field wording, Flow, quota, link, or end screen changes already noted in the codebook.

• Do not overwrite the raw export with recoded or filtered data.
• Do not remove respondents only because their answer is inconvenient for the hypothesis.
• Do not change option labels after launch without documenting how old and new labels map.
• Do not rely on memory for pilot submissions; mark or log test responses when the pilot happens.
• Do not discard condition assignment or randomized order columns, because they explain what each respondent saw.
• Do not treat every automated flag as an exclusion; review the flag detail and the study protocol first.

• Survey Data Quality Checks
• Quality Review Queue and Cleaning Log
• Author-Defined Response Quality Rules
• Survey Variable Names and Recodes
• Pull Survey Responses With the API