Use Checks To Support Data Quality
Attention checks, comprehension checks, and manipulation checks help researchers understand whether respondents read instructions, understood a treatment, or experienced the intended experimental manipulation. They should answer a specific research or quality-control question, not simply add a hurdle because the survey feels important.
Plan each check before launch: where it appears, whether a failed check routes the respondent out, whether the result is an exclusion rule, and which export variable analysts should use.
Preview: Respondent View
Preview
Walk through respondent-facing behavior in the same preview surface.
Types Of Checks
- Attention checks: confirm respondents are reading instructions or options carefully, such as selecting a named option in a low-stakes item.
- Comprehension checks: confirm respondents understand a task, policy, vignette, or treatment before they answer outcome questions.
- Manipulation checks: measure whether the treatment changed the intended perception, belief, or awareness.
- Validation checks: make sure numeric or text answers follow the expected format, usually through required questions and validation rules.
Configure Checks
Step 1: Place the check near the task it evaluates. Put a comprehension check after instructions or a stimulus, and put an attention check where it will not reveal the treatment logic.
Step 1: Place the Check
Survey Editor
No-persist demo using the real builder shell.
Step 2: Add the check in Block Builder. Use Multiple Choice for closed checks, Short Answer with validation for typed checks, and Grid Matrix for repeated comprehension items.
Step 2: Add Check in Block Builder
Block Builder: Attention Check
The prompt instructs respondents to select a specific option, letting you detect inattentive responses in the export.
Step 3: Name the variable. Give the check a clear question name, such as attention_check, comprehension_policy, or manipulation_check, so it is easy to find in Export.
Step 3: Name the Variable
Block Builder: Question Name
The export variable label is the column name for this question in downloaded data.
Step 4: Route only when needed. If the answer determines whether respondents continue, open Flow and branch from the block that contains the check.
Step 4: Route via Flow
Block BuilderPreviewDeployFlow Canvas
Interactive routing surface from the app, running in no-persist help mode.
Step 5: Preview both outcomes. In Preview, submit one passing response and one failing response to confirm the paths and exports.
Step 5: Preview Both Outcomes
Block BuilderPreviewDeployPreview: Attention Check
Select 'Somewhat agree' to pass the check. Select any other option to produce a failing response for testing the exclusion path in Flow.
Common Design Choices
- Exclusion check: use only when the protocol says a failed answer should remove the respondent from analysis or collection.
- Diagnostic check: keep the respondent in the survey, export the variable, and decide during analysis how to use it.
- Comprehension retry: route a failed respondent to an instruction reminder, then let them answer again if the protocol allows retries.
- Manipulation measure: avoid using it as an attention check if it is substantively part of the outcome model.
Analysis Notes
Decide before launch whether checks are exclusion criteria, diagnostic variables, or outcomes. Document that decision in a pre-analysis plan or codebook so the export is interpreted consistently. If a check changes after launch, record the date, old wording, new wording, and affected response ids.
Deploy: Publish and Links
Deploy
Configure publishing, links, quotas, and theme from the Deploy tab.