Author-Defined Response Quality Rules

Author-defined response quality rules tell Domandata which responses should be flagged before analysis. They are stored with the survey, evaluated on each response, and exported as quality flags, rule matches, cleaning actions, replication-package metadata, and API fields.

Use rules for planned criteria: explicit correct answers for checks, impossible answer combinations, minimum dwell-time expectations, and source-specific review defaults. Keep subjective judgment in the quality review notes instead of encoding it as a hidden rule.

• Attention or manipulation answer: choose the check question and enter one or more accepted answers.
• Impossible pair: choose two questions and the answers that should not appear together.
• Minimum dwell: set the minimum total time, in seconds, expected for the survey response.
• Source default: apply a default include, exclude, or unreviewed decision plus tags for a submission source.

Rules add flags and rule-match details. They do not erase the raw response. Manual include/exclude decisions still take precedence over source defaults.

1. Step 1: Create the check question. In Block Builder, add a multiple choice, dropdown, short answer, or other check that has a clear expected answer.

   Step 1: Create Check Question

2. Step 2: Open Export. Submit a pilot response first so the Export tab can show response fields and the Quality Review panel.

   Step 2: Open Export

3. Step 3: Add the expected answer. In Quality Review, select the check question and enter the correct answer exactly as it should be accepted.

   Step 3: Add Expected Answer

4. Step 4: Pilot a pass and a fail. Submit one response that should pass and one that should fail, then confirm the failed response receives a quality flag.

   Step 4: Pilot Pass and Fail

Impossible-combination rules flag responses where two answers should not both be true. Examples include selecting None of the above and a substantive option, claiming no prior purchase while later reporting a purchase frequency, or failing a screener while also matching an eligible condition.

• Use clear answer labels or recoded values that are stable in the export.
• Keep each rule narrow enough that a reviewer can understand the reason from the cleaning log.
• Use separate rules for conceptually different problems instead of one large catch-all rule.

Minimum dwell-time rules flag responses that move through the survey faster than the study design allows. Use them when a stimulus, instruction page, or multi-page task requires a minimum reading or exposure time.

Timing rules work best when Capture page dwell time is enabled before launch. If timing metadata is missing for a response, Domandata will not invent a failure for that rule.

Source defaults apply review decisions and tags by submission source. They are useful for pilot links, known test batches, panel waves that should start in review, or recruitment channels that need source-specific cleaning rules.

• Use exclude for sources that should never enter analysis, such as internal test links.
• Use unreviewed with tags for sources that need human review but should not be removed automatically.
• Use include sparingly, usually when a source is trusted and flags should still remain visible.

• Quality Flags: the automated flag labels raised by the response.
• Quality Flag Details: the explanation for each flag.
• Quality Rule Matches: the author-defined rules that matched.
• Quality Review Source: whether the review decision is manual, source default, or none.
• Quality Cleaning Action: the include, review, or exclude recommendation used by analysis templates.

These fields travel into response exports, replication packages, analysis templates, and API pulls.

• Quality Review Queue and Cleaning Log
• Maintaining Survey Response Quality
• Attention and Manipulation Checks
• Export Research Survey Responses