Randomize Survey Questions and Answer Options

Randomization helps reduce order effects in social science research surveys. Depending on the measure, researchers may randomize answer options, matrix rows, conjoint rows or columns, slider orientation, or the order of questions inside a block.

1. Step 1: Open Block Builder. Select the question or block whose order could bias responses.

   Step 1: Open Block Builder

2. Step 2: Click Options on the question card. The Options button opens the configuration panel where order controls live.

   Step 2: Click Options on the question card

3. Step 3: Choose the order setting. Use the order dropdown for answer order, row order, column order, ranking item order, conjoint attribute order, or slider flip.

   Step 3: Set the order in the options panel

4. Step 4: Keep meaningful sequences fixed. Do not randomize consent, instructions, dependent follow-ups, or ordered scales where sequence matters.

   Step 4: Review fixed versus randomized options

5. Step 5: Preview multiple times. Open Preview repeatedly to see whether random draws and fixed options behave as expected.

   Step 5: Preview randomized question

6. Step 6: Export order data. Open Export and include order fields when display order affects analysis.

   Step 6: Include order fields in Export

• Multiple Choice: fixed, reverse, or shuffle answer options.
• Ranking: fixed, reverse, or shuffle the starting item order.
• Grid Matrix: randomize rows, columns, or both for repeated measures.
• Conjoint: randomize attribute rows and alternative columns independently (columns can also Flip), keep chosen attributes fixed with Keep in place when rows are reordered, or turn on Balance combinations across respondents to rotate the whole sample through a shared, evenly balanced sequence of valid combinations instead of independent draws (small designs only — see Conjoint Survey Questions).
• Card Sort: fixed, shuffle, or Flip, set separately for cards and categories, plus Show subset to give each respondent a random subset of cards.
• Constant Sum: set the initial item order and optionally show each respondent a random subset of the items.
• Slider: flip orientation while keeping stored values consistent.
• Blocks: shuffle questions inside a block when question order is not meaningful.

When several questions share the same answer options and use shuffle (or a randomized subset), respondents usually expect the same relative positions on every item — for example, every Likert item in a battery showing "Strongly Agree" in the same slot. Survey Options includes Consistent display order for this (on by default for new surveys). Questions with identical option sets keep the same shuffled order for each respondent; different option sets shuffle independently. Scope can be the whole survey or within each block.

Turn on Also coordinate flip direction when reverse or random-flip questions should stay aligned the same way — useful for slider grids and reversed scales. Choose whether flip coordination applies to all compatible questions or grid questions only.

The order controls on this page are a different kind of randomization from experimental condition arms, random-mode variables, or permutation assignments. An order shuffle still shows every option, row, column, or question to every respondent exactly once — only the sequence changes. Because exposure is already even by construction, there is nothing to balance and no "even vs. random" toggle to make: order randomization is inherently exposure-neutral. Contrast this with assignment-type randomization (which arm, which outcome, which permutation slot a respondent gets), where an even/random choice is meaningful because different respondents can end up with different, unevenly distributed values. See How Randomization Works for the full picture across both kinds.

When randomization affects interpretation, include order information in the export. Order snapshots help researchers audit what each respondent saw and model display-order effects if needed. See Export Research Survey Responses.

• Do not randomize consent, instructions, or dependent follow-up sequences.
• Pin Other options last when respondents need an easy catch-all answer.
• Pilot randomized paths before publishing.
• Document randomization choices in your codebook.

• How Randomization Works in Survey Experiments
• Export Survey Data to Stata, R, and Python
• Multiple Choice Survey Questions
• Grid Matrix Survey Questions
• Conjoint Analysis Survey Design