Two Kinds Of Random
Every randomization feature in Domandata resolves down to one of two mechanisms, and it helps to have plain names for both:
- Even (block/balanced) — drawn from a hat. As responses come in, Domandata keeps a running count per option and picks whichever is currently furthest below its target share. Like drawing marbles from a hat without putting them back: the pool empties out roughly evenly no matter how many draws happen. Sample sizes converge on the target split even in a small or unevenly-dropped-out sample.
- Random (simple) — rolled like a die. Each respondent's draw is independent of every other respondent's. Like rolling a die: over a large sample the outcomes approach the target split, but any given run can drift, and a small sample can end up visibly lopsided.
Flow: Conditions and Routing
Which One Should A Study Use?
- Choose even when the analysis plan needs guaranteed, or close to guaranteed, cell sizes — a small-N experiment, a design where an unbalanced sample would be hard to reweight, or any study where "what if the last five respondents all landed in the same arm" is not an acceptable risk.
- Choose random when the design specifically calls for independent per-respondent draws — some permutation-based or randomization-inference approaches rely on that independence — or when the sample is large enough that expected balance is good enough and simplicity matters more.
- Either way, the target percentages stay in your control. This setting only changes how strictly the target split is enforced across the sample, not what the split is.
Where This Choice Appears
The even/random choice is exposed as a toggle everywhere assignment is genuinely stochastic and the balance matters across the whole sample. It is not offered where it would not mean anything — see "Where It Does Not Apply" below.
- Experimental condition arms: a "Balance arm sizes" toggle on each condition set, for both Random Assignment and Stratify by variable modes. Stratified balancing claims lazily, once the blocking question is answered, so it can balance within each group without knowing the group in advance. See Create Experimental Conditions and Sample Allocation Across Condition Arms.
- Random-mode flow variables: a "Balance outcome sizes" toggle, same mechanism, for hidden randomized values like incentive amounts or message variants. See Random Variables and Permutation Assignments.
- Conjoint attribute-level combinations: a "Balance combinations across respondents" toggle rotates the whole sample through a shared, evenly balanced sequence of valid combinations, for designs small enough to balance exactly. See Conjoint Survey Questions.
- MaxDiff trial sets: always balanced by design — Domandata draws each respondent's trials from a shuffled, refill-on-exhaustion pool, so item exposure stays roughly even across the sample automatically. There is no separate toggle because there is no useful "random" alternative to offer. See MaxDiff Survey Questions.
- The account-level default: a "Balance new arms and outcomes" preference sets which choice new condition sets, random variables, and conjoint level weights start with. It only seeds new ones — existing configurations are never changed retroactively, and every instance can still be switched individually. See Customize Builder Defaults and Preferences.
Where It Does Not Apply
- Question, answer, row, and column order shuffles (Multiple Choice, Ranking, Grid Matrix, Card Sort, block question order, and similar) show every item to every respondent exactly once — only the order changes. There is nothing to balance, because exposure is already even by construction; there is no "uneven order" failure mode. See Randomize Survey Questions and Answer Options.
- Permutation assignments already guarantee their own balance by choice of sampling mode: Without Replacement guarantees no repeats within one respondent, and With Replacement is an independent draw by design — neither needs a separate even/random toggle. See Random Variables and Permutation Assignments.
- Deterministic (rules-based) assignment is not random at all — respondents are routed by a fixed rule on an earlier answer or variable, so balancing does not apply.
Testing Never Consumes A Real Balance Slot
Everywhere "even" balancing exists, Preview and live-test links always fall back to an independent random draw instead of claiming a real slot. This is intentional: a real study's balance counters must only reflect real, published-link respondents, and a researcher needs to be able to click through Preview repeatedly while building a survey without skewing the eventual sample. It also means repeated previews will not visibly look balanced even when "even" is turned on — that is expected, not a bug.
Preview: Independent Draw, Not a Real Claim
What To Check Before Launch
- Every condition set, random variable, and conjoint design that needs guaranteed balance has its "Balance..." toggle turned on — it defaults to off (random) unless your account preference is set to even.
- Assignment percentages total 100% wherever they are set.
- Stratification variables resolve before the split they feed.
- A few Preview passes look reasonable, keeping in mind Preview never reflects real-sample balance.
- After a few real published-link responses, live per-arm counts (shown next to each arm name in the Branching panel) match expectations.