Conjoint Analysis Survey Design

Conjoint questions support choice-based preference experiments in social science research surveys. Use them when respondents should compare alternatives made from randomized attributes, such as candidates, policies, products, jobs, messages, neighborhoods, or institutional profiles. A conjoint question can also hold a single alternative — useful when the goal is to show respondents one profile description to read or react to (a vignette), rather than ask them to choose between options.

A conjoint task is strongest when each attribute and level maps to a clear research hypothesis. Keep the table (or vignette) readable so respondents can compare alternatives without fatigue.

• Alternative columns represent the options respondents choose between. Use a single column to display one profile without a forced choice.
• Attribute rows define the features shown for each alternative.
• Level options are randomly drawn for each respondent task.
• Prohibited combinations stop specific attribute-level pairs (or larger sets) from ever being drawn into the same alternative, so the design never produces implausible profiles. Reserve this for pairs that are genuinely impossible or nonsensical (a policy position no real candidate would hold, a price and size that can't co-occur) — recent methods research on “odd” attribute combinations finds respondents attend to and answer implausible profiles about as reliably as plausible ones, so excluding a combination for data-quality reasons alone isn't necessary and just shrinks your design space.
• Number of tasks controls how many times the choice set is repeated (fresh levels each time, one task per page), up to 50. Task count itself is not the main respondent-fatigue lever in the current literature — choice quality holds up well across many repeated tasks. The number of attributes shown per task matters more for both fatigue and statistical power; see the sample-size guidance below when deciding how many to include.
• Uneven level weights let researchers sample some levels more often when the design requires it. With Balance level exposure turned on, that balance holds across a respondent's whole set of tasks, not just within any one task's own alternatives.
• Row and column order controls help reduce presentation effects. Column order can be fixed, shuffled, or Flipped (authored order for about half of respondents, reversed for the rest). When attribute rows are reordered, mark specific attributes Keep in place when rows are reordered so they hold their authored position — for example, to keep Price as the last row for everyone.
• Display as Table or Vignette switches between the attribute-by-alternative grid and a descriptive paragraph per alternative, shown in a box or styled like plain question text.
• Allow selecting an alternative controls whether the table or vignette is clickable. It turns off automatically for a single-alternative design (there is nothing to compare), and can be set either way regardless of alternative count.
• A “None of these” opt-out alternative lets respondents decline every profile shown instead of being forced to pick one they'd reject in real life — see below.
• Image attribute levels let a level show a picture alongside its text, for designs that need a visual profile rather than a text table — see below.
• An attention-check task instructs respondents which alternative to pick on one specific task, to identify inattentive respondents — see below.
• A fixed holdout task shows every respondent the identical, author-chosen profile on one task instead of a fresh random draw, so a preference model fit on the rest of their tasks can be checked against their real choice on it — see below.
• A dealbreaker screening step lets respondents mark attribute levels they'd never accept before their tasks begin, so those levels never appear for them again — see below.
• Balance combinations across respondents rotates every respondent through a shared, evenly balanced sequence of valid attribute-level combinations instead of independent random draws, when the design is small enough to guarantee that balance.

1. Step 1: Open Block Builder. Open the survey, stay on Block Builder, and add the block where the conjoint task should appear.

   Step 1: Open Block Builder

2. Step 2: Add the question. Choose Conjoint from the question type selector.

   Step 2: Choose Conjoint question type

3. Step 3: Name alternatives. Label the columns, such as Candidate A and Candidate B, so respondents understand what they are comparing. Add only one column instead if you want to display a single profile for respondents to read rather than choose between.

   Step 3: Name alternatives

4. Step 4: Add attributes, levels, and any prohibited combinations. Add each attribute row and at least two level options for that attribute. Below the attribute rows, use Prohibited Combinations to keep specific level pairs — a price and a location that would never realistically co-occur, for example — from ever being drawn into the same alternative.

   Step 4: Add attributes and levels

5. Step 5: Set task count and display controls. Choose how many choice tasks each participant sees (for example, 5 repeats, each on its own page). Turn on attribute names, row order controls, column order controls, or level weights only when they match the randomization design. Under Display as, choose Table for the standard grid or Vignette for a descriptive paragraph per alternative — shown in a box, or styled like plain question text. Allow selecting an alternative controls whether the table or vignette is clickable; it turns off by itself for a single-alternative design, and can be switched either way manually.

   Step 5: Set display controls

6. Step 6: Balance combinations across respondents (optional). Turn on Balance combinations across respondents to rotate every respondent through a shared, evenly balanced sequence of valid attribute-level combinations instead of independent random draws. The options panel shows the exact number of valid combinations and whether the design is small enough to balance exactly — above that limit, Domandata falls back to standard randomization automatically.

   Step 6: Balance combinations across respondents

7. Step 7: Preview random draws. Open Preview several times to inspect table (or vignette) readability, mobile layout, and assignment variation.

   Step 7: Preview random draws

A conjoint that forces a choice among only the alternatives shown can distort the estimated preference: a respondent who dislikes every profile still has to pick one, which can shift results away from what respondents would genuinely choose given a real opt-out. Two recent studies replicating published conjoints (on abstention, and separately on electoral conjoints specifically) found that adding a realistic opt-out changed the size — and sometimes the direction — of the estimated effect.

Turn on Add a “None of these” option in the options panel to add one. It renders as its own element below the table or vignette — never mixed in with the randomized alternative columns, so it's never mistaken for a profile with attributes of its own. It is excluded entirely from attribute-level drawing, cross-respondent balancing, and prohibited-combination checks. On export, it appears as its own distinct value (recoded as 0 by default, separate from the 1-based codes for real alternatives), so choice-share analysis can treat “opted out” as its own category rather than a missing response.

Add an image to a level option to show a picture alongside its text in the respondent-facing table or vignette — useful when a design needs a visual profile (a candidate's photo, a product shot, a neighborhood image) rather than a plain text table. Ecological-validity research on “visual conjoint” designs argues that a text-only table can understate how much an attribute like appearance actually shapes real-world choices, since people increasingly evaluate candidates and products through images, not tables.

In Block Builder, each level option has its own optional image slot beneath its text — click Add image to upload a .jpg, .png, or .svg (2.5 MB max, the same limit used for other in-question images). The level's text stays the value used everywhere else — export, recode, and piping — the image is a rendering enhancement only, shown next to that same text, never instead of it.

Turn on Add an attention-check task to turn one specific task into an instructional manipulation check: respondents are told, in plain text above that task, which alternative to pick — regardless of the (still randomized) attributes shown for it. Anyone who picks something else is flagged as a failed attention/manipulation check, the same flag category used elsewhere in Domandata, so it shows up alongside your other quality flags in Export and response review.

Choose which task gets the check (only meaningful when Number of tasks is more than 1) and which alternative respondents are told to pick. Leave the instructions blank to use a generated sentence naming that alternative, or write your own. The attributes shown on that task are still drawn normally — only the instruction, not the profile content, tells respondents what to do — so the task doesn't need a specially engineered “obviously best” profile.

Silicon Sample personas are told the same instruction when generating pilot data, so a simulated run isn't artificially flagged for failing a check it was never told about.

The options panel shows a rough respondent-count estimate: given a minimum detectable effect (in percentage points) you set, it reports approximately how many respondents are needed to reliably detect a shift of that size on the attribute with the most levels — the one that needs the most data, since each respondent-task exposes fewer profile views to any one of its levels. This is a standard rule-of-thumb heuristic (a two-proportion power calculation at conventional 80% power / 5% significance), not a substitute for a full simulation-based power analysis, and it does not know your specific hypotheses or expected effect sizes.

The estimate also flags that interaction or subgroup effects need substantially more data than a main effect of the same size — often several times as much, since comparing two group-specific estimates has more noise than one overall estimate, and more again if the interaction itself is expected to be smaller than the main effect. If a hypothesis in your design compares subgroups (e.g., does the effect of a candidate's party differ by respondent partisanship?), plan sample size around that comparison, not just the main-effect estimate.

A fixed holdout task (below) doesn't contribute new randomized data, so it's automatically excluded from this estimate — the respondent count shown reflects only the tasks that actually vary.

A conjoint battery's randomized tasks are what let you estimate a preference model — but nothing in a standard design lets you check whether that fitted model actually predicts real choices. A fixed holdout task fills that gap: one task number is pinned to a single, author-chosen profile shown identically to every respondent, instead of a fresh random draw. After data collection, fit a model (AMCE or marginal means) on each respondent's other tasks, then compare its predicted choice for the fixed profile against what that respondent actually picked — a standard out-of-sample validation practice for choice-based conjoint designs.

Turn on Add a fixed holdout task in the options panel, choose which task it replaces, and set the exact level shown for every attribute row under each alternative column. Respondents see nothing different about that task — it renders exactly like any other, with no indication it's fixed, since telling them would defeat the point.

A dealbreaker screening step is a lightweight version of adaptive choice-based conjoint (ACBC): before a respondent sees any of their own choice tasks, they mark specific attribute levels as unacceptable (“I would never accept a profile with this”). Those levels are then excluded from every one of that respondent's remaining tasks — a level they rule out never comes back to haunt a later task. The survey-methodology community hasn't settled whether the reduced respondent burden of never seeing a ruled-out level is worth the added complexity and the extra step before the first real task, so treat this as a deliberate design choice for studies where dealbreaker exclusion specifically matters to your research question, not a default to turn on for every conjoint.

Turn on Add a dealbreaker screening step in the options panel, choose which attributes get screened (all of them by default), and optionally write your own instructions. Respondents can't proceed past the step while it would leave a screened attribute with zero acceptable levels remaining — they must leave at least one level of each screened attribute unmarked.

Each screened attribute exports as its own column listing what that respondent marked unacceptable — diagnostic information for understanding a respondent's restricted choice set, not something you typically feed into an AMCE/marginal-means model directly.

Conjoint exports the selected alternative and the assignment information needed to analyze which levels were shown, regardless of whether the design is displayed as a table or a vignette — the underlying assignment data is the same either way. When Allow selecting an alternative is off (the default for a single alternative), there is nothing for the respondent to click, so no selected-alternative value is recorded; the shown profile still exports through the assignment snapshot. When a “None of these” option is enabled and picked, it exports as its own distinct value rather than a missing response — see above. Preserve the design once live data collection begins. If the conjoint appears inside a larger experiment, use experimental conditions to control which blocks respondents see.

Domandata exports the design and the raw choice data; estimating the AMCE (average marginal component effect) or marginal means happens downstream, in your own analysis (R's cregg/cjoint, Stata, or Python). Current methods discussion notes that AMCE and marginal means can tell different stories from the same data — AMCE is a causal estimate that can be dominated by respondents with especially strong preferences, while marginal means are a more purely descriptive summary — so consider reporting both rather than defaulting to one.

Use the conjoint: piped text token to reuse what a participant saw in a conjoint question — a later question, a thank-you screen, or the question's own labels can restate the profile instead of leaving it behind on the conjoint page. This works the moment the conjoint question has rendered, even before Allow selecting an alternative is on or a choice has been made.

{{conjoint:question_name}}

By itself, that pipes every alternative shown in task 1, formatted as Table or Vignette to match the question's own Display As setting. Add a path after the question name for more specific references:

An attribute reference (like Price above) matches the attribute row's label, case-insensitively; row2 works instead if you'd rather reference it by position. A recap only ever shows attribute names when Show attribute names is on for that question — matching what the participant actually saw, since a design that hides attribute names is often hiding them on purpose.

{{conjoint:question_name.table}} renders as a real, styled grid — attribute rows down the side, one column per alternative shown — it just isn't clickable the way the live conjoint question is, since piped text can only ever be read, not interacted with. {{conjoint:question_name.selected}} and similar “what was picked” tokens resolve to an empty string (not the literal token) while the task is still unanswered, since that's an expected, temporary state rather than a broken reference — pair it with {{conjoint:question_name.answered}} if the surrounding text needs to read differently before and after a choice is made. If a “None of these” option is enabled, picking it counts as answered, and .selected pipes its label (an attribute reference off it, like .selected.Price, resolves blank since there's no profile to read a cell from).

To ask a participant why they chose (or opted out of) a profile, attach an ordinary open-ended follow-up question after the conjoint and pipe the choice into its prompt, for example {{conjoint:question_name.selected}} in a prompt like “Why did you pick that option?” This already works with the piping described above — no separate configuration needed.

• Vignette Experiment Example
• How to Randomize Treatment Conditions
• Randomize Survey Questions and Answer Options
• Export Survey Data to Stata, R, and Python
• Ranking Survey Questions
• Pipe Text from Variables and Answers