Tree Testing Survey Questions

Tree Testing asks respondents to navigate a fixed, author-defined category hierarchy to find where they'd expect a specific item to live — "Where would you go to update your billing address?" — without any of the visual design or navigation labels of a real site or app in the way. It is the standard complementary method to Card Sort in information-architecture research: a card sort discovers a candidate structure from how respondents naturally group items; a tree test then validates whether that structure (or any proposed structure) actually works before you build it.

One question runs a full set of findability tasks against the same tree, the same way a MaxDiff question runs multiple trials internally. Tree Testing is in beta: it requires Standard plan or above and beta tester access to appear in the question type picker, and shows a violet "Beta" badge wherever it appears in the editor.

• Tree: the category hierarchy respondents navigate. Categories can nest — a top-level "Account" category can contain "Billing" and "Security" sub-categories, for example.
• Tasks: each task is a findability prompt ("Where would you look to find X?") plus one or more nodes in the tree you've marked as the correct destination. Allowing more than one correct node accommodates a tree with a genuinely reasonable second home for an item.
• Any level is a valid answer: respondents aren't limited to picking a leaf node — at any point while navigating, they can confirm the category they're currently looking at as their final answer, or drill further in.
• Give up: optionally let respondents abandon a task instead of forcing a guess when they genuinely wouldn't know where to look.
• Task order: randomized per respondent by default, for order counterbalancing.

• Don't echo the tree's own labels. A task worded as "Find the Billing category" tells a respondent exactly which word to look for — they're pattern-matching text, not demonstrating real findability. Describe a realistic scenario using different words instead: "You want to update the card on file for your account," not "Find Billing."
• Aim for specific, not leading. A task should point at one clear goal without being so vague that half the tree looks plausible. Domandata's Instrument Review flags the most literal version of label-echoing automatically (a task whose wording contains one of its own correct destination's exact labels), but reviewing the rest of your wording is still on you.
• Pilot before fielding. Run the tree test on a handful of colleagues first — task wording that reads as clear to the person who wrote the tree often isn't clear to someone seeing it for the first time.

1. Step 1: Open Block Builder. Open the survey and select the block where the findability task belongs.

   Step 1: Open Block Builder

2. Step 2: Add a Tree Testing question. Choose Tree Testing from the question type selector.

   Step 2: Choose Tree Testing question type

3. Step 3: Build the tree. Add top-level categories, then add sub-categories under any node that needs them.

4. Step 4: Add tasks. Write each findability prompt and mark its correct destination node (or nodes) in the tree.

5. Step 5: Decide on give-up and task order. Choose whether respondents can abandon a task, and whether task order is randomized per respondent.

6. Step 6: Name and code it. Add a readable variable name before piloting so the per-task columns are easy to interpret in export.

   Step 6: Variable Name

7. Step 7: Preview the navigation. Click through the tree yourself to confirm the hierarchy reads clearly before fielding.

   Step 7: Preview the navigation

Each response exports one column group per authored task — the prompt, the final destination the respondent confirmed, success (whether that destination was one of the authored correct nodes), directness (whether the respondent ever backtracked to a shallower point before confirming), whether they gave up, how long the task took, and which top-level category they clicked first — keyed by the task's authored position, not the shuffled order any individual respondent saw.

Domandata also computes summary columns per response: overall success rate, directness rate, mean task duration, and the direct/indirect breakdowns of both success (direct success — reached the right place with no backtracking; indirect success — got there, but only after backtracking) and give-up (skip rate — direct skip, abandoned without exploring the tree at all; indirect skip, gave up only after some navigation). These mirror the standard breakdowns Optimal Workshop's Treejack reports.

Interpreting success rate: a commonly cited benchmark (Albert & Tullis) treats a tree test's overall success rate as Poor below 40%, Fair from 41-60%, Good from 61-80%, Very Good from 80-90%, and Excellent above 90% — though a mission-critical task (e.g. finding emergency contact information) deserves a higher bar than a routine one, and a tree test's scores read lower than a live product's would since it strips away visual design and search.

Sample size: published guidance for tree testing generally runs larger than intuition from small-sample usability testing suggests — a standalone tree test commonly targets somewhere in the 50-150 respondent range, and comparing more than one candidate tree structure generally wants roughly twice that. This is a genuinely different regime from the "five users is enough" heuristic sometimes cited for discount usability walkthroughs: that heuristic is for qualitative, observational testing, while a tree test is a quantitative, comparative measurement (you're estimating a success-rate proportion, often across conditions), where a handful of respondents carries real sampling error. Treat published numbers as a range to plan around, not a single confident target.

• Card Sort Survey Questions, the method tree testing typically follows
• MaxDiff Survey Questions for another trial-based question format
• Survey Variable Names and Recodes
• Export Research Survey Responses