All Research Guides

A Guide From Domandata

How to Choose Online Experiment Software for Psychology

Written by Domandata, which makes a survey platform. We name our product where it fits and say where another tool fits better. Claims about other tools link to their own documentation, checked ; confirm anything decisive with the vendor or your institution.

Psychology studies run online range from questionnaires with a few manipulations to full cognitive paradigms that need precise stimulus timing. Survey platforms handle the first well; dedicated experiment builders handle the second.

The main trade-offs are how much you want to code, how you want to pay, and where data must be stored. This guide maps the main tools against those trade-offs.

The short answer

If you do not want to code, use Gorilla or Labvanced. If you want a graphical builder with code underneath, use PsychoPy, which runs online through Pavlovia, or Inquisit for its ready-made test library. If you are comfortable with JavaScript and want to pay nothing, build in jsPsych and host on JATOS. For questionnaire-led studies with a timed task or two, a research survey platform such as Domandata or Qualtrics is often enough.

How to choose an online experiment tool

  1. 1

    How much timing precision you need

    Response times measured in a browser depend on the participant's device and browser. Paradigms where a few milliseconds matter need tools built for timing; questionnaires with response-time measures need less. Validate on the devices your participants use.

  2. 2

    How much you want to code

    No-code builders get studies running quickly but can limit unusual designs. Code-first tools can do anything, at the cost of programming and testing time.

  3. 3

    How you pay

    Per-participant tokens or credits scale with sample size; annual licenses suit labs running many studies; open-source tools are free but need hosting. Price your planned studies under each model.

  4. 4

    Where data is stored

    If your ethics board or data-protection office requires EU hosting or a processing agreement, check hosting location and agreements before building.

  5. 5

    Questionnaires alongside tasks

    Most studies combine consent, demographics, and scales with the task. Check how well each tool handles both, or plan to link two tools and carry the participant ID across.

  6. 6

    Recruitment integration

    Confirm the tool captures participant IDs from Prolific, CloudResearch, or your subject pool and redirects people back when they finish.

Experiment tools by working style

No coding

Visual builders for tasks and questionnaires. Gorilla charges per participant and hosts in Ireland; Labvanced uses annual licenses, hosts in Germany, and offers webcam eye tracking on premium tiers.

Gorilla Experiment Builder

No-code builder for online behavioral experiments and questionnaires, common in psychology.

Pricing model
Building and testing are free; you pay per participant with tokens, with academic discounts and lab or institution subscriptions.
Where it runs
Cloud only, on Microsoft Azure in Ireland, with a GDPR data-processing agreement.

Strengths

  • Reaction-time tasks and questionnaires in one study without coding.
  • A code editor is available when the visual builder is not enough.
  • Recruitment targets cap how much a study can spend.
  • Exports trial-level (long) or one-row-per-participant data as CSV or Excel.

Watch out for

  • Cost scales with sample size, since every participant uses a token.
  • Makes no HIPAA claim; compliance responsibility sits with the researcher.

Sources:Gorilla Pricing FAQGorilla Data ExportsGorilla Due Diligence

Labvanced

Browser-based no-code experiment builder that includes webcam eye tracking.

Pricing model
Yearly licenses in tiers capped by recordings per year, plus a small free plan; eye tracking requires a premium tier.
Where it runs
Cloud, on servers in Falkenstein, Germany.

Strengths

  • Webcam eye tracking that stores gaze coordinates rather than video.
  • No coding required for most designs.

Watch out for

  • Its public security page does not say whether a data-processing agreement is offered.
  • The free plan allows only a handful of recordings.

Sources:Labvanced LicensesLabvanced SecurityLabvanced Eye Tracking

A builder with code underneath

PsychoPy is free for lab use and runs online through Pavlovia, where piloting is free. Inquisit is licensed and comes with a large library of standard cognitive tests.

PsychoPy and Pavlovia

Open-source software for building cognitive and behavioral experiments with a graphical Builder or Python code; Pavlovia runs them online.

Pricing model
PsychoPy is free. On Pavlovia, piloting is free and collecting data is paid per participant or through an institutional license.
Where it runs
PsychoPy runs on your own computer for lab studies; Pavlovia is a cloud service.

Strengths

  • Designed for precise stimulus timing in cognitive tasks.
  • A graphical Builder, with Python for anything custom.
  • Free for in-lab studies.

Watch out for

  • Advanced work needs Python or JavaScript.
  • Online fielding costs money unless your institution holds a license.

Sources:PsychoPyPsychoPy and Pavlovia PricingPavlovia Piloting and Running

Inquisit

Reaction-time and cognitive testing software from Millisecond, with a lab app, a web service, and a large library of ready-made tasks.

Pricing model
Term or perpetual licenses, from single-experiment to department and campus licenses.
Where it runs
Inquisit Lab runs on lab computers; Inquisit Web runs studies online on computers and mobile devices.

Strengths

  • Built for timing precision in cognitive tasks.
  • A large library of standard tests included with a license.

Watch out for

  • A paid license is needed to run studies.
  • Programming happens in its own scripting language in the Lab app.

Sources:MillisecondInquisit Web Academic Licensing

Code-first and free

Write the experiment in jsPsych and host it on your own JATOS server, or on the free MindProbe instance. No license fees, but you own the programming and maintenance.

jsPsych

Free JavaScript library for behavioral experiments that run in the participant's browser.

Pricing model
Free and open source under the MIT license.
Where it runs
None included; host it yourself or with a service such as JATOS, Cognition.run, or Pavlovia.

Strengths

  • Complete flexibility, with a large plugin ecosystem.
  • No license fees.

Watch out for

  • Requires JavaScript programming.
  • You arrange hosting, data storage, and security yourself.

Sources:jsPsych: Running ExperimentsjsPsych on GitHub

JATOS

Free, self-hosted server for running online studies built with jsPsych, lab.js, PsychoPy, OSWeb, or plain web code.

Pricing model
Free and open source (Apache 2.0); MindProbe is a free hosted instance sponsored by a European psychology society.
Where it runs
Self-hosted on your own server, or on MindProbe.

Strengths

  • Hosts studies from most open-source experiment tools in one place.
  • Supports real-time group studies and recruitment-platform links.

Watch out for

  • You run and maintain the server unless you use MindProbe.

Sources:JATOSWhat Is JATOS

Questionnaire-led studies with timed elements

When the study is mostly scales and manipulations with a simple timed task, a survey platform keeps everything in one instrument. Domandata's reaction-time question timestamps stimulus onset after it is painted; for full paradigms, a dedicated tool gives more control.

DomandataOur product

Browser-based survey platform built for academic social-science research: survey experiments, conjoint, and analysis-ready data.

Pricing model
Free plan for drafting and previewing; fielding a study, Flow logic, and exports need a paid individual or team plan with published self-serve pricing.
Where it runs
Cloud only, on U.S.-based infrastructure providers.

Strengths

  • Randomized treatment arms (weighted, stratified, or balanced) with the assigned condition saved in every export.
  • Conjoint, MaxDiff, list experiments, and reaction-time tasks built in, without custom JavaScript.
  • Native SPSS (.sav) and Stata (.dta) exports with a generated codebook, plus Qualtrics .qsf import.
  • Links OSF preregistrations and deposits replication materials to Harvard Dataverse or Zenodo.

Watch out for

  • Does not sign HIPAA Business Associate Agreements, so it is not suitable for studies that require one.
  • Cannot be self-hosted, and the data-hosting region is not fixed by contract.
  • The free plan cannot collect responses; fielding a study requires a paid plan.
  • A newer, smaller vendor: its SOC 2 Type I audit is still in progress, which some IT reviews ask for.

Sources:Domandata PricingDomandata Trust Center

Qualtrics

Enterprise survey platform that many universities license campus-wide, and the default tool in much of academic survey research.

Pricing model
Usually an institutional license at no personal cost to students and staff, on terms set by your university's agreement. A capped free account is meant for trying the platform.
Where it runs
Cloud only. Publishes ISO 27001 and FedRAMP certifications.

Strengths

  • If your university licenses it, it is already paid for and usually reviewed by IT.
  • Mature, deep feature set, including Survey Flow randomizers and quotas.
  • Exports to CSV, Excel, and SPSS.

Watch out for

  • Access depends on your institution; leaving it can mean losing the license.
  • The free account caps responses, active surveys, and questions per survey, which rules out most studies.

Sources:Qualtrics Free AccountsQualtrics Account CreationQualtrics RandomizerQualtrics Security

Before you run an online experiment

  • Timing was checked on the browsers and devices participants will use
  • Device, browser, and input method are recorded with the data
  • Participants on unsupported devices are screened out politely
  • Instructions and practice trials were tested with naive pilots
  • Data saves trial by trial, so a dropout does not lose everything
  • Recruitment IDs and completion redirects work end to end
  • Hosting location and data agreements satisfy your ethics approval

Frequently Asked Questions

Should I use Gorilla or Pavlovia?

Gorilla is a no-code builder that hosts the whole study, charging per participant. Pavlovia hosts experiments built in PsychoPy, and also jsPsych and similar tools, charging per participant or through an institutional license, with free piloting. Choose Gorilla to avoid code, and Pavlovia if your lab already builds in PsychoPy.

Is online reaction-time data accurate enough for research?

For many paradigms, yes, provided you use a tool designed for timing and account for variation between devices and browsers. Record device and browser information, pilot on the hardware participants use, and favor within-subject comparisons, which are less sensitive to device differences.

Can I run eye tracking online?

Webcam-based eye tracking is available in some browser tools, such as Labvanced on its premium tiers. It is less precise than lab eye trackers, so validate it for your stimuli and expected effect sizes before relying on it.

What is the cheapest way to run online psychology experiments?

jsPsych is free and can be hosted on JATOS or the free MindProbe server, and PsychoPy is free for lab studies. These cost programming time instead of money. For paid hosting, compare per-participant pricing against an annual license using your planned sample sizes.

Keep Reading

Reaction Time Questions in Domandata

How timing, trials, and response modes work in a survey.

Choosing a Recruitment Platform

Prolific, CloudResearch, panels, and probability samples.

GDPR-Compliant Survey Software

Hosting and processing agreements for EU research.

Why Domandata

Our own case for Domandata, and when to use another tool.