Investigating causal relationships with power: Online experiments

Ulf-Dietrich Reips (Social and Economic Psychology, University of Zurich)

An up-to-date overview of basics, techniques, methods, tricks, and tools for Internet-based experimentation. The talk summarizes principles of (online and offline) experimentation and ways of dealing with issues in experimental design, security, recruitment, sampling, self-selection, multiple submissions, reactance-free question design, response time measurement, dropout, error estimation, distributed experimentation, data handling, data quality, and log file analysis.

Among other methods, the presenter will explain the warm-up technique, seriousness check, sub-sampling procedures, multiple site entry, ways to check for motivational confounding and when and why the high hurdle technique may work or not. Free Web services are introduced, like vasgenerator.net for creating visual analogue scales, the ”::web experiment list::” for recruitment of participants, and Scientific LogAnalyzer for data analysis. Empirical results from investigations into the validity of several of the techniques for Internet-based experimenting will be discussed.