Information Visualisation: the design space of experimental methodologies

Empirical studies in Information Visualisation research have become more commonplace in the past two to three decades. While formerly the research focus was primarily on utilising the power of novel technologies for presenting data and information in innovative ways, perspectives have changed over time so that evaluating the worth of visualisations (for user, for task, for context) is now considered a crucial stage of the research process.

In brief, evaluating a visualisation technique requires engaging human participants in using the technique, observing how they use it, and determining whether it is successful in its aim. There are, however, a multitude of different experimental methodologies that a researcher can chose from in order to assess the worth of a visualisation. In addition, all experiments are constrained by the choice of parameters - including choice of participants. The consequence of this is that a substantial proportion of the information visualisation research literature comprises the results of valid, reliably-conducted empirical studies that are, however, naturally bounded by experimental choices.

Psychological research has a long history of replication – that is, repeating experiments to verify their results, particularly with changes to parameters or context. Replicating an experiment and validating it gives us more confidence in its results. Replication is rare in Information Visualisation research; there are so many interesting experimental research questions to address that researchers seldom wish to repeat work that they (or others) have already completed and published.

This PhD project will investigate replication in empirical Information Visualisation by first considering the range of experimental methodologies used in the literature and their parameter space, and analysing them in the context of replication and triangulation. Some carefully chosen studies will be replicated, and different methodologies will be compared for their effectiveness. The outcomes of this project are expected to be both (a) evidence of validated (or otherwise) replicated studies, and (more importantly) (b) an analysis of factors underlying the multi-dimensional space of experimental methodologies.

This is a very broad topic that allows the student to choose a scope (e.g. the nature of the technology, participant demographics, visualisation) of interest to them.