Experimental Methods for Computer Science Research

Experimental methods comprise the set of skills and techniques for minimizing error in acquiring and communicating measurements. One undervalued aspect of experimental research is the importance of communicating experimental outcomes clearly to readers. The way in which an experiment is done can affect what is written about it, and the way the results are described can affect the way an experiment is done. It´s a two-way street, and each direction counts. The first part of the tutorial will cover a range of methodological details that are critical to good experimentation. This will be done in the context of writing a conference or journal paper that includes such details. This part of the tutorial is geared particularly toward students, but professionals may benefit from the different perspective that the tutorial offers. The second part of the tutorial considers the mechanics of actually running a complicated experiment. It deals with a number of practical aspects of the design, construction, and conduct of experiments in computer science research. The goal will be to see how a particular experiment in keystroke forensics was designed, how the design was extended into a plan of action, how the action was executed in the laboratory, and how the data were wrapped up at the end. Students will see the consequences of poor design decisions, and how to mitigate them. Students will learn by example how to design and conduct rigorous experiments whose results are dependable.