Experimental design: a hypothesis for the case study

Example hypotheses in the Bodgett and Scarper experiment

``Information retrieval tasks can be accomplished more quickly with the new system than comparable systems''

We could test this hypothesis by timing experimenal subjects as they use the different systems

``Information retrieval tasks can be accomplished more accurately with the new system than comparable systems''

We could test this by setting the subjects tasks which need not just to be completed, but completed with a certain accuracy. Then we could judge the accuracy of each subject by counting the number of tasks that were completed with an outcome judged to be of adequate accuracy

``Users report that they prefer to use the new system than comparable systems''

We could test this by asking the users to rate their subjective impressions of using the two systems. Because of the subjectivity, it is important to minimize inter-subject variability. This can be achieved by getting the subjects to use the different systems sequentially, and analysing the difference between their ratings of the different systems

``Users require fewer user interface gestures to accomplish a particular task with the new system that comparable systems''

In this case we merely need to count the number of user interface actions invoked each time a task is undertaken by a given subject. This is tedious work, but not difficult