|
|
Experimental design: comments on generalization of the case study
Generalization issues in the Bodgett and Scarper experiment
The problem with the way Bodgett and Scarper conducted their experiment is not
necessarily the size of the experimental group, although this may indeed be a
problem, but the population which the sample represents. There is no reason to
think that a group of the experimenters' family and friends are representative
of anything useful. What population did the experimenters have in mind?
Probably they did not think about this at all; if they had they would have had
to decide whether the sample should represent experienced computer users,
computing professionals, the general public, or something else entirely.
In any event, it needs to be obvious to the reader of the paper
how far the results can be generalized to cases in which they (the readers) are
interested.
Even if a useful experimental group were selected, there could still be
problems with bias. People like to please, on the whole, and will tend to
return results that they think the experimenters will like. They may not do
this consciously, but it is no less disruptive for all that.
|
