Experimental design: generalization

A good experiment allows its results to be generalized

As a potential buyer of cat food, what would be important to me is whether my cat would prefer MoggyScoff to other leading brands. I am not interested in what other people's cats like.

However, I could reasonably assume that my cat was not very different from other cats. So if a sufficient number of other people's cats preferred MoggyScoff to other brands, I would be reasonably happy to buy it myself. The problem with questioning ten people at a supermarket checkout is that the result, while it may be perfectly true of these ten peoples' cats, probably does not allow us to make any claim about other peoples' cats.

We say that the finding is not generalizable or does not generalize. An experiment that does generalize allows us to make inferences about the results that would be found if the experiment were carried out on a large population.