|
|
Experimental design: the standard model
The standard model
For about 50 years, experimental scientists have used a standard template for
planning the conduct of experiments.
1. Determine what question(s) you want to answer
This is the first, and probably the most important step. If the experimenters
do not know what they want to find out, the results will invariably be
difficult to interpret. It may be necessary to consult a statistician at this
stage to ensure that the questions being asked can in fact be answered with a
satisfactory degree of confidence
2. Assert a hypothesis and a null hypothesis
The questions are phrased in the form of a hypothesis that is either true or
false. This leads to a null hypothesis, which will be accepted or rejected
depending on the results collected
3. Assert the confidence level required to reject the null hypothesis, and
the desired statistical power
We want to reject the null hypothesis if we are reasonably confident that it is
false. This confidence is expressed as the probability of rejecting the null
hypothesis as a result of chance random variations in the experimental data.
The statistical power is a measure of the sensitivity of the experiment.
A high power corresponds to a high likelihood of rejecting the null hypothesis,
if it is in fact false. However, the number of experimental runs and
experimental subjects increases with the power
4. Determine the method required to test the hypothesis
This includes determining the method of sampling, the order of tests and to
whom they will be applied, and the mechanism of data analysis
5. Determine the number of experimental subjects and experimental runs required to
achieve the required confidence and power
In practice it is often difficult to get enough experimental subjects to
achieve the optimal statistical power. This is not necessarily an indication
that the experiment should not proceed, but it does mean that extra care has to
be taken in the interpretation of results
6. Collect data
Data should be collected in such a way as to minimize bias. This may mean, for
example, that the experimenter should minimize contact with the experimental
subjects, to avoid influencing them unduly
7. Accept or reject the null hypothesis
The decision to accept or reject will usually be on the basis of the outcome of
a statistical test
|
