Randomized experiment

A randomized experiment is an experiment, observation units are allocated in the different in their effects to be evaluated treatments at random. The random assignment means that the observation units should not differ on average ( with the exception of treatments). To the contrary, the quasi - experiment.

Ronald Fisher's randomized experiment

Ronald Fisher is considered the inventor of the randomized experiment. In The Design of Experiments ( 1935), he described his concept is an example known today. In this case you want to check whether a woman based on a taste test can distinguish a cup of tea with milk, whether the cup of the milk or the tea was added first. At Fishers time was the dominant approach to such questions is to keep covariates that could affect actual results, constant. In this case, this would mean, for example, the temperature and strength of the tea, the amount of added sugar or milk, or the nature of the cup with both treatments ( first tea, milk first) align exactly. Fisher rejected this approach for two reasons. First, it was impossible. Secondly, it is even if it would almost possible to expensive.

Instead of the prevailing dictum to keep all factors constant, Fisher suggested before, to keep anything constant, namely, by randomization. Fisher suggested to clarify the concrete question, only to be charged with four cups of milk with tea, and only then to fill four other cups with tea with milk. The woman is informed that only four cups of milk, then tea, and four others only tea, have then get milk, but not that these are cups each. The eight cups are presented in random order before the woman. Your task is to allocate the cups each the correct group by taste tests. The number of cups is so. The order of presentation of the cups is a random variable, and each presentation is the realization of this random variable. A particular presentation can be described, for example, with. All presentations are possible elements of the set Ω of all possible presentations. Third, a result will be observed. If the woman in the example above of all cups assign correct, would be. Last to decide whether the null hypothesis ( woman can not taste if the first cup of tea or milk is added) has to be rejected at a given significance level the experiment.

Before carrying out randomized experiments all possible outcomes should be predicted. Central to this is the number of elements in Ω. Since this experiment Fishers is a permutation, can be calculated as follows:

There are thus 70 possible arrangements ( and also 70 possible outcomes ). Fisher then asked how large the probability is that the woman assigns correctly by chance alone all eight cups. This probability is. If so found, one can conclude with an error probability of less than 2% that the woman actually has the ability herauszuschmecken the order of the pouring out of tea and milk. Under a less strict definition of ability are welcome by the two allocation error, the error probability would be already. Under this definition, the experiment described above did not have adequate statistical power and more.

Core elements

Rosenbaum (2002) summarizes the key elements of a randomized experiment as follows:

• Experiments do not require homogeneity of the treatment units
• Experiments do not require a random sample of a population of treatment units
• In order to draw a valid conclusion about the effects of treatment from one experiment, the treatments must be randomly assigned to the treatment units
• Chance to play in the experiment only in connection with the assignment of treatments to treatment units a role.

Types of randomized experiments and statistical tests

Fisher's method was the gold standard in many areas such as agriculture, computer science, manufacturing processes, medical or welfare. In addition to the completely randomized experiment variants exist such as the block design (block diagram) or paired randomized experiments. In addition, a number of statistical tests (as opposed to non-randomized experiments) commented in randomized experiments with virtually no assumptions exist. Rosenbaum (2002) summarizes it as follows:

• Tests for binary results: Fisher's exact test, Mantel-Haenszel statistic, McNemar test
• Tests for ordinal results: mantle (1959 ) Extension of the Mantel-Haenszel statistic
• Tests for a single stratum with interval scale and ratio scale: Wilcoxon rank sum test
• Tests for ordinal results ( with a large number of strata in comparison to the number of samples ): Hodges -Lehmann estimator

Criticism of social randomized experiments

Although the randomized experiment since Fisher has proven in many applications very useful, have been criticized in the past three decades, which was directed against the human use. In particular, it was criticized that some people are denied treatments by assigning control groups, which may be unethical and / or illegal.

James Heckman and colleagues also emphasized the need for modeling the processes that lead people to participation or non -participation in programs or treatments. The criticism was also committed to combating the fundamental assumption of the randomized experiment that randomization selection bias eliminated.