The Solomon four-group design is a type of experiment where participants get randomly assigned to either 1 of 4 groups that differ in whether the participants receive the treatment or not, and whether the outcome of interest is measured once or twice in each group.
The four groups in this design are (see figure below):
- A treatment group with both pre-intervention and post-intervention measurements (a.k.a. pretest and posttest)
- A control group with both pretest and posttest measurements
- A treatment group with only a posttest measurement
- A control group with only a posttest measurement
The objective is to assess the efficacy of the treatment (or intervention).
Important notes regarding the Solomon four-group design:
- The assignment of participants is done at random.
- The outcome of interest is measured simultaneously across groups.
- The design is asymmetric: Only participants in groups 1 and 2 receive pretest measurements (1A and 2A in the figure above), however, all participants receive posttest measurements (1B, 2B, 3B and 4B).
Advantages of the Solomon four-group design
Why would we need 4 groups? More specifically, what additional information can we get with a Solomon four-group design compared to a standard experimental design with 2 groups (1 treatment group and 1 control group)?
First, let’s start with:
How the Solomon four-group design resembles a two-group experimental design?
Notice that the first 2 groups of this design resemble the pretest-posttest control group design. And therefore, the first 2 groups are enough to control threats to internal validity such as the effects of:
- Selection
- History
- Maturation
- Mortality
- Regression to the mean
- Instrumentation
(For a detailed explanation, see my other article: Pretest-Posttest Control Group Design)
What makes the Solomon four-group design better than a standard two-group design?
The Solomon four-group design was developed to:
- Control threats to internal validity: Such as bias and confounding. Something that a standard (two-group) experimental design can control.
- Control threats to external validity: Such as pretest sensitization. Something that a standard (two-group) experimental design cannot control.
What is pretest sensitization?
Pretest sensitization (a.k.a. interaction between pretest and treatment) occurs when the use of a pretest increases or decreases the responsiveness of the participants to the study intervention.
For example:
Consider a pretest that contains questions that non-deliberately make participants more aware and concerned regarding the consequences of smoking. Such pretest may sensitize participants to make them more responsive to a smoking cessation intervention.
In fact, it could be that this intervention is only effective for participants who took such pretest.
Why pretest sensitization is an external validity problem?
The general population from which participants were selected for the study will not be pretested, therefore, will not be sensitized to the intervention.
Consequently, the effect of the intervention on the general population will be different than the one detected in the study. Therefore, the study will suffer from a generalizability (a.k.a. an external validity) problem.
How the Solomon four-group design helps detecting pretest sensitization?
An interaction between pretest and treatment (or pretest sensitization) would exist if the pretest influences the outcome of the posttest, only in the presence of the intervention.
This can be assessed in 3 steps:
Step #1: Calculate the effect of the pretest on the outcome of the posttest in the presence of the intervention: This is the difference between 1B and 3B.
Step #2: Calculate the effect of the pretest on the outcome of the posttest in the absence of the intervention:
This the difference between 2B and 4B.
Step #3: Compare the result of step 1 with that of step 2 (that is “the difference between 1B and 3B” with “the difference between 2B and 4B”):
A difference between these 2 quantities means that the effect of the pretest is different in the presence of the intervention, in other words, there is an interaction between pretest and treatment.
Limitations of the Solomon four-group design
The Solomon four-group design is not used a lot in practice because of its high cost, since compared to a standard design with 2 groups it requires twice the sample size, time, materials, resources, and personnel/work.
Example of a study that used the Solomon four-group design
Dehghan et al. investigated VR (Virtual Reality) technology as a method to reduce anxiety in children undergoing surgery.
Design details
A Solomon four-group design was used in order to control any pretest sensitization.
The pretest and posttest measured the anxiety score by using a standardized questionnaire.
The intervention used was a VR technique that simulates step-by-step going into an operation room.
Results
The study concluded that VR technology reduced pre-operative anxiety in children by acting as a distraction method.
Limitations
While the treatment allocation was done at random, participants included in the study were sampled in a non-random way from the population of all children undergoing surgery.
This means that the study participants may not represent well the population of interest, therefore causing a problem of generalizability of the study results.
Note however, that this external validity issue is not specific to the Solomon four-group design, instead it is a consequence of the sampling method used.
References
- Campbell DT, Stanley JC. Experimental and Quasi-Experimental Designs for Research. Wadsworth; 1963.
- Solomon Four-Group Design. In: The SAGE Encyclopedia of Educational Research, Measurement, and Evaluation. SAGE Publications, Inc.; 2018.