The one-group posttest-only design (a.k.a. one-shot case study) is a type of quasi-experiment in which the outcome of interest is measured only once after exposing a non-random group of participants to a certain intervention.
The objective is to evaluate the effect of that intervention which can be:
- A training program
- A policy change
- A medical treatment, etc.
As in other quasi-experiments, the group of participants who receive the intervention is selected in a non-random way (for example according to their choosing or that of the researcher).
The one-group posttest-only design is especially characterized by having:
- No control group
- No measurements before the intervention
It is the simplest and weakest of the quasi-experimental designs in terms of level of evidence as the measured outcome cannot be compared to a measurement before the intervention nor to a control group.
So the outcome will be compared to what we assume will happen if the intervention was not implemented. This is generally based on expert knowledge and speculation.
Next we will discuss cases where this design can be useful and its limitations in the study of a causal relationship between the intervention and the outcome.
Advantages and Limitations of the one-group posttest-only design
Advantages of the one-group posttest-only design
1. Advantages related to the non-random selection of participants:
- Ethical considerations: Random selection of participants is considered unethical when the intervention is believed to be harmful (for example exposing people to smoking or dangerous chemicals) or on the contrary when it is believed to be so beneficial that it would be malevolent not to offer it to all participants (for example a groundbreaking treatment or medical operation).
- Difficulty to adequately randomize subjects and locations: In some cases where the intervention acts on a group of people at a given location, it becomes infeasible to adequately randomize subjects (ex. an intervention that reduces pollution in a given area).
2. Advantages related to the simplicity of this design:
- Feasible with fewer resources than most designs: The one-group posttest-only design is especially useful when the intervention must be quickly introduced and we do not have enough time to take pre-intervention measurements. Other designs may also require a larger sample size or a higher cost to account for the follow-up of a control group.
- No temporality issue: Since the outcome is measured after the intervention, we can be certain that it occurred after it, which is important for inferring a causal relationship between the two.
Limitations of the one-group posttest-only design
1. Selection bias:
Because participants were not chosen at random, it is certainly possible that those who volunteered are not representative of the population of interest on which we intend to draw our conclusions.
2. Limitation due to maturation:
Because we don’t have a control group nor a pre-intervention measurement of the variable of interest, the post-intervention measurement will be compared to what we believe or assume would happen was there no intervention at all.
The problem is when the outcome of interest has a natural fluctuation pattern (maturation effect) that we don’t know about.
So since certain factors are essentially hard to predict and since 1 measurement is certainly not enough to understand the natural pattern of an outcome, therefore with the one-group posttest-only design, we can hardly infer any causal relationship between intervention and outcome.
3. Limitation due to history:
The idea here is that we may have a historical event, which may also influence the outcome, occurring at the same time as the intervention.
The problem is that this event can now be an alternative explanation of the observed outcome. The only way out of this is if the effect of this event on the outcome is well-known and documented in order to account for it in our data analysis.
This is why most of the time we prefer other designs that include a control group (made of people who were exposed to the historical event but not to the intervention) as it provides us with a reference to compare to.
Example of a study that used the one-group posttest-only design
In 2018, Tsai et al. designed an exercise program for older adults based on traditional Chinese medicine ideas, and wanted to test its feasibility, safety and helpfulness.
So they conducted a one-group posttest-only study as a pilot test with 31 older adult volunteers. Then they evaluated these participants (using open-ended questions) after receiving the intervention (the exercise program).
The study concluded that the program was safe, helpful and suitable for older adults.
What can we learn from this example?
1. Work within the design limitations:
Notice that the outcome measured was the feasibility of the program and not its health effects on older adults.
The purpose of the study was to design an exercise program based on the participants’ feedback. So a pilot one-group posttest-only study was enough to do so.
For studying the health effects of this program on older adults a randomized controlled trial will certainly be necessary.
2. Be careful with generalization when working with non-randomly selected participants:
For instance, participants who volunteered to be in this study were all physically active older adults who exercise regularly.
Therefore, the study results may not generalize to all the elderly population.
References
- Shadish WR, Cook TD, Campbell DT. Experimental and Quasi-Experimental Designs for Generalized Causal Inference. 2nd Edition. Cengage Learning; 2001.
- Campbell DT, Stanley J. Experimental and Quasi-Experimental Designs for Research. 1st Edition. Cengage Learning; 1963.