Why it's important to understand the different types of clinical studies
Dear Reader,
Welcome to the first inaugural Kwok Reads post! For our first topic of discussion, I thought it would be essential to understand the different types of studies so we can make informed decisions regarding our health that are based on strong evidence.
I’m sure that you’ve noticed that there is a plethora of conflicting information on whether a food or nutrient is beneficial or not. One report can indicate that consumption of sugar sweetened beverages can increase blood pressure, whereas another would state that there is no effect. My least favorite conclusion? The data is inconclusive. This is very common and so very frustrating! The media tends to report on significant and exciting findings no matter how weak the evidence. I want us to be able to think for ourselves and decide whether the evidence is reliable.
OK, I admit that this topic may not be everyone’s cup of tea, but it’s something really worth checking out because I believe that being able to look at the evidence with a critical eye is an essential skill that helps us make informed decisions about our health and well being.
So, here goes. Make a cup of your favorite coffee or tea, sit back and put on your thinking caps.
After reading this post, you will have an increased awareness and understanding of:
- Differences between observational and experimental studies
- Pros and cons of these types of studies
- Why you should avoid making decisions about your diet based on observational data alone
- And be able to come to a conclusion on the strength of evidence that is being presented
There will be checkpoints along the way. Use them as an opportunity to take a breath and process what you've learned so far.
Most published data linking food with chronic illnesses tend to be observational studies and not experimental studies (Maki, 2014). Although observational studies are less time consuming and costly to conduct, they seldom prove much.
Understanding observational studies
In observational studies, investigators merely observe a group of people over time and see what happens. For example, a group of 100 people are followed over 5 years to see how many eventually develop diabetes. This is why observational studies may also be known as epidemiologic studies – they can help observe disease prevalence. To take it a step further, the investigators may notice that the participants who did not develop diabetes are leaner and tended to drink more coffee (at least 4 cups a day) than those who did develop diabetes. They may conclude that people with lower body weight and who drank at least 4 cups of coffee a day had a lower risk of developing diabetes.
OK. PAUSE. This is a fictional scenario. I am not advocating that you, dear Reader, start throwing back pots of coffee to reduce diabetes risk. This would be making a decision about your diet/habit based on very weak evidence.
Despite the limitations of observational studies, I still include them when I am conducting a literature search. Why? They are helpful for hypothesis forming. The associations presented by observational studies helps me ask other questions such as:
- What is it about coffee that links it to a reduced diabetes risk?
Is it because of caffeine intake?
What antioxidants or other phytonutrients (plant chemicals) are found in coffee that can be beneficial?
- Are there differences in characteristics between coffee drinkers and non-drinkers?
Younger vs. older
Biologically male vs. female
Lifestyle- smoker? Exercise vs. sedentary
Education level
As you can see, observational studies only provide a snapshot and we really should dive deeper to find more supporting evidence before we can make informed decisions for ourselves. One way to do this is to look at experimental studies.
Checkpoint 1: Observational studies are aptly named for what they provide – observations and associations, but NOT cause. Observational studies are NOT designed to discern causal relationships.
Checkpoint 2: Observational studies provide mostly weak evidence.
Understanding experimental studies
Experimental studies (also known as clinical trials or interventional studies) assess the effects of a given intervention. Interventions may include, but are not limited to, a prescribed diet/exercise regimen, medication/supplement, or patient education. Experimental studies test a certain intervention against another intervention or to a control group (placebo). Because of this, evidence derived from experimental studies is usually stronger than what is derived from observational studies.
Randomized control trials (RCTs) are typically considered the gold standard for reviewing evidence. I say ‘typically’ because the evidence is only as good as the study design. And here is why:
- Randomization is important because a study participant is assigned to an intervention by chance. This helps with reducing bias. A trial can be open labeled or blinded.
- In open labeled trials, both the investigators and participants know what intervention is being given. This design may be chosen due to limitations time, cost and ethics. For example, it would be unethical to not provide cancer treatment.
- On the contrary, in a blinded trial either the participants alone are not aware of the intervention being given (single-blinded) or both the participants and investigators don’t know (double-blinded).
- To add to the complexity, a trial can be double blinded randomized placebo controlled. This is where an intervention is compared to no intervention.
Take a breath. Are you still with me? Good. Let’s continue. If not, that’s ok too. Take a breath and contemplate the checkpoints and infographics. Use them like rest stops. Continue moving forward when you feel you are ready.
So, we’ve established that study design is important. In addition to randomization, it is also important to pay attention to who is being included in the study group. What is the average age of the group? What is the ethnic composition? Are all genders included and if so, what is the ratio? These factors contribute to the external validity of the evidence presented.
It would not be appropriate to apply the evidence from an all young and healthy male demographic to the middle aged or older general population. The measured outcomes should be applicable to a similar set of demographics in the general population. For example, clinical trials that help establish an appropriate vitamin D dosing range in a younger and lean group may not be applicable to those who are obese and/or older.
It should be noted whether the study participants are human or partly human! Many studies are done with mice (murine model). The animals are assessed under controlled environments with set diets. These studies lack external validity, so I consider the level of evidence weak. However, murine studies are very helpful in trying to understand the mechanisms of action of drugs, nutrients, or other interventions. As for the partly human studies? These are called in vitro studies (aka petri dish studies). Cells (may be human or not) are cultivated in petri dishes and interventions are introduced within a controlled environment to assess their effects. These are also helpful in trying to understand how drugs/supplements/nutrients exert their effects.
Finally, measures of outcomes are just as important. We want to make sure that the outcomes assessed are relevant (e.g. measuring hair length in a heart disease trial). Ideally RCTs would measure the development of a disease or number of deaths. However, these outcomes may take several decades and the study becomes too expensive to complete. Alternatively, RCTs measure changes in biomarkers that may be associated with disease. For example, hemoglobin A1c and diabetes. But this is assuming that biomarkers have a direct impact on outcome – good or bad. This might not be the case, but it’s the best we have at this time.
Checkpoint 3: Experimental studies (aka interventional studies, clinical trials) provider stronger evidence than observational studies.
Checkpoint 4: Evidence derived from experimental studies are only as good as its design and outcomes measured.
Checkpoint 5: External validity is like asking yourself what you have in common with the group being studied.
How can we improve the process? I am hoping that more funding can be provided to larger randomized trials with longer durations. It may cost millions, but just think of the millions we can save if it means helping people better prevent or treat chronic illnesses (Trepanowski, 2018). Dear Reader, what answers are you seeking? What would your ideal RCT design look like to test your hypothesis? Please comment below!
We made it! Thank you very much for taking the time and coming along with me to consider why it is so important to take a closer look at the evidence. This is a glimpse into my thought process whenever I approach a topic.
There are other statistical aspects that I look at in order for me to make my own conclusions if an intervention has any clinical significance. Leave a comment below and let me know if you’d be interested in diving deeper into this topic! If you like what you see, comment and SUBSCRIBE!
Until next time!
References:
Maki, K. C., Slavin, J. L., Rains, T. M., & Kris-Etherton, P. M. (2014). Limitations of observational evidence: implications for evidence-based dietary recommendations. Advances in nutrition (Bethesda, Md.), 5(1), 7–15. https://doi.org/10.3945/an.113.004929
Trepanowski, J. F., & Ioannidis, J. (2018). Perspective: Limiting Dependence on Nonrandomized Studies and Improving Randomized Trials in Human Nutrition Research: Why and How. Advances in nutrition (Bethesda, Md.), 9(4), 367–377. https://doi.org/10.1093/advances/nmy014