Study design and levels of evidence
Every claim in ophthalmology rests on a study, and the design of that study decides how far the claim can be trusted. This lesson explains the main designs from case report to meta-analysis, the biases that distort them, and the safeguards of a good trial, using landmark eye trials as examples.
The hierarchy of evidence
A study tries to answer one question: does an exposure or treatment change an outcome. Three things can produce a false answer. Chance is random variation, handled by statistics. Bias is a systematic error in how patients were chosen or measured. Confounding is a third factor that is linked to both the exposure and the outcome. Study designs differ in how well they protect against the last two.
The hierarchy of evidence ranks designs by that protection.
- Systematic reviews and meta-analyses of randomised controlled trials (strongest).
- Individual randomised controlled trials.
- Cohort studies.
- Case-control studies.
- Cross-sectional studies.
- Case series and case reports.
- Expert opinion (weakest).
The ranking is a guide, not a law. A large, careful cohort study can be more reliable than a small, badly run trial, and some questions cannot be randomised at all: nobody can be allocated to smoke. The first split to make is between observational studies, where the researcher only watches, and experimental studies, where the researcher assigns the treatment.
Observational studies
A case report describes one patient and a case series describes several. They have no comparison group, so they cannot show cause, but they are how new diseases and unexpected drug effects are first noticed. They generate hypotheses for stronger designs to test.
A cross-sectional study measures exposure and disease in a sample at one point in time, like a snapshot. It gives prevalence, the proportion who have the condition at that moment. Because both are measured together, it cannot show which came first. A survey that examines a random sample of adults to count how many have glaucoma is cross-sectional.
A case-control study starts from the outcome. People with the disease (cases) and similar people without it (controls) are identified, and the researcher looks back to compare past exposure. It is quick, cheap and the most efficient design for a rare disease, because the cases are collected rather than waited for. It cannot give incidence or prevalence, so its result is an odds ratio. Its weaknesses are the choice of controls and reliance on memory or old records.
A cohort study starts from the exposure. A group is classified as exposed or unexposed and followed over time to see who develops the outcome. Exposure is recorded before disease, so the time order is clear, and the design gives incidence and a relative risk. A prospective cohort follows people forward from today. A retrospective cohort uses existing records to do the same thing faster. Cohort studies are slow, expensive and poor for rare outcomes, since thousands must be followed to see a few cases.
| Cross-sectional | Case-control | Cohort | |
|---|---|---|---|
| Starting point | A sample at one time | Disease (cases and controls) | Exposure (exposed and unexposed) |
| Direction | None, a snapshot | Looks backward | Follows forward |
| Main measure | Prevalence | Odds ratio | Incidence, relative risk |
| Best for | Burden of common disease | Rare diseases | Rare exposures, several outcomes |
| Main weakness | Cannot show time order | Recall bias, choice of controls | Cost, time, loss to follow-up |
Randomised controlled trials
In a randomised controlled trial (RCT) the researchers assign each participant to a treatment by chance. Randomisation is the key strength. With enough participants, the groups end up alike in every factor that affects outcome, measured or not, so the only systematic difference between them is the treatment. No observational design can do this for unmeasured factors.
Three safeguards keep that advantage. Allocation concealment means the person recruiting a patient cannot know or predict the next assignment, so cannot steer sicker patients into one arm. Blinding, called masking in eye research for obvious reasons, means that participants, clinicians or outcome assessors do not know who received what. A placebo or sham procedure makes this possible. Intention to treat analysis means every participant is analysed in the group they were randomised to, whether or not they took the treatment or switched. Analysing only those who completed treatment (per protocol) breaks the balance that randomisation created and usually flatters the treatment.
| Trial | Design | Finding |
|---|---|---|
| Optic Neuritis Treatment Trial (1992) | 457 patients at 15 centres randomised to oral prednisone, intravenous methylprednisolone then oral prednisone, or oral placebo | Intravenous steroid sped recovery but acuity at six months was no better. Oral prednisone alone gave no benefit and more new episodes (relative risk 1.79) |
| Ocular Hypertension Treatment Study (2002) | 1636 people with raised pressure and no glaucoma randomised to pressure-lowering drops or observation, with masked readers | At 60 months glaucoma had developed in 4.4% with treatment and 9.5% with observation |
| LiGHT (2019) | 718 untreated patients at six UK hospitals randomised to selective laser trabeculoplasty or drops, observer-masked, intention to treat | Quality of life was the same. 74.2% of the laser group needed no drops at 36 months |
| AREDS (2001) | 3640 participants in a double-masked, placebo-controlled trial of antioxidants and zinc | Antioxidants plus zinc reduced the odds of advanced AMD (odds ratio 0.72) |
| UKPDS 33 (1998) | 3867 people with new type 2 diabetes randomised to intensive or conventional glucose control | 25% fewer microvascular endpoints, including retinal photocoagulation |
The Optic Neuritis Treatment Trial shows why a placebo arm matters. Oral steroid was common practice before the trial, and only a randomised comparison revealed that it did not help and was linked to more recurrences. Prednisone was the trial drug: the UK equivalent is prednisolone. LiGHT shows masking where a sham is impractical: patients knew whether they had laser, so the assessors measuring outcomes were masked instead.
Systematic review and meta-analysis
A systematic review answers a focused question by finding every relevant study with a search strategy that is written down in advance, then judging each study's quality by set rules. This is what separates it from an ordinary narrative review, in which an author picks the papers. A meta-analysis is the statistical step that may follow: the results of the separate studies are pooled into one estimate, with larger and more precise studies given more weight. Pooling increases power, so an effect too small for any single trial to prove can become clear.
The result is drawn as a forest plot: one line per study showing its estimate and confidence interval, and a diamond at the bottom for the pooled result. A meta-analysis is only as good as its ingredients. Heterogeneity means the studies differ too much in patients, methods or results to be sensibly combined. Publication bias arises because studies with positive results are more likely to be published, so the pooled estimate from published work can overstate the benefit.
Bias
Bias is a systematic error that pushes the result in one direction. A bigger sample does not fix it: a biased study of ten thousand people gives a precise wrong answer. The four types that are examined most often are below.
| Bias | What goes wrong | Reduced by |
|---|---|---|
| Selection bias | The groups compared differ from the start, or the sample does not represent the population | Randomisation with allocation concealment, random sampling, careful choice of controls |
| Recall bias | Cases remember or report past exposures more fully than controls | Using records made before the disease, blinding participants to the hypothesis |
| Observer bias | The assessor's knowledge of the group influences the measurement | Masked outcome assessors, objective outcomes |
| Attrition bias | Participants who drop out differ from those who stay, and drop out unequally between groups | Good follow-up, intention to treat analysis |
Confounding
A confounder is a factor associated with the exposure and, separately, a cause of the outcome, without lying on the causal path between them. It can create an association that is not causal or hide one that is. The textbook example is coffee and heart disease: coffee drinkers were more likely to smoke, and smoking causes heart disease, so coffee appeared guilty. In eye research age is the usual suspect, because age is linked to almost every exposure and to cataract, glaucoma and macular degeneration.
Confounding can be dealt with at the design stage or in the analysis.
- Randomisation: balances known and unknown confounders between groups. It is the only method that handles the unknown ones.
- Restriction: recruit only one category, for example non-smokers.
- Matching: choose each control to match a case for age and sex.
- Stratification: analyse smokers and non-smokers separately.
- Regression: adjust for several confounders at once, giving an adjusted estimate.
Summary
- The hierarchy of evidence runs from expert opinion and case reports, through cross-sectional, case-control and cohort studies, to randomised trials and systematic reviews of trials.
- A cross-sectional study is a snapshot that gives prevalence and cannot show which came first.
- A case-control study starts from disease, looks back for exposure, suits rare diseases and gives an odds ratio.
- A cohort study starts from exposure, follows forward, and gives incidence and relative risk.
- Randomisation balances known and unknown confounders, which no observational design can do.
- Allocation concealment prevents selection bias at entry, masking prevents observer bias, and intention to treat analysis guards against attrition bias.
- A systematic review uses a predefined search and appraisal, and a meta-analysis pools the results, shown as a forest plot.
- Recall bias is the characteristic weakness of case-control studies.
- A confounder is linked to both exposure and outcome and is not on the causal path.
- The Optic Neuritis Treatment Trial showed that oral prednisone alone did not help and was linked to more recurrences.
Check yourself
Quick recall on this lesson. These are easier than the question bank.
Written for revision. It is not a source of clinical advice, and guidance changes.