Skip to content
Pass Duke Elder
Textbook / Chapter 16 / Statistics, epidemiology and public health

Study design and levels of evidence

Every claim in ophthalmology rests on a study, and the design of that study decides how far the claim can be trusted. This lesson explains the main designs from case report to meta-analysis, the biases that distort them, and the safeguards of a good trial, using landmark eye trials as examples.

8 minute read0 linked questions

The hierarchy of evidence

A study tries to answer one question: does an exposure or treatment change an outcome. Three things can produce a false answer. Chance is random variation, handled by statistics. Bias is a systematic error in how patients were chosen or measured. Confounding is a third factor that is linked to both the exposure and the outcome. Study designs differ in how well they protect against the last two.

The hierarchy of evidence ranks designs by that protection.

  1. Systematic reviews and meta-analyses of randomised controlled trials (strongest).
  2. Individual randomised controlled trials.
  3. Cohort studies.
  4. Case-control studies.
  5. Cross-sectional studies.
  6. Case series and case reports.
  7. Expert opinion (weakest).

The ranking is a guide, not a law. A large, careful cohort study can be more reliable than a small, badly run trial, and some questions cannot be randomised at all: nobody can be allocated to smoke. The first split to make is between observational studies, where the researcher only watches, and experimental studies, where the researcher assigns the treatment.

Observational studies

A case report describes one patient and a case series describes several. They have no comparison group, so they cannot show cause, but they are how new diseases and unexpected drug effects are first noticed. They generate hypotheses for stronger designs to test.

A cross-sectional study measures exposure and disease in a sample at one point in time, like a snapshot. It gives prevalence, the proportion who have the condition at that moment. Because both are measured together, it cannot show which came first. A survey that examines a random sample of adults to count how many have glaucoma is cross-sectional.

A case-control study starts from the outcome. People with the disease (cases) and similar people without it (controls) are identified, and the researcher looks back to compare past exposure. It is quick, cheap and the most efficient design for a rare disease, because the cases are collected rather than waited for. It cannot give incidence or prevalence, so its result is an odds ratio. Its weaknesses are the choice of controls and reliance on memory or old records.

A cohort study starts from the exposure. A group is classified as exposed or unexposed and followed over time to see who develops the outcome. Exposure is recorded before disease, so the time order is clear, and the design gives incidence and a relative risk. A prospective cohort follows people forward from today. A retrospective cohort uses existing records to do the same thing faster. Cohort studies are slow, expensive and poor for rare outcomes, since thousands must be followed to see a few cases.

Cross-sectionalCase-controlCohort
Starting pointA sample at one timeDisease (cases and controls)Exposure (exposed and unexposed)
DirectionNone, a snapshotLooks backwardFollows forward
Main measurePrevalenceOdds ratioIncidence, relative risk
Best forBurden of common diseaseRare diseasesRare exposures, several outcomes
Main weaknessCannot show time orderRecall bias, choice of controlsCost, time, loss to follow-up
The main observational designs

Randomised controlled trials

In a randomised controlled trial (RCT) the researchers assign each participant to a treatment by chance. Randomisation is the key strength. With enough participants, the groups end up alike in every factor that affects outcome, measured or not, so the only systematic difference between them is the treatment. No observational design can do this for unmeasured factors.

Three safeguards keep that advantage. Allocation concealment means the person recruiting a patient cannot know or predict the next assignment, so cannot steer sicker patients into one arm. Blinding, called masking in eye research for obvious reasons, means that participants, clinicians or outcome assessors do not know who received what. A placebo or sham procedure makes this possible. Intention to treat analysis means every participant is analysed in the group they were randomised to, whether or not they took the treatment or switched. Analysing only those who completed treatment (per protocol) breaks the balance that randomisation created and usually flatters the treatment.

TrialDesignFinding
Optic Neuritis Treatment Trial (1992)457 patients at 15 centres randomised to oral prednisone, intravenous methylprednisolone then oral prednisone, or oral placeboIntravenous steroid sped recovery but acuity at six months was no better. Oral prednisone alone gave no benefit and more new episodes (relative risk 1.79)
Ocular Hypertension Treatment Study (2002)1636 people with raised pressure and no glaucoma randomised to pressure-lowering drops or observation, with masked readersAt 60 months glaucoma had developed in 4.4% with treatment and 9.5% with observation
LiGHT (2019)718 untreated patients at six UK hospitals randomised to selective laser trabeculoplasty or drops, observer-masked, intention to treatQuality of life was the same. 74.2% of the laser group needed no drops at 36 months
AREDS (2001)3640 participants in a double-masked, placebo-controlled trial of antioxidants and zincAntioxidants plus zinc reduced the odds of advanced AMD (odds ratio 0.72)
UKPDS 33 (1998)3867 people with new type 2 diabetes randomised to intensive or conventional glucose control25% fewer microvascular endpoints, including retinal photocoagulation
Landmark eye trials and what each illustrates

The Optic Neuritis Treatment Trial shows why a placebo arm matters. Oral steroid was common practice before the trial, and only a randomised comparison revealed that it did not help and was linked to more recurrences. Prednisone was the trial drug: the UK equivalent is prednisolone. LiGHT shows masking where a sham is impractical: patients knew whether they had laser, so the assessors measuring outcomes were masked instead.

Systematic review and meta-analysis

A systematic review answers a focused question by finding every relevant study with a search strategy that is written down in advance, then judging each study's quality by set rules. This is what separates it from an ordinary narrative review, in which an author picks the papers. A meta-analysis is the statistical step that may follow: the results of the separate studies are pooled into one estimate, with larger and more precise studies given more weight. Pooling increases power, so an effect too small for any single trial to prove can become clear.

The result is drawn as a forest plot: one line per study showing its estimate and confidence interval, and a diamond at the bottom for the pooled result. A meta-analysis is only as good as its ingredients. Heterogeneity means the studies differ too much in patients, methods or results to be sensibly combined. Publication bias arises because studies with positive results are more likely to be published, so the pooled estimate from published work can overstate the benefit.

Bias

Bias is a systematic error that pushes the result in one direction. A bigger sample does not fix it: a biased study of ten thousand people gives a precise wrong answer. The four types that are examined most often are below.

BiasWhat goes wrongReduced by
Selection biasThe groups compared differ from the start, or the sample does not represent the populationRandomisation with allocation concealment, random sampling, careful choice of controls
Recall biasCases remember or report past exposures more fully than controlsUsing records made before the disease, blinding participants to the hypothesis
Observer biasThe assessor's knowledge of the group influences the measurementMasked outcome assessors, objective outcomes
Attrition biasParticipants who drop out differ from those who stay, and drop out unequally between groupsGood follow-up, intention to treat analysis
Common biases and how they are reduced

Confounding

A confounder is a factor associated with the exposure and, separately, a cause of the outcome, without lying on the causal path between them. It can create an association that is not causal or hide one that is. The textbook example is coffee and heart disease: coffee drinkers were more likely to smoke, and smoking causes heart disease, so coffee appeared guilty. In eye research age is the usual suspect, because age is linked to almost every exposure and to cataract, glaucoma and macular degeneration.

Confounding can be dealt with at the design stage or in the analysis.

  • Randomisation: balances known and unknown confounders between groups. It is the only method that handles the unknown ones.
  • Restriction: recruit only one category, for example non-smokers.
  • Matching: choose each control to match a case for age and sex.
  • Stratification: analyse smokers and non-smokers separately.
  • Regression: adjust for several confounders at once, giving an adjusted estimate.

Summary

  • The hierarchy of evidence runs from expert opinion and case reports, through cross-sectional, case-control and cohort studies, to randomised trials and systematic reviews of trials.
  • A cross-sectional study is a snapshot that gives prevalence and cannot show which came first.
  • A case-control study starts from disease, looks back for exposure, suits rare diseases and gives an odds ratio.
  • A cohort study starts from exposure, follows forward, and gives incidence and relative risk.
  • Randomisation balances known and unknown confounders, which no observational design can do.
  • Allocation concealment prevents selection bias at entry, masking prevents observer bias, and intention to treat analysis guards against attrition bias.
  • A systematic review uses a predefined search and appraisal, and a meta-analysis pools the results, shown as a forest plot.
  • Recall bias is the characteristic weakness of case-control studies.
  • A confounder is linked to both exposure and outcome and is not on the causal path.
  • The Optic Neuritis Treatment Trial showed that oral prednisone alone did not help and was linked to more recurrences.

Check yourself

Quick recall on this lesson. These are easier than the question bank.

1. Which design begins with people who have a disease and people who do not, then looks back at past exposure?
2. Analysing every participant in the group they were randomised to, whatever treatment they actually received, is called what?
3. Patients with a disease report past exposures more fully than healthy controls. Which bias is this?
4. In the Optic Neuritis Treatment Trial, which arm had a higher rate of new episodes of optic neuritis than placebo?
Practise this topic

Written for revision. It is not a source of clinical advice, and guidance changes.