Return to Evaluating the Evidence

Supplement:
Evaluating the validity of a Etiology/Harm study

1. Were there clearly identified comparison groups that were similar with respect to important determinants of outcome, other than the one of interest?

The choice of comparison groups has a significant influence on the credibility of the study results. Careful attention should be placed on the study type and design. Characteristics of the exposed and non-exposed patients need to be carefully documented and their comparability needs to be demonstrated.

2. Were the exposures and outcomes measured in the same way in the groups being compared?

Both groups should be measured by the same method. The measurement should avoid any kind of bias, whether from recall bias (by patient motivation to help) or by interviewer bias (probing by interviewers for the "right" answer). Sometimes using objective data, such as medical records, can eliminate bias.

3. Was follow-up sufficiently long and complete?

Patients unavailable for complete follow-up may compromise the validity of the experiment because often these patients have very different outcomes than those that stayed with the study. This information must be factored into the study results.

4. Is the temporal relationship correct?

The exposure must precede the adverse outcome. The intervention, whether therapeutic or harmful, must have happened before the adverse outcome occurred.

5. Is there a dose-response gradient?

The results are more useful if it can be shown that the adverse effect is increased when the quantity or duration of exposure to the harmful agent is increased. (The risk of lung cancer increases with the number of cigarettes smoked per day.) In terms of prevention, this relationship means that the greater the quantity or duration of a beneficial exposure, the less likely that an adverse event will occur.

Key issues for Etiology Studies: similarity of comparison groups outcomes and exposures measured same for both groups follow-up of sufficient length (80% or better) temporal relationship dose response gradient

 

What are the Results? Strength of inference:   Outcome + Outcome - Exposure + a b Exposure - c d RCT or Prospective cohort studies: Relative Risk (RR) is the risk of the outcome in the exposed group divided by the risk of the outcome in the unexposed group: RR = a /(a + b) divided by c/(c + d) "RR of 3.0 means that the outcome occurs 3 times more often in those exposed versus unexposed." Case-control studies: Odds Ratio (OR) is the odds of previous exposure in a case divided by the odds of exposure in a control patient: OR = (a / c) divided by (b / d) = ad/bc "OR of 3.0 means that cases were 3 times more likely to have been exposed than were control patients." Confounding Variable is one whose influence distorts the true relationship between a potential risk factor and the clinical outcome of interest.

Source: Levine M ; Walter S ; Lee H ; Haines T ; Holbrook A ; Moyer V. Users' guides to the medical literature. IV. How to use an article about harm. Evidence-Based Medicine Working Group. JAMA 1994 May 25; 271(20):1615-9.

Return to Evaluating the Evidence