Return to Evaluating the Evidence

Supplement:

Evaluating the validity of a Diagnostic study

1. Was there an independent, blind comparison with a reference ("gold") standard?

Patients in the study should have undergone both the diagnostic test in question and the reference or "gold" standard.

The "gold" standard refers to the commonly accepted "proof" that they do or do not have the target disorder; the "gold" standard might be an autopsy or biopsy. The "gold" standard provides objective criteria (e.g., laboratory test not requiring interpretation) OR a current clinical standard (e.g., a venogram for deep venous thrombosis) for diagnosis. Sometimes there may not be a widely accepted "gold" standard; the author will then need to clearly justify their selection of the reference test.

Clinicians evaluating the tests should be blinded. The results of one test should not be known to those who are conducting or evaluating the other test.

2. Did the patient sample include an appropriate spectrum of patients to whom the diagnostic test will be applied in clinical practice?

For the information to be truly useful, the test should be applied to a broad spectrum of patients: those with mild and severe cases, as well as early and late cases, and with patients both treated and untreated for the target disease. The test should also be applied to patients with disorders that are commonly confused with the target disease.

3. Did the results of the test being evaluated influence the decision to perform the reference standard?

Researchers should conduct both tests regardless of the results of the test in question. Researchers should not be tempted to forego the "gold" standard test, if the outcome of the test in question is negative.

4. Were the methods for performing the test described in sufficient detail to permit replication?

The methodology for conducting the test should be presented in enough detail so that it can be conducted again within the appropriate setting. This may include dosage levels, patient preparations, timing, etc.

Key issues for Diagnostic Studies:

  • blinding

  • identified gold standard test

  • patient sample

  • each patient gets both tests

 

Are the results valid?

Likelihood ratios indicates the likelihood that a given test result would be expected in a patient with the target disorder compared to the likelihood that the same result would be expected in a patient without that disorder.

 

Disease
+

Disease
-

Test +

a

b

Test -

c

d

  • LR + = a / (a + c) divided by b / (b +d)

  • LR - = c / (a + c) divided by d / (b + d)

  • Sensitivity measures the proportion of patients with the disease who also test positive for the disease. [ a / (a + c)]

  • Specificity measures the proportion of patients without the disease who also test negative for the disease. [d / (b + d)]

  • A good test is both highly sensitive and highly specific.

Pretest Probabilities are estimated from published studies of prevalence, data from your practice setting, and your clinical intuition.

How much do LRs change disease likelihood?

LRs greater than 10 or less than 0.1

cause large changes

LRs 5 - 10 or 0.1 - 0.2

cause moderate changes

LRs 2 - 5 or 0.2 - 0.5

cause small changes

LRs less than 2 or greater than 0.5

cause tiny changes

LRs = 1.0

cause no change at all

Source: Jaeschke R ; Guyatt G ; Sackett DL. Users' guides to the medical literature. III. How to use an article about a diagnostic test. A. Are the results of the study valid? Evidence-Based Medicine Working Group. JAMA 1994 Feb 2;271(5):389-91.

Note: For criteria for other types of studies, see the following
supplements:  Therapy | Prognosis | Etiology/ Harm

Return to Evaluating the Evidence