Introduction to Evidence-Based Practice

Home

Welcome

ACQUIRE
Literature Search

Evaluating the validity of a Prognosis study

Are the results Valid?

1. Was the sample of patients representative?

The patients groups should be clearly defined and representative of the spectrum of disease found in most practices. Failure to clearly define the patients who entered the study increases the risk that the sample is unrepresentative. To help you decide about the appropriateness of the sample, look for a clear description of which patients were included and excluded from a study. The way the sample was selected should be clearly specified, along with the objective criteria used to diagnose the patients with the disorder."

2. Were the patients sufficiently homogeneous with respect to prognostic factors?

Prognostic factors are characteristics of a particular patient that can be used to more accurately predict that patient's disease course. These factors, which can be demographic (age, sex, race, etc.), disease specific (e.g., stage of a tumor or disease), or comorbid (other conditions existing in the patient at the same time), can also help predict good or bad outcomes.

In comparing the prognosis of the 2 study groups, researchers should consider whether or not the patient's clinical characteristics are similar. It may be that adjustments have to made based on prognostic factors to get a true picture of the clinical outcome. This may require clinical experience or knowledge of the underlying biology to determine if all relevant factors were considered.

3. Was the follow-up sufficiently complete?

Follow-up should be complete and all patients accounted for at the end of the study. Patients who are lost to follow-up may often suffer the adverse outcome of interest and therefore, if not accounted for, may bias the results of the study. Determining if the number of patients lost to follow up affects the validity depends on the proportion of patients lost and the proportion of patients suffering the adverse outcome.

Patients should be followed until they fully recover or one of the disease outcomes occurs. The follow-up should be long enough to develop a valid picture of the extent of the outcome of interest. Follow-up should include at least 80% of participants until the occurrence of a major study end point or to the end of the study.

4. Were objective and unbiased outcome criteria used?

Some outcomes are clearly defined, such as death or full recovery. In between, can exist a wide range of outcomes that may be less clearly defined. Investigators should establish specific criteria that define each possible outcome of the disease and use these same criteria during patient follow-up. Investigators making judgments about the clinical outcomes may have to be "blinded" to the patient characteristics and prognostic factors in order to eliminate possible bias in their observations.

Investigators making judgments about the clinical outcomes may have to be "blinded" to the patient characteristics and prognostic factors in order to eliminate possible bias in their observations.

Key issues for Prognosis Studies:

  • well-defined sample
  • similar prognosis
  • follow-up complete
  • objective and unbias outcome criteria

What are the results?

How likely are the outcomes over time?
    * What are the event rates at different points in time?
    * If event rates vary with time, are the results shown using a survival curve?

How precise are the estimates of likelihood?
    * What is the confident interval for the principle event rate?
    * How do confidence intervals change over time?

Issues of prognosis
Prognosis of a disease refers to its possible outcomes and the likelihood that each one will occur.

Prognostic Results are the numbers of events that occur over time, expressed in:

  • absolute terms: e.g. 5 year survival rate
  • relative terms: e.g. risk from prognostic factor
  • survival curves: cumulative events over time

A Prognostic Factor is a patient characteristic that can predict that patient's eventual outcome:

  • demographic: e.g. sex, age, race
  • disease-specific: e.g. tumor stage
  • comorbidity: other co-existing conditions

Articles that report prognostic factors often use two independent patient samples:

  • derivation sets ask - what factors might predict patient outcomes?
  • validation sets ask - do these prognostic factors predict patient outcomes accurately?

How can I apply the results to patient care?

Were the study patients and their management similar to those in your practice?
Were disease therapies applied equally across subgroups in the study?
Were disease therapies applied equality over time?

Was the follow-up sufficiently long?
Were patients followed long enough to detect outcomes of interest to your practice?

Can you use the results in the management of patients in your practice?
Does the impact of a prognostic factor on outcome events cross a therapeutic threshold?

Source: Guyatt, G. Rennie, D. Meade, MO, Cook, DJ.  Users' Guide to Medical Literature: A Manual for Evidence-Based Clinical Practice, 2nd Edition 2008.

Note: For criteria for other types of studies, see the following supplements:
 Therapy | Diagnosis | Etiology/ Harm | Systematic Review

Previous References Next

Revised July 2010