Virtual Mentor. December 2004, Volume 6, Number 12.

Journal Discussion

• e-mail |
• print

A New Process for Writing Clinical Guidelines

The GRADE system aspires toward uniform standards for medical practice guidelines.

Brian Horvath, MPH

GRADE Working Group. Grading quality of evidence and strength of recommendations. BMJ. 2004;328:1490-1494.

Proposal of new methodologies for writing clinical practice guidelines is now a common occurrence. Many groups and organizations, including the American Medical Association and the Institute of Medicine, have proposed careful procedures for creating guidelines [1-4]. The underlying objective of practice guidelines is to assist clinicians and patients in making difficult health care decisions. Sometimes even seemingly simple decisions require the assistance of guidelines (see, for example, the 3 clinical cases in this issue of Virtual Mentor). The need for guidelines arose partly from concerns over unexpected variation in medical care [5-6], and from the desire to control health care costs [7].

As more groups have begun using practice guidelines, critiques of their methodologies have also evolved. Some studies have expressed concern that guidelines do not adhere well to their methodological standards [8-9]. Another concern is that guideline recommendations are not timely due to infrequent revision [10]. Finally, the focus of early guidelines on the effectiveness of interventions may have confused users that the strength of a recommendation is contingent upon the magnitude of a clinical response rather than upon the strength of supporting evidence that a clinical response does or does not exist [11].

Concerned about this current health care environment, not only in the United States but other countries as well, the GRADE working group has proposed a new process for writing practice guidelines [12]. According to their Web site [13], the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) Working Group is an international collaboration of people intent on creating a common, sensible approach to practice guidelines. Their previous work has examined the use of numbers and symbols to communicate grades of evidence from existing guidelines [14]. Now the GRADE group has moved beyond the communication of existing information to propose a comprehensive and systematic methodology for creating new practice guidelines.

After opening with a convincing explanation of the need for practice guidelines, the GRADE article deftly outlines the difficult choices that practitioners and patients must routinely make when faced with clinical decisions—choosing which potential outcomes to consider, what evidence is important to each outcome, and how to judge the quality of that evidence. The authors criticize the current environment where the presence of several competing guideline systems with differing methodologies causes confusion. The article then illustrates the many positive features of practice guidelines, such as the potential to prevent errors and disseminate clinical information.

To accomplish these goals, the authors propose an open system for making judgments about the quality of evidence and the strength of recommendations. For example, the GRADE system requires reviewers to rigorously examine 4 key and sequential elements when judging the quality of evidence: study design, study quality, consistency, and directness. Doing this, they maintain, should lead to a concise and explicit statement about the quality of evidence.

The next major consideration in the GRADE system is the strength of recommendations. The article highlights the trade-offs between benefits and harms inherent in making a recommendation, acknowledging that the outcomes and their seriousness may vary greatly depending on the patients’ clinical histories and social environment. Partly to address these issues, the GRADE clinical guidelines would be specific to patient groups and practice settings. While the GRADE system stresses that the foremost considerations are health benefits and harms, it also suggests the role of incremental health care costs. Unfortunately, like many other guideline systems, the GRADE proposal offers little in response to the vexing questions of when or how such costs should be considered. One of its aims, however, is to make the process more transparent and open to public scrutiny, and this is a laudable goal.

The GRADE system is a thoughtful approach to an acknowledged problem. Writing clinical guidelines challenges the authors to make simple and concise recommendations while still retaining enough complexity to be useful under a variety of individual circumstances. The GRADE system proposes a specific and lengthy process to help retain the needed complexity while ultimately producing a clear recommendation. The process clearly identifies all decision points, allowing others to more fully evaluate each guideline.

The authors make clear that their article is only a summary of their methodology; some important questions remain unanswered. For example, the article does not discuss the selection of panel members to write the guidelines. In a system with explicit methodologies for making recommendations, determining panel composition also requires clear procedures. The identity of panel members greatly influences how users view the guidelines. Furthermore, the article does not address the problem of time. An ideal guideline system would adjust recommendations to changes in technology and the practice environment over time, but no such process is identified in this article. Arguably, the process by which guidelines are updated is as significant as that by which they’re developed; out of date guidelines may, in some cases, be worse than no guidelines at all. A final concern is deciding which outcomes are most important to consider. Although the article provides admirable methodologies for evaluating given outcomes, choosing which outcomes are relevant is a value judgment that would benefit from more discussion.

These caveats are not flaws as much as areas in need of amplification. The real issues are not procedural limitations but rather the perceived need for a new system. The article proposes a strong process for writing clinical guidelines, but it is not obvious that this system is vastly superior to any of the systems already in use. The presence of multiple guideline systems is confusing. Having one standardized system for writing guidelines would be beneficial, and the GRADE system is admirable. Of more use than designing a new system, however, are articles that specifically compare and contrast existing systems with the ultimate goal of standardizing one approach over all others. After choosing one process for writing clinical guidelines, future discussions can shift from technical issues of methodology to the more clinically relevant difficulties inherent in medical decisions.

References
1. American Medical Association, Office of Quality Assurance. Attributes to Guide the Development and Evaluation of Practice Parameters. Chicago, Illinois: American Medical Association; 1990.
2. Woolf SH. Manual for Clinical Practice Guideline Development. Rockville, Maryland: Agency for Health Care Policy and Research; 1991.
3. Guyatt GH, Sackett DL, Sinclair JC, et al, for the Evidence Based Medicine Working Group. User’s guide to the medical literature IX: a method for grading health care recommendations. JAMA.1995;274:1800-1804.
4. Atkins D, Best D, Shapiro EN, eds. Third US Preventive Services Task Force: background, methods and first recommendations. Am J Prev Med. 2001;20:1-108.
5. Goulding MR. Inappropriate medication prescribing for elderly ambulatory care patients. Arch Intern Med. 2004;164:305-312.
6. Chassin MR, Brook RH, Park RE, et al. Variations in the use of medical and surgical services by the Medicare population. New Engl J Med. 1986;314:285-290.
7. Saarni SI, Gylling HA. Evidence based medicine guidelines: a solution to rationing or politics disguised as science? J Med Ethics. 2004;30:171-175.
8. Shaneyfelt TM, Mayo-Smith MF, Rothwangl J. Are guidelines following guidelines? The methodological quality of clinical practice guidelines in the peer-reviewed medical literature. JAMA. 1999;281:1900-1905.
9. Peloso PM, Carroll LJ, Cassidy JD, et al. Critical evaluation of the existing guidelines on mild traumatic brain injury. J Rehabil Med. 2004;43 Suppl:106-112.
10. Washington DL, Bernstein SJ, Kahan JP, et al. Reliability of clinical guideline development using mail-only versus in-person expert panels. Med Care. 2003;41:1374-1381.
11. Harbour R, Miller J. A new system for grading recommendations in evidence based guidelines. BMJ. 2001;323:334-336.
12. Atkins D, Best D, Briss PA, et al, and the GRADE working group. Grading quality of evidence and strength of recommendations. BMJ. 2004;328:1490-1494.
13. GRADE working group. Available at: http://www.gradeworkinggroup.org. Accessed September 1, 2004.
14. Schunemann HJ, Best D, Vist G, et al, for the GRADE working group. Letters, numbers, symbols and words: how to communicate grades of evidence and recommendations. CMAJ. 2003;169:677-680.

Brian Horvath is a recent MPH graduate from the University of North Carolina at Chapel Hill. He is a medical student at Duke University, where he is continuing research into Medicaid case management programs. He plans to pursue a career in dermatology.