HM Medical Clinic

Medlib.am

GRADE- an introduction Esther van Zuuren Leiden University Medical Centre

Grading of Recommendations Assessment, Development and Evaluation

• Background and rationale for revisiting guideline methodology • GRADE approach • Quality of the Evidence • Strength of Recommendations • GRADE working group

Why grading system? • Medical evidence, or the recommendations that are based on the evidence, can be of different quality • People draw conclusions about the quality of evidence and strength of recommendations • Insufficiënt attention to quality of evidence risks inappropriate guidelines • Different grading systems

Different grading systems Hierarchy of evidence The GRADE approach • Method of grading quality of evidence and strength of recommendations => evidence-based recommendations • Developed by the GRADE working group • Methodologists, guideline developers, clinicians • To overcome shortcomings of present grading The GRADE approach systematic reviews authors and guideline
1. Specific question 2. Identify all patient important outcomes 3. Judge relative importance of outcomes 4. Summarize all relevant evidence 5. Grade quality of evidence for each outcome The quality of the evidence reflects the extent of our confidence that the estimates of the effect are correct The GRADE approach 6. Overall quality of evidence across outcomes 7. Include judgments on values and preferences 8. Balance of desirable and undesirable effects 9. Balance of net benefits and costs 10. Grade strength of the recommendation 11. Formulate recommendation 12. Implement and evaluate Grade the evidence High
true effect lies close to the estimate of the
effect
Moderate
O: true effect is likely to be close to the estimate
of the effect, but there is a possibility that it is
substantially different
OO: true effect may be substantially different from the estimate of effect Very low
OOO: true effect is likely to be substantially different
from the estimate of effect
GRADEing the evidence • RCTs: start high • Observational studies: start low • 5 factors can reduce the quality • 3 factors can increase the quality • Across studies, by outcome • Explain your decisions on down/upgrading Factors that lower the quality 1. Study limitations 2. Inconsistency of results 3. Indirectness of evidence 5. Publication bias If factor is present: downgrade level of evidence by 1 (‘serious') or 2 levels (‘very serious') Study limitations (RCTs) Limitations in design and execution; •Lack of allocation concealment •Lack of blinding •Large losses to follow-up •Failure to adhere to an ITT analysis •Failure to report outcomes Example of risk of bias Down graded 2 levels1) Allocation concealment inadequate: the physicians had access to the computer-generated randomisation list.
2) Not blinded.
Inconsistency of results Unexplained heterogeneity of results across studies 1) Clinical heterogeneity may arise from differences in: different effect in sicker populations larger effect with higher doses diminishing effect with time 2) Methodological heterogeneity (differences in study design) Example of inconsistency Indirectness of evidence Evidence comes from different research question •Indirect comparison: drug A – drug B A – placebo and B – placebo •Population: oseltamivir prophylaxis for avian flu •Comparator: new drug – flexible doses of haloperidol fixed doses of haloperidol •Outcome: diabetic complications development of biochemical diabetes • Small sample size • Small number of events • Wide confidence intervals • Uncertainty about the magnitude of effect Systematic under- or overestimate of the effect due to selective publication of studies: •Investigators fail to report studies (typically those that show no effect) •Suspicion: evidence is limited to small number of trials, all showing benefit and funded by industry •Check: funnel plot, trial registers books Factors that can increase quality 1. Large magnitude of effect Large: RR >2 or RR <0.5 (1 level) Very large: RR >5 or RR <0.2 (2 levels) 2. Plausible biases underestimate the true effect or increase the effect if no effect was observed 3. Dose-gradient response Summary of Findings table Table 4 Summary of Findings of minoxidil versus placebo21-26,28 Minoxidil compared to Placebo for Female Pattern Hair Loss
Patient or population: Patients with Female Pattern Hair Loss
Settings: Multi centre hospital setting
Intervention: Minoxidil
Comparison: Placebo
Outcomes
Illustrative comparative risks* (95% CI)
Relative
Quality of the Comments
Participants evidence
Corresponding risk (studies)
Minoxidil
The proportion of participants with self-
Study population
rated clinically significant hair regrowth at
134 per 1000
250 per 1000
the end of the study
3 and 5 point scales 1 Follow-up: 24-32 weeks Moderate
Change in 'Quality of life' using any
validated and recognized generic or
disease-specific instrument4 - not reported
Adverse effects: safety and tolerability and Study population
any reported adverse events 1% minoxidil
moderate5
121 per 1000
136 per 1000
Laboratory values, blood pressure, participant- reported a/e at recall Follow-up: 24 weeks Moderate
Adverse effects: safety and tolerability and Study population
any reported adverse events 2% minoxidil
moderate5
27 per 1000
38 per 1000
Structured interview, physical examination and laboratory investigations Follow-up: 32-48 weeks Moderate
Adverse effects: safety and tolerability and Study population
any reported adverse events 5% minoxidil
moderate5
41 per 1000
144 per 1000
Structured interview, physical examination and laboratory investigations Follow-up: 48 weeks Moderate
Change in total hair count from baseline to The mean change in total hair count
The mean change in total hair count from the end of the study
from baseline to the end of the study baseline to the end of the study in the Hair counts in 1-1.5 cm2 area. Scale from: - ranged across control groups from intervention groups was -3.25 to 20.64
13.28 higher
Strength of recommendation "The extent to which one can be confident that adherence to a recommendation will do more good than harm" GRADE specifies only two categories of the strength of a recommendation 1. Strong recommendation: Do it or don't do it 2. Weak recommendation: Probably do it, or probably don't Strength of recommendation Determinants of the strength of •Quality of the evidence •Balance between desirable and undesirable effects •Values and preferences Strength of recommendations • Strong recommendations – strong methods – large precise effect – few down sides of therapy • Weak recommendations – weak methods– imprecise estimate– small effect– substantial down sides GRADE working group GRADE Working Group •2x a year meeting "GRADE working group •Publications BMJ (2008), JCE (2010-12) • GRADE is gaining acceptance as international • It provides criteria for evidence assessment across questions and outcomes • It provides criteria for moving from evidence to • It is simple, transparant and systematic – 4 categories of quality of evidence – 2 grades for strength of recommendation • Transparency in decision making and judgements is

Source: http://www.medlib.am/conference/Esther_van_Zuuren.pdf