Quality versus risk of bias assessment of palliative care trials: comparison of two tools

Sarina R Isenberg; Dio Kavalieratos; Ronald Chow; Lisa Le; Pete Wegier; Camilla Zimmermann

doi:10.1136/bmjspcare-2020-002539

Quality versus risk of bias assessment of palliative care trials: comparison of two tools

BMJ Support Palliat Care. 2020 Nov 18:bmjspcare-2020-002539. doi: 10.1136/bmjspcare-2020-002539. Online ahead of print.

Authors

Sarina R Isenberg^{1

2}, Dio Kavalieratos³, Ronald Chow⁴, Lisa Le⁵, Pete Wegier^{2

6}, Camilla Zimmermann^{4

7}

Affiliations

¹ Bruyère Research Institute, Ottawa, Ontario, Canada SIsenberg@bruyere.org.
² Department of Family and Community Medicine, University of Toronto, Toronto, Ontario, Canada.
³ Department of Family and Preventive Medicine, Emory University, Atlanta, Georgia, USA.
⁴ Division of Palliative Care and Department of Supportive Care, University Health Network, Toronto, Ontario, Canada.
⁵ Biostatistics, Princess Margaret Cancer Centre, Toronto, Ontario, Canada.
⁶ Humber River Hospital, Toronto, Ontario, Canada.
⁷ Division of Medical Oncology and Division of Palliative Medicine, Department of Medicine, University of Toronto, Toronto, Ontario, Canada.

PMID: 33208350
DOI: 10.1136/bmjspcare-2020-002539

Abstract

Background: Randomised controlled trials (RCTs) of palliative care interventions are challenging to conduct and evaluate. Tools used to judge the quality of RCTs do not account for the complexities of conducting research in seriously ill populations and may artificially downgrade confidence in palliative care research.

Objective: To compare assessments from the Palliative Care Trial Assessment Tool (PCTAT) and Cochrane Risk of Bias (RoB) tool.

Design: Reviewers assessed 43 RCTs using PCTAT and RoB. We compared assessments of each trial, assessed overall agreement (weighted kappa (K_w)) and examined (dis)agreement for comparable items. We assessed quality of life at 1-3 months among trials grouped according to RoB or PCTAT score (using meta-analysis) and whether RoB or quality improved over time (Cochran-Armitage trend test).

Results: Of 43 trials, those rated low RoB had a mean PCTAT score of 73 (SD 10); those rated high RoB had a mean PCTAT score of 56 (SD 14). Overall K_w was 0.33 (95% CI 0.19 to 0.42). Total agreement between comparable items was observed for 56% of trials (24/43) and total disagreement for 21% (8/43). The standardised mean difference in quality of life was statistically significant among RCTs with low RoB and high PCTAT, but not for those with medium/low PCTAT or high/unclear RoB. Quality of reporting improved over time, whereas RoB did not.

Conclusion: Although there was fair agreement between tools, areas of disagreement/non-comparability suggest the tools capture different aspects of bias/quality. A specific tool to evaluate quality of palliative care trials may be warranted.

Keywords: supportive care.