Advertisement

Low Methodological Quality of Systematic Reviews Published in the Urological Literature (2016-2018)

  • Author Footnotes
    # Maylynn Ding and Leah Soderberg contributed equally.
    Maylynn Ding
    Footnotes
    # Maylynn Ding and Leah Soderberg contributed equally.
    Affiliations
    McMaster University, School of Medicine, Hamilton, ON, Canada
    Search for articles by this author
  • Author Footnotes
    # Maylynn Ding and Leah Soderberg contributed equally.
    Leah Soderberg
    Footnotes
    # Maylynn Ding and Leah Soderberg contributed equally.
    Affiliations
    Minneapolis VA Medical Center, Urology Section, Minneapolis, MN

    University of Minnesota, Department of Urology, Minneapolis, MN
    Search for articles by this author
  • Jae Hung Jung
    Affiliations
    Yonsei University, Wonju College of Medicine, Department of Urology, Wonju, South Korea
    Search for articles by this author
  • Philipp Dahm
    Correspondence
    Address correspondence to: Philipp Dahm, M.D., M.H.Sc., Minneapolis VA Health Care System, Urology Section, 112D, One Veterans Drive, Minneapolis, MN 55417.
    Affiliations
    Minneapolis VA Medical Center, Urology Section, Minneapolis, MN

    University of Minnesota, Department of Urology, Minneapolis, MN
    Search for articles by this author
  • Author Footnotes
    # Maylynn Ding and Leah Soderberg contributed equally.
Published:January 15, 2020DOI:https://doi.org/10.1016/j.urology.2020.01.004

      Abstract

      Objective

      To investigate the methodological quality of systematic reviews (SRs) published in the urological literature.

      Methods

      PubMed was systematically searched for SRs related to questions of prevention and therapy published in 5 major urology journals (January, 2016 to December, 2018). Two reviewers followed a written a priori protocol to independently screen references in Rayyan and abstract data using a piloted form based on the 16 domains of AMSTAR-2. We performed preplanned statistical hypothesis testing by journal of publication in SPSS version 24.0.

      Results

      Our search identified 260 relevant references, 144 of which ultimately met inclusion criteria. The largest contributors by journal of publication were European Urology (53; 36.8%) followed by Urology (36; 25.0%), and BJU International (24; 16.6%). The most common clinical topics were oncology (64; 44.4%) and voiding dysfunction (32; 22.2%) followed by stones/endourology (14; 9.7%). Just over one-third (52; 36.2%) of reviews had a registered protocol. Nearly all studies (139; 96.5%) searched at least 2 databases. Less than one-third (46; 31.9%) also searched trial registries and one-fifth (30; 20.8%) consulted experts for additional trials. Few studies (14; 10.4%) provided a list of potentially relevant but excluded studies. Only 6 (4.2%) studies met all AMSTAR-2 critical domains as a prerequisite for high-quality reviews.

      Conclusion

      A large number of SRs are published in the urological literature each year, yet their quality is suboptimal. There is a need for educating authors, peer reviewers, and editors alike on established standards for high-quality SRs to ensure improvement in the future.
      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Urology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Dickersin K
        To reform U.S. health care, start with systematic reviews.
        Science. 2010; 329: 516-517
        • Kuehn BM
        IOM sets out “gold standard” practices for creating guidelines, systematic reviews.
        JAMA. 2011; 305: 1846-1848
        • Tseng TY
        • Dahm P
        • Poolman RW
        • Preminger GM
        • Canales BJ
        • Montori VM
        How to use a systematic literature review and meta-analysis.
        J Urol. 2008; 180: 1249-1256
        • Whiting P
        • Savovic J
        • Higgins JP
        • et al.
        ROBIS: a new tool to assess risk of bias in systematic reviews was developed.
        J Clin Epidemiol. 2016; 69: 225-234
        • Shea BJ
        • Bouter LM
        • Peterson J
        • et al.
        External validation of a measurement tool to assess systematic reviews (AMSTAR).
        PLoS One. 2007; 2: e1350
        • Shea BJ
        • Grimshaw JM
        • Wells GA
        • et al.
        Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews.
        BMC Med Res Methodol. 2007; 7: 10
        • Shea BJ
        • Hamel C
        • Wells GA
        • et al.
        AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews.
        J Clin Epidemiol. 2009; 62: 1013-1020
        • Shea BJ
        • Reeves BC
        • Wells G
        • et al.
        AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both.
        BMJ. 2017; 358: j4008
        • MacDonald SL
        • Canfield SE
        • Fesperman SF
        • Dahm P
        Assessment of the methodological quality of systematic reviews published in the urological literature from 1998 to 2008.
        J Urol. 2010; 184: 648-653
        • Han JL
        • Gandhi S
        • Bockoven CG
        • Narayan VM
        • Dahm P
        The landscape of systematic reviews in urology (1998 to 2015): an assessment of methodological quality.
        BJU Int. 2017; 119: 638-649
        • Han JL
        • Gandhi S
        • Bockoven CG
        • Narayan VM
        • Dahm P
        The landscape of systematic reviews in urology (1998 to 2015): an assessment of methodological quality.
        BJU Int. 2017; 119: 638-649
        • Corbyons K
        • Han J
        • Neuberger MM
        • Dahm P
        Methodological quality of systematic reviews published in the urological literature from 1998 to 2012.
        J Urol. 2015; 194: 1374-1379
        • Li M
        • Xing X
        • Yao L
        • et al.
        Acupuncture for treatment of anxiety, an overview of systematic reviews.
        Complement Ther Med. 2019; 43: 247-252
        • Pieper D
        • Puljak L
        • Gonzalez-Lorenzo M
        • Minozzi S
        Minor differences were found between AMSTAR 2 and ROBIS in the assessment of systematic reviews including both randomized and nonrandomized studies.
        J Clin Epidemiol. 2019; 108: 26-33
        • Lorenz RC
        • Matthias K
        • Pieper D
        • et al.
        A psychometric study found AMSTAR 2 to be a valid and moderately reliable appraisal tool.
        J Clin Epidemiol. 2019; 114: 133-140
        • Braga LH
        • Pemberton J
        • Demaria J
        • Lorenzo AJ
        Methodological concerns and quality appraisal of contemporary systematic reviews and meta-analyses in pediatric urology.
        J Urol. 2011; 186: 266-271
        • Booth A
        • Clarke M
        • Dooley G
        • et al.
        The nuts and bolts of PROSPERO: an international prospective register of systematic reviews.
        Syst Rev. 2012; 1: 2
        • Khaleel S
        • Sathianathen N
        • Balaji P
        • Dahm P
        Quality of urological systematic reviews registered in PROSPERO.
        BJU Int. 2019; 124: 195-196
        • Narayan VM
        • Cone EB
        • Smith D
        • Scales Jr., CD
        • Dahm P
        Improved reporting of randomized controlled trials in the urologic literature.
        Eur Urol. 2016; 70: 1044-1049
        • Singh JC
        • Dahm P.
        Evidence-based urology in practice: what are levels of evidence?.
        BJU Int. 2009; 103: 860-861
        • Glasziou P
        • Altman DG
        • Bossuyt P
        • et al.
        Reducing waste from incomplete or unusable reports of biomedical research.
        Lancet. 2014; 383: 267-276
        • Hollingsworth JM
        • Canales BK
        • Rogers MA
        • et al.
        Alpha blockers for treatment of ureteric stones: systematic review and meta-analysis.
        BMJ. 2016; 355: i6112
        • Niraula S
        • Le LW
        • Tannock IF
        Treatment of prostate cancer with intermittent versus continuous androgen deprivation: a systematic review of randomized trials.
        J Clin Oncol. 2013; 31: 2029-2036
        • McVary KT
        • Dahm P
        • Kohler TS
        • et al.
        Surgical management of lower urinary tract symptoms attributed to benign prostatic hyperplasia: aua guideline amendment 2019.
        J Urol. 2019; 202: 592-598
        • Jung JH
        • McCutcheon KA
        • Reddy B
        • et al.
        Prostatic urethral lift for the treatment of lower urinary tract symptoms in men with benign prostatic hyperplasia.
        Cochrane Database Syst Rev. 2019; 5: CD012832
        • Jung JH
        • Shin TY
        • McCutcheon KA
        • et al.
        Prostatic arterial embolization for the treatment of lower urinary tract symptoms in men with benign prostatic hyperplasia.
        Cochrane Database Syst Rev. 2017;
        • Hwang EC
        • Jung JH
        • Borofsky M
        • Kim MH
        • Dahm P
        Aquablation of the prostate for the treatment of lower urinary tract symptoms in men with benign prostatic hyperplasia.
        Cochrane Database Syst Rev. 2019; 2CD013143
        • Dahm P.
        Raising the bar for systematic reviews with Assessment of Multiple Systematic Reviews (AMSTAR).
        BJU Int. 2017; 119: 193
        • Jung JH
        • Dahm P.
        Reaching for the stars—rating the quality of systematic reviews with the Assessment of Multiple Systematic Reviews (AMSTAR) 2.
        BJU Int. 2018; 122: 717-718