Experiences With Quality Improvement in Training: A National Survey of Urology Residents


      To determine the quality improvement (QI) experiences of urology residents in the United States.

      Materials and Methods

      An Institutional Review Board approved national survey of Urology residents was administered in June 2019. The survey instrument sought to understand QI curricular methods and experiences of urology residents as well as their knowledge of QI fundamentals.


      Of 465 invited residents, 159 (34%) responded. Respondents represented all 8 AUA sections. The majority of respondents self-identified as white (66%). Females made up 30% of respondents.
      Less than 1/3 of residents (32%) report use of online modules, whereas nearly 2/3 (63%) report lecture-based didactics. Fifteen percent of residents report no QI curriculum. While nearly 2/3 of residents report receiving training in QI principles (64%), far less report receiving training in how to apply QI methodology (44%). Only 29% of residents report being required to lead a QI project. No differences were seen by training level.
      Only 3 respondents had heard of the Institute of Medicine's aims for quality improvement. Respondents had better knowledge of process, balancing and outcome measures, but less than half of respondents (49%) were able to correctly identify all 3. Having any QI curriculum is associated with correctly answering knowledge questions (P = .03).


      From a resident perspective, urology residency programs currently provide QI curricula that are inadequate to ensure residents achieve Accreditation Council for Graduate Medical Education milestones. It is imperative the urology community as a whole address these curricular gaps to ensure graduates are prepared to deliver high-quality, cost-conscious care to their patients once in practice.
      Abbreviations: ACGME (Accreditation Council for Graduate Medical Education), ABU (American Board of Urology), ACS (American College of Surgeons), AUA (American Urological Association), HVC (high value care), PBLI (practice-based learning and improvement), PS (patient safety), PGY (post-graduate year), QI (quality improvement), QIPS (Quality Improvement and Patient Safety Committee, of the AUA), QITI (Quality In-Training Initiative), SBP (systems-based practice)
      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Urology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Kohn LT
        • Corrigan J
        • Donaldson MS
        To Err is Human Building A Safer Health System.
        National Academy Press, Washington, D.C.2000
        • Committee on Quality Health Care in America - Institute of Medicine
        Crossing the Quality Chasm: a New Health System for the 21st Century.
        National Academy Press, Washingtion D.C.2001
        • Agency for Healthcare Research and Quality
        Module 4. approaches to quality improvement.
        Practice Facilitation Handbook. 2013; (Accessed May 12, 2020)
        • Bertsch J
        QIPS overview.
        Practice Resources. 2018; (Accessed March 31 2020)
        • Accreditation Council for Graduate Medical Education
        ACGME outcome project: competency descriptions. 2002; (Accessed January 15, 2017)
        • Combes JR
        • Arespacochaga E.
        Physician competencies for a 21st century health care system.
        J Grad Med Educ. 2012; 4: 401-405
        • Butler JM
        • Anderson KA
        • Supiano MA
        • Weir CR
        It feels like a lot of extra work": resident attitudes about quality improvement and implications for an effective learning health care system.
        Acad Med. 2017; 92: 984-990
        • Hoffman RL
        • Medbery RL
        • Vandermeer TJ
        • Morris JB
        • Kelz RR
        The educational toolbox: kick start your educational program in quality improvement.
        J Surg Educ. 2015; 72: e111-e116
        • Kelz RR
        • Sellers MM
        • Reinke CE
        • Medbery RL
        • Morris J
        • Ko C
        Quality in-training initiative–a solution to the need for education in quality improvement: results from a survey of program directors.
        J Am Coll Surg. 2013; 217
        • Ziemba JB
        • Matlaga BR
        • Tessier CD
        Educational resources for resident training in quality improvement: a national survey of urology residency program directors.
        Urol Pract. 2018; 5: 398-404
        • Purnell SM
        • Wolf L
        • Millar MM
        • Smith BK
        A national survey of integrated vascular surgery residents' experiences with and attitudes about quality improvement during residency.
        J Surg Educ. 2020; 77: 158-165
        • Artino Jr., AR
        • La Rochelle JS
        • Dezee KJ
        • Gehlbach H
        Developing questionnaires for educational research: AMEE guide No. 87.
        Med Teach. 2014; 36: 463-474
        • Moen R.
        Foundation and History of the PDSA Cycle.
        The W. Edwards Deming Institute;, Ketchum, ID2009
      1. Institute for healthcare improvement. Determining what to measure.
        Adv Meas Improv. 2015; (Accessed May 12, 2020)
        • Phillips AW
        • Reddy S
        • Durning SJ
        Improving response rates and evaluating nonresponse bias in surveys: AMEE guide No. 102.
        Med Teach. 2016; 38: 217-228
        • Harris PA
        • Taylor R
        • Thielke R
        • Payne J
        • Gonzalez N
        • Conde JG
        Research electronic data capture (REDCap)–a metadata-driven methodology and workflow process for providing translational research informatics support.
        J Biomed Inform. 2009; 42: 377-381
        • Malhotra NR
        • Vavra AK
        • Glasgow RE
        • Smith BK
        Six steps to engage residents in quality improvement education.
        Resources Surg Educ. 2020; (Accessed April 1, 2020)
        • Miller GE.
        The assessment of clinical skills/competence/performance.
        Acad Med. 1990; 65: S63-S67
        • Greenberg DR
        • Sohlberg EM
        • Shkolyar E
        • Shah JB
        Critical appraisal of quality improvement publications in the urologic literature.
        Urol Pract. 2019;
        • Pallari E
        • Khadjesari Z
        • Green JSA
        • Sevdalis N
        Development and implementation of a national quality improvement skills curriculum for urology residents in the United Kingdom: a prospective multi-method, multi-center study.
        Am J Surg. 2019; 217: 233-243
      2. Accreditation council for graduate medical education. The urology milestone project. 2016. Accessed June 30, 2019.

        • Weiss KB
        • Wagner R
        • Nasca TJ
        Development, testing, and implementation of the ACGME clinical learning environment review (CLER) program.
        J Grad Med Educ. 2012; 4: 396-398
      3. Accreditation council for graduate medical education. ACGME Common Program Requirements (Residency). 2018. Accessed March 31 2020.

        • Wong BM
        • Goldman J
        • Goguen JM
        • et al.
        Faculty-resident "co-learning": a longitudinal exploration of an innovative model for faculty development in quality improvement.
        Acad Med. 2017; 92: 1151-1159
        • Medbery RL
        • Sellers MM
        • Ko CY
        • Kelz RR
        The unmet need for a national surgical quality improvement curriculum: a systematic review.
        J Surg Educ. 2014; 71: 613-631
      4. Blackwood S, Boodry CI, Colavita P, Cudd JS, Donovan J. Practical QI: the basics of quality improvement education. 2017. Accessed January 28, 2020.

        • Pallari E
        • Khadjesari Z
        • Biyani CS
        • et al.
        Pilot implementation and evaluation of a national quality improvement taught curriculum for urology residents: lessons from the United Kingdom.
        Am J Surg. 2020; 219: 269-277
      5. American Urological Association. Education Council. Linthicum, MD 2019.

        • Stern K
        Patient safety.
        AUA Core Curriculum. 2018; (Accessed March 30, 2020)
        • Lane-Fall MB
        • Davis JJ
        • Clapp JT
        • Myers JS
        • Riesenberg LA
        What every graduating resident needs to know about quality improvement and patient safety: a content analysis of 26 sets of ACGME milestones.
        Acad Med. 2018; 93: 904-910
        • Dowling RA.
        What 2019 MIPS changes mean for your practice.
        Urology Times. Business of Urology, 2019
        • Artino Jr., AR
        • Durning SJ
        • Sklar DP
        Guidelines for reporting survey-based research submitted to academic medicine.
        Acad Med. 2018; 93: 337-340
        • Bartel Sheehan K
        E-mail survey response rates: a review.
        J Comput Mediat Communi. 2006; 6
      6. American Urological Association. Urology residents in the United States and across the globe 2016 - 2018 AUA Census 2018.