Advertisement

The Applicant's Perspective on Urology Residency Interviews: A Qualitative Analysis

      Objective

      To better understand what urology applicants look for on interview day and what they care about in selecting a residency program through an analysis of anonymous online posts about the urology interview process.

      Methods

      We collected 3 years (2016-2018) of comments and posts from the Interview Impressions tab of the Urology Match Google Sheet for 133 urology residencies. Qualitative data analysis was performed using grounded theory methodology.

      Results

      We identified 6 categories of themes on (1) interview day structure, (2) diverse faculty, (3) program culture, (4) surgical training, (5) research, and (6) program benefits. These themes appeared in comments for 77%-86% of the residency programs except for research which was present for 44% of the programs. The efficiency and structure of interview day are very important. Applicants also care about young and diverse fellowship-trained faculty across a wide breadth of subspecialties. They believe they are able to discern the program culture and collegiality between residents and faculty. Applicants want a balance of surgical and clinical training with a focus on robotics and surgical autonomy. Not all applicants are interested in research but those that are appreciate a strong support system. Finally, additional program benefits and the positives and negatives of the program's location are frequently discussed.

      Conclusion

      Analysis of anonymous social media posts can help improve the interview process for applicants and programs alike. Programs can also identify areas of improvement for residency training. Our findings provide additional insight towards the ultimate goal of improving the match process.
      Applying to urology residency continues to be expensive, stressful, and competitive. For the last six years, the match rate for medical students has ranged from 68% to 91%.

      Association AU. Urology residency match statistics. Vol2019.

      In the 2019 cycle, there were 434 registered applicants for 339 positions across 136 registered programs.

      Association AU. Urology residency match statistics. Vol2019.

      Each applicant applied to an average of 71 programs and went on an average of 13 interviews.

      Association AU. Urology residency match statistics. Vol2019.

      The interviews are an integral aspect of the application process. Faculty and current residents are able to meet and interact with the applicants in person to determine who would fit well with the program's culture. Applicants are able to learn about each program's training curriculum and research opportunities and identify programs that would ultimately provide the best opportunities to pursue their career goals.
      The interview days are one of the only ways applicants can learn objective and subjective information about each program during this whole process. Nevertheless, these interview days are increasingly expensive and often inefficient.
      • Nikonow TN
      • Lyon TD
      • Jackman SV
      • Averch TD
      Survey of applicant experience and cost in the urology match: opportunities for reform.
      While others have described potential solutions to reduce the burden to applicants including using video interviews, coordinating regional interview dates, not requiring rotators to return for the formal interview, and restricting the total number of applications, there is little known about how applicants actually perceive the interviews and how the interview day itself can influence their perceptions on various training programs.
      • Shah SK
      • Arora S
      • Skipper B
      • Kalishman S
      • Timm TC
      • Smith AY
      Randomized evaluation of a web based interview process for urology resident selection.

      Hamilton BD.Society of Academic Urologists Annual Meeting 2019.

      • Koo K
      • Cone EB.
      When more is less: the burden of increasing urology residency applications.
      • Weissbart SJ
      • Hall SJ
      • Fultz BR
      • Stock JA
      The urology match as a prisoner's dilemma: a game theory perspective.
      In this study, we analyze an anonymous online forum of urology applicants to describe their experiences with the interview process at each residency program. Our objective is to identity characteristics of positive and negative interview experiences. We also aim to further understand what applicants find important in a residency program. By exploring how applicants perceive the interview experience, we hope to improve the application process for future medical students and programs.

      METHODS

      The Urology Match website (https://www.urologymatch.com/) was established in 2003 to help guide medical students interested in applying into urology.
      • Kutikov A
      • Morgan TM
      • Resnick MJ
      The impact of residency match information disseminated by a third-party website.
      In addition to educational articles and interviews, this website has also provided a discussion forum where applicants, trainees, and programs were able to disseminate information and answer questions. In the past several years, the majority of the discussion has migrated to a large Google Sheet that has been maintained and updated for each application cycle. (https://docs.google.com/spreadsheets/d/1qV5r88PEZ-bUIdLf2haGI2zp_xX0lbQHWbipdap17M4/edit#gid=902107043).
      This Urology Match Google Sheet contains multiple tabs with detailed qualitative and quantitative information about the programs and residents including the match results, applicant statistics, away rotation experiences, and program logistics including the call schedule and rotation sites. Another tab titled “Interview Impressions” specifically allows for applicants to describe their experiences at each program's interview day. Applicants are encouraged to report their impression of each program including the positives, negatives, and any other general comments. Applicants can input free text directly onto the Google Sheet. All the free text for each program is compiled in one of several cells, often without any formatting. Thus, their comments can range from single words to long paragraphs. In addition to comments about the programs, authors can have discussions and conversations with one another through the forum as well. All responses are anonymous and can be updated, deleted, or edited at any time. The forum's format did not allow for the analysis of discrete responses or comments. Multiple words, phrases, sentences from different writers were present for each program. Phrases from different authors were lumped into one long run-on phrase without any organization so it was impossible to differentiate amongst discrete responses. To avoid arbitrarily and likely inaccurately separating ideas, we chose not to attempt to quantify the individual comments themselves but instead we quantified how frequently the thematic categories appeared for the descriptions of each programs.
      We abstracted and reviewed all comments from the Interview Impression tabs for 3 application cycles from 2016 to 2018. Comments for all three years were compiled together for each program as the document was already organized by programs. Grounded theory methodology as described by Kathy Charmaz was used for analysis.
      • Charmaz K.
      • Leavy SNH-BP
      Grounded theory as an emergent method.
      In this form of qualitative analysis, implicit concepts and theories are generated from review of the data through inductive reasoning. By definition, grounded theory is hypothesis-generating through an analytical process of coding data, integrating theoretical categories, and writing analytic narratives. This technique is most appropriate for free-text narrative comments. Our analysis included several stages. First, 2 physicians (H.Z. and C.S.) independently hand-coded the comments line by line to identify concepts. They each generated and categorized preliminary themes based on groupings of similar codes and merged these preliminary themes. These themes were finally grouped into categories or emergent concepts.

      RESULTS

      We identified comments related to the interview day for a total of 133 urology residencies over 3 years. While the actual number of distinct phrases were difficult to parse out, there were a total of 33,351 words reviewed. No data on the demographics of the commenters or the number of commenters could be obtained due to the anonymous nature of the forum.
      Preliminary themes were separated into 6 different categories: (1) efficiency and structure of interview day, (2) diverse fellowship trained faculty, (3) program culture and collegiality, (4) surgical and clinical training, (5) research, and (6) program benefits and location. Of the 133 urology programs in the analysis, there were comments about efficiency and structure of interview day for 114 (86%) programs, diverse, fellowship trained faculty for 105 (79%) programs, culture and collegiality for 116 (87%) programs, surgical and clinical training for 102 (77%) programs, research for 58 (44%) programs, and location and benefits for 102 (79%) programs. Table 1 shows examples of phrases that were coded and felt to be representative comments for each thematic category.
      Table 1Representative quotes for each thematic category
      Efficiency and Structure of Interview Day
      • “Horrendous interview and I was dreading each room coming up next. Didn't get to know us at all on a personal level…”
      • “It was clear the faculty had read my application and asked specific questions from it.”
      • “Terribly organized interview day where you sit in the room and they just pick you randomly. Will be waiting for hours on end to be called for your next interview. Lasts from 8 AM to 4 PM”
      • “6 interviews with faculty/most double up, done by 1 PM. Preinterview drinks and apps.”
      • “Interviews well organized. fancy dinner night before and lots of great lunch and snacks out the whole day.”
      • “Post interview social, only a few residents showed up. Attendings came too, it was very awkward.”
      Diverse, Fellowship Trained Faculty
      • “Every specialty well represented.”
      • “New faculty joining every year, all subspecialities now covered.”
      • “Not a lot of fpmrs or infertility.”
      • “Young and accomplished attendings who are eager to teach residents and start projects.
      • “Expanding faculty – new recon, just hired a new once.”
      • “Chair has brought in many young faculty who are very easy to work with and love to operate and teach. Almost all faculty are fellowship trained.’’
      Program Culture and Collegiality
      • “There is a palpable political divide between attendings and even though the residents are great to get along with, many of them are unhappy.”
      • “Residents are close knit and hang out a lot outside work.
      • “One resident called the program ‘Traditional and hierarchical.’”
      • “The residents barely speak up or ask questions.”
      • “Program feels like a family.”
      Surgical and Clinical Training
      • “Most of the faculty are pretty hands off to the point where chiefs do most of the robotic cases.”
      • “Not a ton of clinical experience (they just operate non-stop which is a plus or minus.”
      • “Very low robotic volume.”
      • “PGY2s spend A LOT of time in clinic.”
      • “Very early resident involvement on robotics, especially at VA which has high robotic volume.”
      • “Fellows do not poach cases.”
      Research
      • “Research block can be more operative heavy if research not your thing.”
      • “Seems to be extremely weak on research. I brought up research and academics during my interview and the attendings did not seem interested at all.”
      • “Tons of research support (statisticians generate methods section and stats)”
      • “Some residents have >10 first author pubs by the time they graduate”
      • “Research very available but not forced upon you.”
      Program Benefits and Location
      • A lot of driving as they cover 4-5 different hospitals.”
      • “…is a beautiful town with a low cost of living… many fun places within driving distance, great school districts, difficult town to be single in.”
      • “Currently have a PA to bedside in robotic cases, and hiring another PA for the floors.”
      • “Very expensive parking as resident,i
      • “No loupes”

      Category 1 Themes: Efficiency and Structure of Interview Day

      Applicants commented on the structure, length, and quantity of the interviews. They preferred well organized and efficient interview days with minimal downtime in between each interview. Applicants also appreciated getting time to speak to the residents alone, either at the preinterview social event or during interview day, and it was important that the majority of the residents were available during this process. In general, applicants preferred personable interviews where the faculty were well prepared and had read the applications beforehand. Likewise, applicants also wanted to know which faculty they were interviewing with ahead of time so they could be prepared. Other comments were also related to the quality of the food and drinks that were provided throughout the day.

      Category 2 Themes: Diverse, Fellowship Trained Faculty

      Applicants were interested in programs had fellowship trained faculty across all urologic subspecialties. Any deficits or weaknesses were commonly highlighted. Comments were made about new faculty hires and how programs were expanding. Applicants appeared to prefer younger faculty who were energic and motivated although they also praised programs where the faculty were well established and “big names” in their respective fields. Gender and racial diversity amongst faculty was another important topic and the presence or lack of female faculty was discussed.

      Category 3 Themes: Program Culture and Collegiality

      The perceived culture of each program was based on how the applicants viewed the relationships amongst the residents and the relationships between the residents and faculty. There was a clear divide between 2 major types of program culture. “Traditional” programs had a noticeable hierarchy and formal training environment. In these settings, the applicants expressed that opinions of the residents were neglected, and they felt that some of these programs were malignant. Other programs displayed more collegiality between the faculty and residents and applicants described these programs as a “family” with a flat organization or structure.

      Category 4 Themes: Surgical and Clinical Training

      Surgical and clinical training are very important for applicants. Applicants expressed a desire to be proficient in both robotic and open cases. Surgical volume and autonomy were discussed although these descriptions were often subjective in nature. While operative independence was desired, too much autonomy could be seen as a negative as applicants preferred programs that offered a well-structured training curriculum and mentorship. Applicants were also wary of fellows that would detract from the educational experience of residents. Applicants seem to acknowledge that learning in clinic was important but also felt that some programs spent too much time in clinic and that it may be a reflection of low operative volume.

      Category 5 Themes: Research

      There were mixed reviews of the role of research in residency. On one hand, research experience in residency is not a priority for all applicants and an extra year for research was could be viewed as a negative. On the other hand, programs without any support or interest in research were also questioned. In general, applicants appreciated the availability of research opportunities but they did not want to feel forced into academic productivity. Programs with strong institutional support including full-time statisticians and readily accessible databases were highlighted.

      Category 6 Themes: Program Benefits and Location

      Applicants also shared the additional benefits that each program offered beyond just the training experience. Most of these benefits were financial in nature and included whether or not the applicants received reimbursements for board exams, loupes, hospital parking, medical licensing, and conference travel. Benefits also extended to the residents’ quality of life in and out of the hospital. For example, hospital efficiency, call schedule, and the presence of mid-level support were recognized. Applicants also discussed the positives and negatives of the surrounding geographic area of each program.

      DISCUSSION

      In this study, we identify 6 different categories of themes related to the interview process for urology applicants. These categories not only encompassed the logistical aspects of the actual interview day and social event but also how applicants perceived the surgical training, culture, research availability, and benefits of each program. These themes consistently appeared for the majority of the programs as 5 of the 6 categories were present for over 75% of the programs listed.
      There has been limited qualitative and quantitative data on this subject for urology residencies. Jacobs et al surveyed 221 urology applicants on their preferred interview format and found that the vast majority would prefer to meet just half or three quarters of the program's faculty in 5-7 interviews.
      • Jacobs JC
      • Guralnick ML
      • Sandlow JI
      • et al.
      Senior medical student opinions regarding the ideal urology interview day.
      Their ideal interview day length would only last half or three quarters of the day and applicants thought that spending time with the residents was the most important aspect of the day itself. This was consistent with our analysis. Comments on the forum expressed frustration at the excessive length of the interview day. At these programs, applicants spent most of this time sitting around and waiting for their next faculty interview. In their minds, this downtime and disorganization reflected poorly on the programs. In our experience, a shorter interview day also gives the applicants more flexibility with their travel plans as many of them head straight to the airport right after the interviews.
      Jacobs et al also surveyed the applicants on what they valued when determining the rank list. The 5 most important criteria in order were resident satisfaction, operative experience, strength of faculty, location, and interview day experience whereas rotations at a VA or children's hospital, reputation, residency duration, and dedicated research time were the five least important criteria.
      • Jacobs JC
      • Guralnick ML
      • Sandlow JI
      • et al.
      Senior medical student opinions regarding the ideal urology interview day.
      Similarly, in another survey of 346 urology applicants, Lebastchi et al also found that applicants viewed operative experience, interactions with current residents, and relationships between faculty and residents as the three most important criteria whereas prestige, graduate placement, faculty reputation, and research opportunities were the least important criteria.
      • Lebastchi AH
      • Khouri RK
      • McLaren ID
      • et al.
      The urology applicant: an analysis of contemporary urology residency candidates.
      These priorities were also well reflected in the forum we analyzed: operative volume and program culture mattered to applicants and were frequently discussed whereas program prestige was rarely mentioned. In addition, these surveys also confirm our qualitative findings that research opportunities are not highly prioritized for all applicants.
      Interviews are a critical aspect of the urology match. According to self-reported data on the Google sheet forum from the 2018 urology match, 53% of applicants matched at an institution where they rotated as a subintern. Thus, the remaining half will match at programs where they interview. It is remarkable that applicants will have to decide where they want to train over the next 5 or 6 years after spending just a day or even half day at some programs. Yet, based on the sheer amount of information on this forum, it seems they are able to get a strong grasp or at least form strong opinions of the culture and quality of training provided. It is also important for many programs to recognize that applicants appear to use the organization and quality of the interview day as a surrogate to the amount of effort they believe programs will put into their residents.
      As a whole, there are obviously significant areas for improvement for the application process. The reported average cost of attending each interview has increased from $330 in 2006 to $500 in 2014.
      • Nikonow TN
      • Lyon TD
      • Jackman SV
      • Averch TD
      Survey of applicant experience and cost in the urology match: opportunities for reform.
      ,
      • Kerfoot BP
      • Asher KP
      • McCullough DL
      Financial and educational costs of the residency interview process for urology applicants.
      As applicants continue to apply to more programs all across the country, these costs will likely continue to rise. In addition, due to the large number of programs and small window of interview dates available, scheduling these interviews can be very challenging and applicants frequently have to decide between 2 programs that share interview dates. Lastly, the activity and volume of this forum are a testament to the notion that programs need to be more transparent to potential applicants.
      • Zhao H
      • Lerman S
      • Freedman A
      Reforming the urology match application process: a role for the residency programs.
      As many of the comments on the forum about operative volume and research productivity are subjective, there is a clear need for programs to provide objective, quantified data about the training that residents receive.
      Strengths of this study include the anonymous nature of this forum. Applicants are able to speak their mind without any fear of repercussions. In addition, this forum is available to the entire applicant pool, not just those who only apply to a single institution. Our analysis was comprehensive and included 3 years worth of comments. The limitations of this study include an unclear sample size of applicants that constitute the forum and an unclear total number of discrete comments. Like all qualitative studies, coding the comments was subjective but the thematic categories that arose were agreed upon between 2 separate coders. In addition, because of the anonymity, comments in the forum can also be made by anyone with access including current residents and faculty so the true demographics of the users are unknown. There is also an inherent selection bias to the forum as people who comment may just represent a vocal minority of applicants. Finally, in this forum, comments can be edited or deleted at any time so the data we analyzed may have been altered at some point.

      CONCLUSION

      Analysis of anonymous social media posts can help improve the interview process for applicants and programs alike. To our knowledge, this is the first qualitative study to investigate how applicants perceive the interview day and interview social event. We find that applicants value interview day structure and efficiency, diverse fellowship trained faculty, program culture, surgical training, research availability, and program benefits. Programs can use this valuable feedback to improve their curriculum and training. Our findings provide additional insight towards the ultimate goal of improving the urology match process and resident training.

      References

      1. Association AU. Urology residency match statistics. Vol2019.

        • Nikonow TN
        • Lyon TD
        • Jackman SV
        • Averch TD
        Survey of applicant experience and cost in the urology match: opportunities for reform.
        J Urol. 2015; 194: 1063-1067https://doi.org/10.1016/j.juro.2015.04.074
        • Shah SK
        • Arora S
        • Skipper B
        • Kalishman S
        • Timm TC
        • Smith AY
        Randomized evaluation of a web based interview process for urology resident selection.
        J Urol. 2012; 187: 1380-1384
      2. Hamilton BD.Society of Academic Urologists Annual Meeting 2019.

        • Koo K
        • Cone EB.
        When more is less: the burden of increasing urology residency applications.
        J Urol. 2019; 202: 669-671https://doi.org/10.1097/JU.0000000000000335
        • Weissbart SJ
        • Hall SJ
        • Fultz BR
        • Stock JA
        The urology match as a prisoner's dilemma: a game theory perspective.
        Urology. 2013; 82: 791-797
        • Kutikov A
        • Morgan TM
        • Resnick MJ
        The impact of residency match information disseminated by a third-party website.
        J Surg Educ. 2009; 66: 212-215
      3. Urology Match 2020: Google sheet. URL:https://docs.google.com/spreadsheets/d/1qV5r88PEZ-bUIdLf2haGI2zp_xX0lbQHWbipdap17M4/edit#gid=1988989648 Accessed November 1, 2019.

        • Charmaz K.
        • Leavy SNH-BP
        Grounded theory as an emergent method.
        Handbook of Emergent Methods. US: The Guilford Press, New York, NY2008: 150-170
        • Jacobs JC
        • Guralnick ML
        • Sandlow JI
        • et al.
        Senior medical student opinions regarding the ideal urology interview day.
        J Surg Educ. 2014; 71: 878-882
        • Lebastchi AH
        • Khouri RK
        • McLaren ID
        • et al.
        The urology applicant: an analysis of contemporary urology residency candidates.
        Urology. 2018; 115: 51-58
        • Kerfoot BP
        • Asher KP
        • McCullough DL
        Financial and educational costs of the residency interview process for urology applicants.
        Urology. 2008; 71: 990-994
        • Zhao H
        • Lerman S
        • Freedman A
        Reforming the urology match application process: a role for the residency programs.
        J Urol. 2019; 203: 44-45https://doi.org/10.1097/JU.0000000000000464