You are viewing the site in preview mode

Skip to content

Advertisement

  • Research
  • Open Access

Radiation oncology resident training in patient safety and quality improvement: a national survey of residency program directors

Radiation Oncology201813:186

https://doi.org/10.1186/s13014-018-1128-5

  • Received: 16 May 2018
  • Accepted: 11 September 2018
  • Published:

Abstract

Background

Physicians and physicists are expected to contribute to patient safety and quality improvement (QI) in Radiation Oncology (RO), but prior studies suggest that training for this may be inadequate. RO and medical physics (MP) program directors (PDs) were surveyed to better understand the current patient safety/QI training in their residency programs.

Methods

PDs were surveyed via email in January 2017. Survey questions inquired about current training, curriculum elements, and barriers to development and/or improvement of safety and QI training.

Results

Eighty-nine RO PDs and 84 MP PDs were surveyed, and 21 RO PDs (28%) and 31 MP PDs (37%) responded. Both RO and MP PDs had favorable opinions of current safety and QI training, and used a range of resources for program development, especially safety and QI publications. Various curriculum elements were reported. Curriculum elements used by RO and MP PDs were similar, except RO were more likely than MP PDs to implement morbidity and mortality (M&M) conference (72% vs. 45%, p < 0.05). RO and MP PDs similarly cited various barriers, but RO PDs were more likely to cite lack of experience than MP PDs (40% vs. 16%, p < 0.05). PDs responded similarly independent of whether they reported using a departmental incident learning system (ILS) or not.

Conclusions

PDs view patient safety/QI as an important part of resident education. Most PDs agreed that residents are adequately exposed to patient safety/QI and prepared to meet the patient safety/QI expectations of clinical practice. This conflicts with other independent studies that indicate a majority of residents feel their patient safety/QI training is inadequate and lacks formal exposure to QI tools.

Keywords

  • Safety
  • Quality improvement
  • Education
  • Residency

Background

Patient safety/quality improvement (QI) is recognized as an essential part of radiation oncology (RO) practice. Physicians and physicists are expected to contribute to patient safety/QI [1], and will be responsible for ensuring the safety of increasingly sophisticated and complex treatments [2] delivered in an environment where quality indicators impact reimbursement [3]. The Accreditation Council for Graduate Medical Education (ACGME) has identified core learning objectives focused on patient safety/QI for inclusion in RO resident training [4]. It is therefore surprising that only a minority of RO and medical physics (MP) residents reported that their training in patient safety/QI concepts is adequate [5]. Most are unfamiliar with important concepts such as root cause analysis (RCA) [6] and are unsatisfied with their patient safety/QI training [7].

Unfortunately, the process and challenges associated with creating and improving resident safety and QI training remain elusive in RO. This study conducted a national survey of PDs of RO and MP residency programs to better understand the current training, curriculum elements, and barriers to development of patient safety/QI training.

Methods

PDs of RO residency programs accredited by the ACGME and therapeutic MP programs accredited by the Commission on Accreditation of Medical Physics Education Programs were identified. The survey was distributed via email using REDCap electronic data capture tools hosted at our institution [8]. Responses were anonymous. The study was deemed exempt from review by the local Institutional Review Board.

The survey included questions assessing demographics, state of current safety/QI training, resources used in development and/or improvement of training, curriculum elements used, and how PDs perceived and assessed the effectiveness of training. The survey was developed by the authors and pilot tested within an academic RO department prior to release (see Additional file 1: for complete survey).

Responses to questions that used the Likert scale were collapsed to a three-point scale for analysis (i.e. “strongly disagree”/“disagree” and “agree”/“strongly agree” combined). Previous research suggests that resident safety and QI training experience may be related to the presence of departmental RO incident learning system (ILS) [5], so responses were compared for participants that reported having a departmental lLS versus those that did not. The differences were tested for statistical significance in Microsoft Excel (v14.6.5) using chi-square tests for Likert scale questions and unpaired t-tests otherwise. P < 0.05 was considered significant.

Results

Participant characteristics

Eighty-nine RO PDs and 84 MP PDs were surveyed, and 21 RO PDs (28%) and 31 MP PDs (37%) responded. Three responses were excluded for failure to identify whether they were RO or MP PDs. RO programs ranged in size from 10 or less residents (N = 18), 11–16 residents (N = 6), and more than 16 residents (N = 1). MP programs had 2 or less (N = 16) or more than 2 residents (N = 14). Most participants (61%) reported that their department is using a RO specific ILS and 58% reported that faculty in their departments present or publish patient safety/QI research.

RO and MP PD’s opinions of patient safety/QI training

PD opinions of resident safety and QI training are shown in Fig. 1a-d. The vast majority of RO and MP PDs (88% vs. 94%) reported that patient safety and QI are an important part of resident education. MP PDs were more likely than RO PDs to agree that residents in their program were enthusiastic about safety training (77% vs. 48%, Fig. 1b). Otherwise, MP and RO PDs responses were similar.
Fig. 1
Fig. 1

a-d. Depicts PD opinions regarding residents and their patient safety and QI training. MP PDs (white) more commonly agreed with all statements than RO PDs (black), but only the difference in responses for about residents’ enthusiasm (b) reached significance (p = 0.03). RO = Radiation Oncology, MP = Medical Physics, PD = Program Director, QI = Quality Improvement

Resource utilization in creating and/or improving safety and QI training programs

The person primarily responsible for developing and implementing the patient safety/QI training experience was most commonly the residency PD (38%), but other faculty members (25%) or the medical director (21%) also lead these efforts. RO and MP PDs reported using a range of resources, including patient safety/QI publications (44% vs. 58%, p = 0.68) (Fig. 2). More RO than MP PDs reported using a morbidity and mortality (M&M) guide (48% vs. 13%, p < 0.005) and a QI project development tool (36% vs. 13%, p < 0.05), such as Plan-Do-Study-Act (PDSA). Otherwise, a range of resources were used similarly (Fig. 2).
Fig. 2
Fig. 2

Depicts the resources used by RO (black) and MP PDs (white) in creating their resident patient safety and QI training experience. RO PDs were significantly more likely than MP PDs to use an M&M facilitator guide and a project development platform (both p < 0.05). RO = Radiation Oncology, MP = Medical Physics, PD = Program Director, M&M = Morbidity and Mortality, Pubs. = Publications

Curricular elements and assessing effectiveness of patient safety/QI training programs

A range of curriculum elements were similarly implemented by RO and MP PDs in resident patient safety/QI training programs (Fig. 3a). These included a safety/QI activity (i.e. RCA), M&M conference, safety/QI project, and didactic activities (Fig. 3), which were all used by greater than 50% of PDs. The only significant differences between RO and MP PDs were that RO PDs reported using safety/QI project requirements for graduation (60% vs. 23%, p < 0.005) and M&M conference as required learning (72% vs. 45%, p < 0.05) more often than MP PDs. Four RO and 5 MP participants provided detailed information about a dedicated patient safety/QI rotation. The safety rotations range from 3 weeks to 6 months in duration, and residents had 10–100% of their time directed toward patient safety/QI education on the rotation.
Fig. 3
Fig. 3

a. Depicts the curricular elements utilized by RO (black) and MP (white) PDs in their resident patient safety and QI training programs. A clinical safety and QI activity, such as a root cause analysis, was commonly used by both RO and MP PDs. RO PDs reported using M&M conference and a safety/QI project requirement more often than MP PDs. b. Depicts how RO (black) and MP (white) PDs are assessing the effectiveness of their patient safety and QI training program. *p < 0.05. RO = Radiation Oncology, MP = Medical Physics

Both RO and MP PDs assessed the effectiveness of their programs using similar methods, most commonly resident feedback (RO 48% vs. MP 61%, p = 0.33) and faculty feedback (RO 32% vs. MP 42%, p = 0.45) (Fig. 3b). RO were more likely than MP PDs to report that they are not assessing the effectiveness of their safety training program (36% vs. 10%, p < 0.05).

Barriers to creating and/or improving patient safety/QI training programs

PDs reported a range of barriers faced in creating and improving a patient safety/QI training experience, but each was reported at a relatively low rate (Fig. 4). Responses were similar for RO and MP PDs, except RO PDs were more likely to report lack of expertise as a barrier to creating or improving their training programs (40% vs. 16%, p < 0.05). Ninety-six percent of participants reported that safety culture was not a barrier, and most reported RO and MP PDs reported that they had adequate access to patient safety/QI leaders in their institution (88% vs. 77%, p = 0.22).
Fig. 4
Fig. 4

Depicts barriers faced by RO (black) and MP (white) PDs in creating a resident patient safety and QI training experience. RO were more likely than MP PDs to report lack of PD expertise and lack of interest by residents as barriers (both p < 0.05). RO = Radiation Oncology, MP = Medical Physics

Seventeen participants offered free-text responses to the question “What advice would you have for program directors seeking to improve the patient safety/QI resident training experience in their program?” These are shown in the Additional file 1. Six described the importance of integrating residents into institutional patient safety/QI activities, and 6 noted the importance of institutional support and culture for improving training.

The influence of departmental use of incident learning systems

Additional file 1 Tables S1 and S2 depict the responses from PDs reporting use of a departmental ILS versus those who did not. PDs with a departmental ILS were less likely to implement a patient safety/QI educational event (e.g. safety research day) than those without a departmental ILS (14.7% vs. 36.8%, p = 0.05). Otherwise there were no significant differences between groups.

Discussion

The goals of this national survey were to understand the current status of patient safety/QI training experiences and identify challenges faced by PDs in developing patient safety/QI training programs. The survey identified important differences between RO and MP PDs and between PDs and prior surveys of residents. These findings may guide improvement of resident patient safety/QI training.

Differences between PD and resident opinions of patient safety/QI training

Most PDs agreed that residents are adequately exposed to patient safety/QI activities (64% RO vs. 84% MP) and are adequately prepared to meet these expectations of clinical practice (60% MD vs. 90% MP). This conflicts with a recent survey of RO and MP residents [5], which shows that less than 40% feel their patient safety/QI training is adequate and most lack formal exposure to key concepts, such as ILS reporting, RCA, and failure modes and effects analysis (FMEA) (Fig. 5a-b), although the survey questions are not identical. A prior survey found low RO and MP resident satisfaction with patient safety/QI training [7] and a recent study by the ACGME also found that most residents have a limited knowledge of RCA [6].
Fig. 5
Fig. 5

a. Depicts responses from faculty and residents regarding the adequacy of resident patient safety and QI training. Faculty = % agreeing with the statement: “I feel that residents who graduate from my program are adequately prepared to meet the patient safety and QI expectations of clinical practice”. Residents = % agreeing with the statement: “I feel that the formal teaching of principles in patient safety and quality management is adequate in my radiation oncology residency”. b. Depicts responses from faculty and residents regarding exposure to patient safety and QI activities during training. Faculty = % agreeing with the statement: “I feel that residents who graduate from my program are adequately prepared to meet the patient safety and QI expectations of clinical practice.”. Residents = % reporting “formal exposure” or “practical experience” with ILS, RCA, and FMEA. ILS = Incident learning system, RCA = root cause analysis, FMEA = failure modes and effects analysis. Note: resident responses are adapted from Spraker et al. [5]

Differences between RO and MP PD opinions of resident safety and QI training

MP PDs had more favorable opinions of resident safety and QI training than RO PDs, with differences in the question about resident enthusiasm reaching statistical significance. This might reflect differences in the clinical expectations and training goals between physicians and physicists. Physicists clinical duties have substantial overlap with safety and QI activities, and they accept primary responsibility of the clinical safety and QI program alongside physicians [911]. This may make practicing physicists especially well-equipped to teach certain aspects of patient safety/QI, such as RCA and FMEA. The finding that MP PDs were less likely than RO PDs to cite lack of expertise as a barrier to program development supports this hypothesis. This also may explain why MP residents are more likely than RO residents to be well-exposed and adequately trained in FMEA [5].

A recent international Delphi study recommended patient safety/QI competencies for RO residents that require proficiency in both procedural and non-technical areas, such as how to engage patients to champion the safety and quality of their treatment [12]. RO educators’ scope of practice including non-technical aspects of patient safety/QI compared to MP PDs may explain why they are more likely to use a M&M facilitator guide and implement M&M.

The current state and future directions of resident patient safety/QI training

Participants most frequently reported using publications and online modules or guides as resources to create and/or improve patient safety/QI training programs. These resources are becoming increasingly available. They include a detailed list of patient safety/QI competencies for RO residents [12], a framework for didactics [13] and web-based modules (i.treatsafely.org, IAEA e-Learning course (www.iaea.org/resources/rpop), Institute for Healthcare Improvement (ihi.org), and a safety culture education program [14]. Additionally, a patient safety education program including a dedicated safety and QI rotation designed for MP residents has been published [15].

We propose two recommendations for improvement of resident patient safety/QI training. First, we recommend a more formalized collaboration between MP and RO PDs or inclusion of patient safety/QI champions to close the gap of self-reported lack of expertise in these topics as a barrier to effective training. Institutional support through faculty development would further augment RO and MP collaboration. Second, PDs should consider integrating resident training into institutional patient safety/QI activities. This was recommended in the free text responses of six participants, is consistent with recommendations from the ACGME [16], and offers exposure to more faculty with expertise in patient safety and QI when this might be lacking in smaller RO programs. Building safety and QI training programs around the departmental ILS may be another effective approach as it enhances effective situated learning [15], and has been linked to an improved patient safety/QI training experience [5].

This study is limited by a response rate of only 28% and 37% for RO and MP PDs, respectively. This is similar to other surveys in this field where response rates range from 10 to 33% [5, 14, 1719]. The responses may not represent the opinions of all PDs. Indeed, over 60% of participants reported using a departmental ILS and half reported publishing or presenting patient safety/QI research. This group may have more experience and exposure to patient safety/QI than average, and this could have biased their responses toward more favorable opinions of patient safety/QI training and toward reporting less barriers and challenges than other PDs face. Another limitation is that the findings reaching statistical significance were not corrected for multiple comparisons. However, the goal of this paper was to identify the current state of patient safety/QI education from the perspective of PDs, not rigorously compare responses between groups.

Conclusions

This national survey of RO and MP PDs found that PDs view patient safety and QI as an important part of resident education. PDs use a range of available resources to implement diverse educational activities. Participants identified several important barriers to creating and improving patient safety/QI training, including lack of expertise. Most PDs agreed that residents are adequately exposed to patient safety/QI and prepared to meet the patient safety/QI expectations of clinical practice. This conflicts with other independent studies that indicate a majority of residents feel their patient safety/QI training is inadequate and lacks formal exposure to QI tools.

Abbreviations

ACGME: 

Accreditation Council for Graduate Medical Education

ASTRO: 

American Society for Radiation Oncology

FMEA: 

Failure Modes and Effects Analysis

ILS: 

Incident Learning System

MP: 

Medical Physics

PD: 

Program Director

QI: 

Quality Improvement

RCA: 

Root Cause Analysis

RO: 

Radiation Oncology

Declarations

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Authors’ contributions

All authors conceptualized the study and designed the survey. MS and KS sent the surveys and MS analyzed the responses. MS, EF, MN, and JZ were major contributors in writing the manuscript. All authors read and approved the final manuscript.

Authors’ information

This manuscript references a prior publication from our group, and some prior data is adapted into a new figure for the discussion (Fig. 5). Reference: Spraker, M. B., M. Nyflot, K. Hendrickson, E. Ford, G. Kane, and J. Zeng, 2017, A survey of residents’ experience with patient safety and quality improvement concepts in radiation oncology: Pract Radiat Oncol, v. 7, p. e253-e259.

Ethics approval and consent to participate

The study was deemed exempt from review by the University of Washington Institutional Review Board (Study #00000910).

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Authors’ Affiliations

(1)
Department of Radiation Oncology, Washington University in St. Louis, 4921 Parkview Place, CAM LL, CB 8224, St. Louis, MO 63110, USA
(2)
Department of Radiation Oncology, University of Washington, Seattle, WA, USA
(3)
Department of Radiation Oncology and Molecular Radiation Sciences, Johns Hopkins University, Baltimore, MD, USA
(4)
Department of Radiation Oncology, University of California San Francisco, San Francisco, CA, USA

References

  1. Marks LB, et al. The need for physician leadership in creating a culture of safety. Int J Radiat Oncol Biol Phys. 2011;79(5):1287–9.View ArticleGoogle Scholar
  2. Marks LB, et al. The impact of advanced technologies on treatment deviations in radiation treatment delivery. Int J Radiat Oncol Biol Phys. 2007;69(5):1579–86.View ArticleGoogle Scholar
  3. CMS Quality Strategy. [cited 2015 March 13]; Available from: https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/Value-Based-Programs/CMS-Quality-Strategy.html.
  4. ACGME Common Program Requirements. 2017 [cited 2017; Available from: https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/Value-Based-Programs/CMS-Quality-Strategy.html.
  5. Spraker MB, et al. A survey of residents' experience with patient safety and quality improvement concepts in radiation oncology. Pract Radiat Oncol. 2017;7(4):e253–9.View ArticleGoogle Scholar
  6. Wagner R, et al. Detailed findings from the CLER National Report of findings 2016. J Grad Med Educ. 2016;8(2 1):35–54.View ArticleGoogle Scholar
  7. Fogh S, Pawlicki T. A report on quality and safety education for radiation oncology residents. Int J Radiat Oncol Biol Phys. 2014;90(5):988–9.View ArticleGoogle Scholar
  8. Harris PA, et al. Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377–81.View ArticleGoogle Scholar
  9. Asp L, et al. AAPM report no. 38: statement on the role of a physicist in radiation oncology. New York: American Association of Physicists in Medicine; 1993.Google Scholar
  10. Safety is no accident: A framework for quality radiation oncology and care. 2012.Google Scholar
  11. Zietman A, Palta J, Steinberg M. Safety is no accident: a framework for quality radiation oncology and care. Arlington: American Society for Radiation Oncology; 2012.Google Scholar
  12. Adleman J, et al. Development of a quality and safety competency curriculum for radiation oncology residency: an international Delphi study. Int J Radiat Oncol Biol Phys. 2017;98(2):428–37.View ArticleGoogle Scholar
  13. Yeung A, Greenwalt J. A framework for quality improvement and patient safety education in radiation oncology residency programs. Pract Radiat Oncol. 2015;5(6):423–6.View ArticleGoogle Scholar
  14. Woodhouse KD, et al. The implementation and assessment of a quality and safety culture education program in a large radiation oncology department. Pract Radiat Oncol. 2016;6(4):e127–34.View ArticleGoogle Scholar
  15. Ford EC, et al. A patient safety education program in a medical physics residency. J Appl Clin Med Phys. 2017;18(6):268–74.View ArticleGoogle Scholar
  16. ACGME Program Requirements for Graduate Medical Education in Radiation Oncology. 2014 [cited 2016 May 7th]; Available from: https://www.acgme.org/Portals/0/PFAssets/ProgramRequirements/430_radiation_oncology_2017-07-01.pdf.
  17. Jani AB, et al. Results of the 2014 survey of the Association of Directors of radiation oncology programs (ADROP). Pract Radiat Oncol. 2015;5(6):e673–8.View ArticleGoogle Scholar
  18. Church JA, et al. National study to determine the comfort levels of radiation therapists and medical dosimetrists to report errors. Pract Radiat Oncol. 2013;3(4):e165–70.View ArticleGoogle Scholar
  19. Hoopes DJ, et al. Practice patterns for peer review in radiation oncology. Pract Radiat Oncol. 2015;5(1):32–8.View ArticleGoogle Scholar

Copyright

© The Author(s). 2018

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate. Please note that comments may be removed without notice if they are flagged by another user or do not comply with our community guidelines.

Advertisement