A Quality Indicator Framework for High-Risk Areas in Obstetrical Care

Published: October 2019

Authors: Dr. Lisa A. Calder, Cara L. Bowman, Qian Yang, Tunde Gondocz, Christina Young, Cathy Zhang, Anna MacIntyre, Renee Darling, Dr. J. Peter O’Neill, Dr. Charmaine Roye, Dr. Guylaine Lefebvre

Acknowledgments: The authors wish to thank Dr. Sharon Caughey for her contributions to the framework, and Ria De Gorter for her assistance in preparing this manuscript. We also thank Joanna Noble from the Healthcare Insurance Reciprocal of Canada for reviewing and commenting on the manuscript.

How to cite this document >>

Introduction

There has recently been an emphasis on the necessity of improving obstetrical care worldwide.^1-3 Obstetrical care is also an important focus for medical-legal risk reduction because, while the frequency of medical-legal cases is relatively rare, the severity of harm for patients and their families and subsequent costs to healthcare systems are high. It has long been purported that engaging in quality improvement efforts may also lead to a potential reduction in medical-legal risk.^4-6 We have recently published an analysis that demonstrates that reduction in patient safety indicators is associated with reduced medical-legal risk when examined regionally across Canada.⁷

Successful quality improvement efforts rely on appropriate measurement. While this is an intuitive principle, there has been a recent call to measure what matters.⁸ Not only is there currently an excess of measurements that do not matter, there is also a tension between what is measured for epidemiological research and what is necessary to measure the impact of quality improvement efforts. Too often, quality improvements are impeded by measurement of only clinical outcomes, for example maternal mortality, which, while important, is so rare in developed countries that detecting a signal of improvement at a local level is significantly challenging.

To advance the quality of care in obstetrics and, hence, decrease medical-legal risk, it is essential that appropriate quality measures are selected. In a joint report with several other Canadian organizations including Accreditation Canada, Salus Global and the Healthcare Insurance Reciprocal of Canada (HIROC), we identified that there are five high-risk areas of practice in obstetrics.⁹ These include: 1) Assisted vaginal delivery; 2) Induction and augmentation of labour; 3) Shoulder dystocia; 4) Collaborative care; and 5) Decision to delivery time for Caesarean section. We created a quality indicator framework to facilitate quality improvement efforts in obstetrical care for the five identified high-risk areas of obstetrics based on medical-legal data.

Methods

We used two data sources to set a foundation for this framework: a medical-legal case review and a systematic literature review.

The medical-legal case review was a retrospective analysis of 691 cases from 2010-14 involving obstetrical care but excluding midwifery care.¹⁰ The cases were sourced from two medical-legal organizations: the Canadian Medical Protective Association, a not-for-profit organization currently representing over 100,000 physicians and the Healthcare Insurance Reciprocal of Canada, the largest provider of medical liability insurance for Canadian healthcare organizations and their employees. The cases were coded using international, national, and internal frameworks, and a descriptive analysis was performed.¹¹ A summary of the results is presented in Table 2.

We performed a systematic literature review to identify quality improvement studies in obstetrics for the five high-risk areas. We searched Medline, CINAHL, Database of Health Technology Assessment, Cochrane databases from 2005-2016, as well as grey literature from websites and repositories of relevant organizations and conferences. Two reviewers independently screened titles, abstracts and full text articles. We found 6,193 citations and included 73 articles. The detailed methods are published elsewhere.¹² A summary of results is presented in Table 3. This review identified that there was a paucity of robust quality indicators for the areas of interest. Subsequently, we performed expert consultation with local obstetricians and nurses to determine challenges in measurement in obstetrics quality improvement.¹³

Based on quality improvement measurement principles, we developed criteria for robust quality metrics.¹⁴ (See Table 2.) This allowed us to evaluate pre-existing quality measures from the literature. We were interested in overall measures to inform quality improvement efforts. We used the Institute for Health Improvement’s 3 types of measures: 1) process, 2) outcome and 3) balancing.¹⁵ We chose not to focus on structural or patient experience measures in this framework.¹⁶ We refined our framework to distinguish measures from indicators. We defined measures according to the National Quality Forum as reference points against which other things can be measured, to bring into comparison against a standard. We defined indicators as a type of measure which gave an indication of a quality of care issue rather than a specific measurement.¹⁶ Thus, the quality indicator framework includes process measures and indicators, clinical outcome measures and indicators, and balancing measures and indicators. We defined a clinical outcome measure as a defined result of care provided.¹⁷

We selected relevant quality indicators and measures from the literature that mapped to the identified high-risk medical-legal areas, and developed new potential measures and indicators as necessary.^18-22 We also examined the national hospital discharge database and a provincial birth registry to which the hospitals were already submitting data and prioritized those measures that could be constructed with existing data. Subsequently, we created the quality indicator framework with expert consultation and iterative refinement. We evaluated the use of select indicators in a pilot project conducted with obstetrical teams at a local community hospital. The purpose of this focused evaluation was to determine the face validity, or a subjective assessment of whether the indicators measure what they were intended to measure. The pilot project was designed to determine the effectiveness of an educational intervention focused on implementing quality improvement projects in obstetrics. The choice of project was based on the identified high-risk areas of practice in the medical-legal case review. The team of 157 frontline healthcare providers were coached on measurement using the quality indicator framework described in this paper.

Results

We report 35 metrics in the quality indicator framework, 39 quality indicators, and 21 measures in Table 1. During the pilot project, we worked with a team that used measures for the decision to perform a Caesarean section. Our definition included the time leading up to the decision to perform the procedure as well as the time from the decision to the procedure occurring. Participants reported that the measures were pragmatic and easy to record. This was despite the fact that system issues contributing to delayed decisions are frequently not recorded in health records and, as such, can be harder to detect.

Limitations

We note several limitations with our quality indicator framework. It is possible we missed key articles with our systematic literature review. We also relied on expert consensus to determine our final list of measures, and that may have resulted in exclusion of potentially useful measures. The measures we propose are restricted to five areas of increased medical-legal risk and do not reflect all aspects of obstetrics care. Finally, not all measures have been validated, and some process of care and clinical measures may not be achievable at some sites depending on data collection resources and the robustness of health record documentation. These metrics are intended to be a starting point and will require local contextualization and, in many cases, further definition to allow for robust application.

Discussion

We created a framework that is pragmatic and measures what matters to improve obstetrical care. The framework focuses on five data-driven high-risk areas in obstetrics that we believe deserve quality improvement focus. We also believe that efforts in these areas will show improvements in the measures provided, and subsequently be associated with reductions in medical-legal risk. As such, this framework may be used as a tool to guide future quality improvement efforts and contribute to such improvements and reductions in medical-legal risk.

Previous studies have described quality indicators for specific areas of obstetrics, with the majority focused on maternal mortality secondary to postpartum hemorrhage, overall Caesarean section rate, and elective inductions.^23,24 We were unable to find in our systematic literature review any universally accepted standards or guidelines for measurement of quality improvement in obstetrics. The UK Royal College of Obstetricians and Gynaecologists developed 18 quality indicators for 7 main categories, of which there is overlap for induction of labour, use of instruments and somewhat for the decision to delivery time for urgent Caesarean section.¹ Likewise, Boulkedid et al conducted a French Delphi panel to determine quality indicators for antenatal and inpatient obstetrical care.²³ They identified 28 indicators with some relevance to induction of labour and assisted delivery.²³ We note that in this document, the vast majority of indicators are clinical, with few process of care measures, and no balancing measures. 

The Joint Commission, Centers for Medicare and Medicaid Services along with the National Quality Forum in the United States released a set of quality measures that for the hospital setting were all clinical measures. These included incidence of episiotomy, elective deliveries >= 37 weeks and < 39 weeks, Caesarean section rates, antenatal steroid prescription, and exclusive breast milk feeding. While these measures may be useful as a broad illustration of obstetrical quality of care at an institutional level, they are insufficient to improve any of the supporting clinical processes. Process of care measures and balancing measures are essential to appreciate the impact of local quality improvement efforts because they are often more timely, easier to measure, occur with sufficient frequency to determine local impact of interventions, and can uncover unintended consequences.

We also found one regional obstetric quality improvement evaluation study by Salus Global, a company that has created a specific quality improvement program to improve institutional obstetrical care in Canada.²⁵ In this article, the outcomes used were all clinical with the exception of length of stay at maternal hospitals. The levels of improvements were not clinically significant (with the exception of length of stay) but may have been more demonstrable if process of care outcomes were utilized. Overall, we noted in the literature a lack of consistency in quality measure use across studies for the five high-risk areas we identified, which limits the comparability of interventions.

Conclusion

Currently, there is a broad variety of published quality improvement efforts in obstetrics, but a lack of focus on key clinical areas, with the exception of post-partum hemorrhage. After analysis of the medical-legal cases in two national databases, we believe there is a pressing need to improve the quality of care in the five identified areas of risk. This quality indicator framework is intended to help address this need.

Implications

References

1. Carroll F, Knight H, Cromwell D, Gurol-Urganci I, van der Meulen J. Patterns of maternity care in English NHS trusts. London, United Kingdom: The Royal College of Obstetricians and Gynaecologists; March 2016.
2. Curtin SC, Gregory KD, Korst LM, Uddin SFG. Maternal morbidity for vaginal and cesarean deliveries, according to previous cesarean history: New data from the birth certificate, 2013. U.S. Department of Health and Human Services, Centers of Disease Control and Prevention; May 20 2015.
3. Rushwan H. The FIGO leadership in Obstetrics and Gynecology for Impact and Change (LOGIC) Initiative in Maternal Newborn Health. International Journal of Gynecology and Obstetrics. 2014;127(Supplement 1):S1-S2.
4. Sakala C, Yang YT, Corry MP. Maternity care and liability: most promising policy strategies for improvement. Womens Health Issues. 2013;23(1):e25-37.
5. Dalton GD, Samaropoulos XF, Dalton AC. Improvements in the safety of patient care can help end the medical malpractice crisis in the United States. Health Policy. 2008;86(2-3):153-162.
6. Pettker CM, Thung SF, Lipkind HS, et al. A comprehensive obstetric patient safety program reduces liability claims and payments. American Journal of Obstetrics & Gynecology. 2014;211(4):319-325.
7. Yang Q, Zhang C, Hines K, Calder LA. Improved hospital safety performance and reduced medical-legal risk: an ecological study using 2 Canadian databases. CMAJ Open. 2018;6(4):E561-566.
8. Berwick DM. Era 3 for Medicine and Health Care. JAMA. 2016;315(13):1329-1330.
9. Obstetrics Services in Canada: Advancing Quality and Strengthening Safety. Ottawa, Ontario: Accreditation Canada, the Healthcare Insurance Reciprocal of Canada, the Canadian Medical Protective Association, Salus Global Corporation;2016.
10. Delivery in focus: Strengthening obstetrical care in Canada. Ottawa, Ontario: Healthcare Insurance Reciprocal of Canada and the Canadian Medical Protective Association;2018.
11. McCleery A, Devenny K, Ogilby C, et al. Using medicallegal data to support safe medical care: A contributing factor coding framework. J Healthc Risk Manag. 2018.
12. A systematic review of quality improvement interventions in labour and delivery. PROSPERO 2016 CRD42016052118 2016. Available from: http://www.crd.york.ac.uk/PROSPERO/display_record.php?ID=CRD42016052118.
13. Lefebvre G, Calder L, Hines K, et al. Methods and lessons from a focused quality improvement program in obstetrics. Society of Obstetricians and Gynaecologists of Canada Annual Clinical and Scientific Conference 2017; June 20-23, 2017; Ottawa, Canada.
14. Science of Improvement: How to Improve. Institute for Healthcare Improvement. Available from: http://www.ihi.org/resources/Pages/HowtoImprove/ScienceofImprovementHowtoImprove.aspx.
15. Science of improvement: Establishing Measures. Institute for Healthcare Improvement.  Available from: http://www.ihi.org/resources/Pages/HowtoImprove/ScienceofImprovementEstablishingMeasures.aspx.
16. Campbell SM, Braspenning J, Hutchinson A, Marshall M. Research methods used in developing and applying quality indicators in primary care. Qual Saf Health Care. 2002;11:358-364.
17. The ABCs of Measurement. Washington, DC: National Quality Forum;2015.
18. Arora KS, Shields LE, Grobman WA, D’Alton ME, Lappen JR, Mercer BM. Triggers, Bundles, Protocols, and Checklists – What Every Maternal Care Provider Needs to Know. American Journal of Obstetrics & Gynecology. 2015.
19. Bonfill X, Roque M, Aller MB, et al. Development of quality of care indicators from systematic reviews: the case of hospital delivery. Implementation Science. 2013;8:42.
20. Measuring Patient Harm in Canadian Hospitals: Technical Report. Ottawa, ON: Canadian Institute for Health Information;2016.
21. How-to Guide: Prevent Obstetrical Adverse Events. Cambridge, MA: Institute for Healthcare Improvement;2012.
22. Vincent C, Burnett S, Carthey J. The measurement and monitoring of safety. London, United Kingdom: The Health Foundation; April 2013.
23. Boulkedid R, Sibony O, Goffinet F, Fauconnier A, Branger B, Alberti C. Quality indicators for continuous monitoring to improve maternal and infant health in maternity departments: a modified Delphi survey of an international multidisciplinary panel. PLoS One. 2013;8(4):e60663.
24. Melman S, Schoorel EC, de Boer K, et al. Development and Measurement of Guidelines-Based Quality Indicators of Caesarean Section Care in the Netherlands: A RAND-Modified Delphi Procedure and Retrospective Medical Chart Review. PLoS One. 2016;11(1):e0145771.
25. Thanh NX, Jacobs P, Wanke MI, Hense A, Sauve R. Outcomes of the introduction of the MOREOB continuing education program in Alberta. Journal of Obstetrics and Gynaecology Canada. 2010;32(8):749-755.
26. Calder L, Bowman C, De Gorter R, Wright E, Yang Q, Lefebvre G. A systematic literature review of quality improvement initiatives in obstetrics. Institute for Healthcare Improvement’s Scientific Symposium on Quality Improvement in Health Care , December 11 2017; Orlando, Florida.

Copyright information: The Canadian Medical Protective Association retain all intellectual property rights for the information presented here, unless otherwise specified.

How to cite this document: Calder LA, Bowman CL, Yang Q, Gondocz T, Young C, Zhang C, MacIntyre A, Darling R, O’Neill JP, Roye C, Lefebvre G. 2019. A quality indicator framework for high-risk areas in obstetrical care. Unpublished manuscript, Canadian Medical Protective Association.