Using data to monitor quality of care
Quality Indicators in Trauma
Project Coordinator: Professor Peter Cameron
Investigators: Cameron Willis
Background: Healthcare is increasingly complex, dynamic, fragmented and high risk. Quality indicators are one method by which the process of care delivery can be measured and evaluated. Quality indicators have been developed and are currently being used in evaluating quality of care across the entire continuum of care, from pre-hospital services, to the emergency department, and then incorporating the hospital admission, surgery, rehabilitation outcomes and long-term quality of life.This project investigated the relationship between Quality Indicators in trauma care and patient outcomes using data from trauma registries.
Aims: This project aimed to determine the validity of routinely used trauma process measures using high quality, routinely collected data sources.
Method: Data from the Victorian State Trauma Registry was used to investigate the relationship between individual indicators and patient level outcomes. Regression methods were used to investigate this relationship and focused on mortality, length of stay and use of the ICU. The second phase of the project investigated the use of composite trauma indicators that combined a number of measures into single scores at the hospital level. This process explored a range of composite approaches using data from VSTR as well as the Trauma Audit and Research Network from the UK.
Results: Very few measures were found to be associated with patient level outcomes. In some instances, indicators were counter intuitively associated with outcomes, whereby better performance was associated with poorer patient outcomes. The relationship between composite scores and outcomes was modelled at the level of the hospital, using poisson regression methods. Results from this investigation suggest that overall performance as measured by the composite scores demonstrates a stronger relationship with in-hospital mortality. Results from this investigation suggest that further exploration of composite measures may be a useful step forward in generating measures with improved validity.
Staff: Cameron Willis, Peter Cameron & Just Stoelwinder
Willis CD, Gabbe BJ, Cameron PA. Measuring quality in trauma care. Injury 2007;38:527-537;
Willis CD, Evans SM, Stoelwinder JU, Cameron PA. Measuring Quality. Australian Health Review 2007; 31:276-281;
Willis CD, Stoelwinder JU, Cameron PA. Interpreting process indicators in Trauma care: Construct validity versus confounding by indication. International Journal for Quality in Healthcare 2008; 20:331-338
Student: Cameron Willis
Contact Person: Cameron Willis - firstname.lastname@example.org
Register of Registries
Investigators:Evans SM, Bohensky M, Cameron PA, McNeil J.
Project Coordinator:Dr Sue Evans
Background: A key agenda for health systems worldwide is the improvement of quality of medical care and reduction of the frequency of adverse events. A high priority has been placed on measuring quality to demonstrate improvements and target efforts to areas where improvements are most needed. 1 Clinical registries provide one of the most accurate and clinically acceptable means of measuring quality, particularly when measuring outcomes of procedures or short term interventions (e.g. surgery, burns, and intensive care).
Aims:The aim of this project was therefore to determine whether existing Australian clinical registries have the potential to assess quality of care. Specifically we set out to identify: General aspects of registries including their coverage, length of operation and funding; what data are collected, how it is collected, stored and registry output; the quality of data within the registries according to recruitment, coding validation and reliability checks; governance structures within registries.
Method:Clinical registries were identified through peer consultation, review of the databases of Medline (1996- August 2006) and CINAHL (1996-August 2006) and web-based searches using key words Australia, national, Victoria, New South Wales, Northern Territory, Queensland, Western Australia, Tasmania, Australian Capital Territory, register, registry, and database.
Results:28 registries were identified. Attributes of the registries are outlined in the presentation found at this link: PRESENTATION
Staff: Dr Sue Evans
Contact Person: Dr Sue Evans - email@example.com
Quality indicators in the private health sector
Project Coordinator: Dr Sue Evans
Background: Quality indicators are used to monitor safety and quality of care because measuring and improving quality of care reduces errors, hospital misadventure and costs. Measurement using indicators forms a key part of the total quality framework. They may be used to determine compliance with standards and clinical requirements. Gaps in care and practice can be identified and addressed. By providing care with a commitment to safety and quality, organisations improve productivity and efficiency, and enhance patient satisfaction. The complexity of healthcare delivery requires a multifaceted approach to measuring quality and safety. Many indicators have been developed without reference to purpose, validity, feasibility or the broader context of the healthcare system and without a coordinated plan. This has resulted in a health system where indicators are prolific in some areas and non-existent in others. It has also resulted in inconsistent duplication of effort in collecting similar indicators as well as conflicting and incoherent measurement of the health system. Relative cost of commitment to safety and quality means that it is important to (a) better understand quality indicators and their purpose; (b) identify those currently used; and (c) narrow the field of quality indicators suitable for collection.
Aims:To identify a core set of clinical indicators that are suitable for collection in private hospitals to monitor the quality of care, and To determine the capacity of private hospitals to collect and contribute data to enable the collection of quality indicators to occur.
Method: A literature review was undertaken sourced from the published literature and from a search of the World Wide Web of organisations involved in health care quality. Questionnaires were developed to survey hospitals and hospital groups.
Results: Please see Report
Status:Project completed 2008
Evans SM, Cameron PA, Wilson S, Stoelwinder J, Hagger V, Copnell B, Sprivulis PC and McNeil JJ. Measuring Quality in Private Hospitals. Australian Centre for Health Research: Melbourne; August 2008.
Evans S, Lowinger J, Sprivulis P, Hagger V, Copnell B, Cameron P . Prioritising quality indicator development across the healthcare system: identifying what to measure. Internal Medicine Journal 2008; 9999(999A).
Copnell B, Hagger V, Wilson S, Evans S, Sprivulis P, Cameron P. Measuring the quality of hospital care: an inventory of indicators . Int Med J 2009; Accepted for publiation 6/3/09. Ref IMJ-0580-2008.
Staff:Dr Sue Evans, Dr Bev Copnell, Dr Virginia Hagger, Dr Sally Wilson
Contact Person:Dr Sue Evans - firstname.lastname@example.org
Fall related adverse events in Victorian Public Hospitals 1998-2008
Investigators:Phase 1 - A/Prof Caroline Brand, Dr Vijaya Sundararajan; Phase 2 - A/Prof Caroline Brand, Dr Vijaya Sundararajan, A/Prof Damien Jolley, Dr Anna Barker
Background: Recent studies in Australia, the US and Canada indicate that between 3.2-10.6% of hospitalisations are associated with an adverse event, with substantial impact on an individual’s health outcomes (mortality, disability and reduced quality of life) as well as organisation and community costs. Measuring and monitoring adverse events is an important driver for quality improvement, as well as contributing to organisational accountability and transparency.Falls are serious adverse events that can be associated with increased morbidity and mortality as well as increased length of in-hospital stay, functional decline and discharge to lower levels of independence.To date most of the literature pertaining to in-hospital falls has been based on incident reporting data. Whilst this data can provide important qualitative information about the nature and causation of falls, it does not provide robust epidemiological information that can form the basis of ongoing monitoring of system performance. Medical record review and prospective observational studies are alternative methods for providing epidemiological data but are time-consuming and costly to conduct.
Aims: The purpose of this study is to investigate, using routinely collected case mix data; the burden of fall events, variation in fall events between peer group organisations and the consequence of fall related adverse events in Victorian public hospitals between July 1st 1998 and June 30th 2008.
Method: A retrospective population cohort study using routinely collected casemix data collected within the Victorian Admitted Episodes dataset (VAED) and the Victorian linked dataset (VAED linked to data from the Registry of Births, Deaths and Marriages).
Status: Phase 1, which has included definition of terms, development of a data extraction algorithm, documentation of rates of falls and fall related fractures, description of risk factors for falls and fall related fractures is near completion. Phase 2, in which variation between peer group organisations will be investigated is in development.
Publications:Rates of in-hospital fall related injuries for people admitted to Victorian Public Hospitals between 1998 and 2007. A(poster presentation) National Forum on Safety and Quality in Health Care, Adelaide 2008.
Contact Person : Caroline Brand - Caroline.Brand@monash.edu
Pilot of a population-based prostate cancer clinical registry
Investigators:Millar J (CIA), Davis I, Bolton D, Giles G, Costello T, McNeil J, Evans S, Frauman A, Frydenberg M, Tai K
Project Coordinator: Sue Evans
Background:Cancer is the leading contributor to the overall burden of disease among Australians. Prostate cancer is the most commonly diagnosed cancer among Australian males and its incidence is growing. It is the most economically costly cancer. Prostate cancer mortality rates in Australia vary according to where men reside. Reasons for this variation are unknown but where variation has been identified in other countries quality of care issues have been identified as being at least partially responsible.
Aims:We propose a population-based prostate cancer registry. This registry will be based on the Victorian Cancer Registry but will move beyond it to collect further key information about the patient and his cancer, including the initial management and outcomes. Where clinical registries have been introduced at a state or national level, they have become one of the most clinically valued tools for quality improvement. They improve care, in part by arming clinicians with information about how their outcomes benchmark with standards and other clinical outcomes, both locally and (sometimes) internationally. Registries identify variation in patterns and processes of care and outcomes and factors that influence adverse outcomes. In addition to their ability to improve quality of care, they also provide a valuable tool to track how innovations in the biomedical sciences translate into longer term outcomes.
Method:A clinical registry will be piloted initially at four Victorian hospitals which together provide treatment for 22% of newly diagnosed men with prostate cancer in Victoria. It will then be progressively rolled out to a further two hospitals. In total over the course of this three year project we anticipate that the registry will capture 35% of all newly diagnosed cases of prostate cancer. The registry will collect a core minimum dataset of epidemiologically-sound data. To be clinically credible, the registry will systematically collect details of treatment provided, important risk factors, complications and mortality for all men diagnosed with prostate cancer. Quality checks of data will ensure completeness of recruitment of eligible population and accuracy of data. Where data are uniformly collected across sites in existing databases, they will be incorporated into the registry. Quality indicators will be collected as part of the registry’s core function and these will be benchmarked between institutions. The registry will be established in an independent research institution with a strong track record in developing clinical registries in both private and public hospitals at a national level. Overseeing the activities of the registry will be a Steering Committee comprising consumers, specialists from the disciplines of Medical Oncology, Radiation Oncology, Urological Oncology, Epidemiology, Health Service Policy and Economics. This group will work together to oversee activities of the registry. Governance structures will ensure that any identified outliers are actioned appropriately.
Results:The registry will begin collecting data from two sites by mid 2009. Ethics approval has been provided at one site, with two more pending. The minimum dataset has been developed and the Steering Committee is established. Data linkage options with the Cancer Council and hospital IT data systems are being canvassed.
Staff:Ms Julie Wood
Contact Person:Dr Sue Evans - email@example.com; Ms Julie Wood - firstname.lastname@example.org
Linking clinical and administrative data to evaluate intensive care outcomes
Background:Performance measurement based on administrative data are being used more regularly internationally. Despite the widespread use of such data, the scientific rigour of quality measurement based on administrative data has been questioned in the past. Inaccuracies and inadequate clinical detail can obscure the findings of quality measurement leading to either the under or over-representation of the actual quality of care provided in an organization.One proposal for enhancing quality reports based on administrative data is to use complementary data, such as those within existing clinical registries. Registries generally have a greater level of clinical detail, which can enhance the accuracy of risk-adjustment models to allow for more accurate benchmarking of outcomes. Additionally, their clinical focus may increase the acceptance of reports based on their data by other clinicians.
Aims: 1.To evaluate the utility and validity of administrative (Victorian Admitted Episodes data, including the Victorian Death Index) and clinical registry data (ANZICS Adult Patient database) for assessing ICU performance (mortality at different time end-points). 2.To establish guidelines for linking relevant datasets, to aid in quality measurement. 3.To develop different statistical models for monitoring (e.g. risk-adjustment models) and interpreting differences in organizational performance.
Method:Project 1: Data Linkage Reporting Guidelines Similar to the CONSORT guidelines for published results of randomised-controlled trials, we aim to develop reporting guidelines for data linkage studies. This tool may include a list of inclusion criteria and quantitative measures that will ensure all of the necessary information has been reported in a consistent way. Initial interviews will be undertaken with stakeholders to identify additional domains and items not found in the literature review. Delphi consensus methods will be used to assess expert agreement on the inclusion of domains and items for reporting data linkage studies (see Figure 1 - conceptual diagram). Experts will include epidemiologists with data linkage experience, technical experts (e.g. computer scientists), academic clinicians and statisticians.
Project 2:Data Linkage Validation A secondary data linkage will be conducted using the patient’s medical record number (URN) and date of birth. This linkage will be considered the “gold standard” due to the uniqueness of the linkage variables, especially when combined. The purpose of conducting this secondary data linkage in parallel is to validate the linkage between the ANZICS and VAED. Validating the ANZICS/DHS data linkage will enable the researchers to determine the quality of the data matching process and identify systematic errors that may have occurred, which could bias the results of the final analysis. The outcomes (mortality at different end-points, discharge locations) of patients will be compared for the ANZICS/DHS linkage and the “gold standard” linkage to determine if the different linkage methodologies will affect the measurement of these outcomes.
Project 3: Data Linkage It is proposed that three databases be linked to achieve the aims of this project: the ANZICS APD, the Victorian Death Index and the VAED. As the ANZICS and DHS data-sets do not contain shared patient identifiers, a matching technique will be developed based on event, demographic and disease-related variables to match the ANZICS and DHS data-sets. The combined data-set will be analysed as described below.
Statistical Analysis:The final ANZICS/DHS merged data-set will include data on a range of intensive care patient factors. Risk-adjustment models will be developed for administrative data only, clinical data only and a linked data-set. These three models will be evaluated in terms of their utility and validity for monitoring outcome measures at different end-points in time. The issue of inherent statistical variation will also be considered and methods for controlling this will be explored.
Results: Project 1, including the development of data linkage reporting guidelines is in its final stages. Data linkage is underway for Project 2. The thesis has been completed and passed in 2011.
Status: Data analysis
Staff: Megan Bohensky
Student/s: Megan Bohensky
Contact Person:Megan Bohensky - email@example.com
Trends in Use of Arthroscopy for Osteoarthritis of the Knee in Victorian Public and Private Sector Hospitals
Investigator: Caroline Brand
Background: Using routinely collected hospital data stored in the Victorian Admitted Episodes Dataset (VAED), we aim to research assess how many people have had knee arthroscopy procedures in Victoria each year since 1998, whether the patterns of use of this procedure are changing or are different in different hospitals and what adverse events are associated with the procedure. We will particularly focus on people with a diagnosis of osteoarthritis with, and without, an associated documented mechanical knee disorder. We will compare the use of knee arthroscopy to total joint replacement procedures to see if there are different patterns in the use of arthroscopy. We will also investigate the types of complications that occur in people with knee arthroscopy by looking at additional ‘in-hospital’ diagnoses that are coded in the VAED. This study offers a number of important benefits for the people of Victoria. Firstly, it will contribute to understanding how arthroscopy is used for people with OA knee. Secondly it may provide information about variation in use of the procedure that would suggest further investigation is needed to assess if there is undersupply or oversupply. Thirdly, we will gain some understanding of the incidence of adverse events associated with the procedure and, finally, it is likely to lead to improvements in data documentation that will improve further the quality of the VAED.
Cancer Clinical Indicator Project
Project Coordinator: Damien Jolley
Summary: The purpose of the project is to: (a) determine methods of analysis appropriate for the clinical indicators, giving consideration to the need for risk rating and identification of results that differ significantly from other areas; (b) determine appropriate presentation of data per ICS; and (c) develop a proposal regarding the ongoing development of the system to continue analysis of the clinical indicators, evaluate the use of the clinical indicators in reducing unwanted variation in practice and recommend appropriate timeframes for reviewing particular clinical indicators. The project involves three clinical indicators per tumour stream; the three tumour streams are colorectal, lung and breast cancer.
The project was completed in November 2009, and we have delivered several presentations to DoH and to the national Health Networks Conference in February. We are please to note that DoH has accepted all the recommendations of the final report and is in the process of implementing these. CPE-PS is involved in this process and will continue to advise DoH in ways to improve the measurement of quality of care in cancer treatment in Victoria, including the development of clinical registries.
Project outcomes: (to date) Comprehensive report to DoH about nine indicators in three tumour streams, 2006-2009;
Twelve recommendations to DoH for further development; DoH accepted all recommendations;
One PhD enrollment in CRE-PS to investigate validity of colon cancer/anastomotic leak indicator.
(in preparation) De-identified methodologic publication to describe how risk adjustment and data displays were generated.
Authors: Andrianopoulos N, Biggs J, Evans S, Jones A, Brand C, Jolley D and Cameron
Publications: In preparation
Contact: Damien Jolley Damien.Jolley@monash.edu
High Performing Health Care Organisations
Investigators: Caroline Brand, Ian Scott, Peter Cameron, Anna Barker
Summary: The CRE-PS project group, in consultation with Professor Michael Vitale, Director, Monash Asia-Pacific Centre for Science and Wealth Creation and Adjunct Professor, Melbourne Business School are performing a literature review on 'High Performing Health Services' for the Victorian Managed Insurance Authority (VMIA). The target audience for the review findings will be VMIA, Department of Health, health service board members, CEOs and senior managers. The purpose of the review is to support VMIA priority activities, and to positively influence health organisation capacity to provide high quality health care.
The literature review will:
• Provide an overview of the use of the term ‘high performing healthcare services’, where the definition of ‘healthcare services’ will include hospitals and hospital associated health services.
• Discuss ways in which high performance can be measured and the strengths and limitations of such measurement
• Identify the organisational characteristics that distinguish high and low performance in health care quality.
• Identify and describe interventions at the level of Board and/or CEO that can effectively enhance these characteristics and/or transform low performing organisations.
The following quality domains for health performance outcomes will be considered within the review; that healthcare is; safe, effective, :patient-centred, timely, efficient, and equitable
The review will involve a structured narrative analysis of peer reviewed quantitative and qualitative literature.
Pilot of a population-based lung cancer clinical registry
Investigators: Stirling R (CIA), Millar J (CIA), Evans S, McNeil J, Evans S, Irving L, Ashley D, Mitchell, D, Haydon A, MacManus M.
Project Coordinator: Peta McLaughlin
On average, lung cancer kills more than 1800 Victorians every year, making it the number one cancer killer in our community (1). Although rates of smoking continue to decline, the large number of former smokers means that death from lung cancer will continue at this rate for several decades. Despite advances in imaging, surgery, chemotherapy, radiotherapy and targeted therapies, the overall 5 year survival rate for lung cancer in Victoria remains unchanged at 11% (2).
One of the most basic tools needed to enhance outcomes for all lung cancer patients across Victoria is accurate, prospective, real-time data. Without such data, it will not be possible to meaningfully translate new biomarkers, screening modalities and therapies into the Victorian community. We propose to establish a Victorian Lung Cancer Registry as an essential platform for all lung cancer research in our state.
AIMS: In this project, we propose to establish the Victorian Lung Cancer Registry in a staged manner, with the ultimate goal of capturing all lung cancer patients across Victoria. A particular focus will be the assessment of quality of care outcomes in patients treated with chemotherapy and/or radiotherapy. The project will be developed in collaboration with the Victorian Biogrid, and will be designed to address the following aims:
(i) Develop and host the registry at the Monash University Department of Epidemiology (Alfred Hospital) using a consensus minimum dataset as recommended by the Registry Working Group, Management and Steering Committees.
(ii) Initiate a pilot data collection study in two metropolitan centres to confirm feasibility and accuracy.
(iii) Progressively expand the participating centres to encompass regional and metropolitan lung cancer patients.
METHOD: The registry will collect a core minimum dataset of epidemiologically-sound data including details of treatment provided, important risk factors, complications and mortality for all patients diagnosed with lung cancer in Victoria.
The Victorian Lung Cancer Registry Steering Committee was established in late 2010 and meets quarterly to oversee the activities of the registry. Members of the Steering Committee include; consumer representatives, specialists from the disciplines of Medical Oncology, Radiation Oncology, Respiratory Oncology, Palliative Oncology, Epidemiology and Pathology.
The registry will begin collecting data from two sites by mid 2011. Ethics applications for 2 of the 6 sites are currently underway. The minimum dataset is in the development phase and the data linkage options with the Cancer Council and hospital IT data systems are currently being explored
STATUS: Development phase
DEPM Staff: Dr Sue Evans (PI and Registry Custodian), Peta McLaughlin
Contact person: Ms Peta McLaughlin
Ph: 03 9003 0040
1. McDermid I (2005) Cancer incidence projections: Australia 2002 to 2011. Australian Institute of Health and Welfare (AIHW), Australasian Association of Cancer Registries (AACR) and the National Cancer Strategies Group (NCSG).
2. Victorian Cancer Council (2010) Cancer Council of Victoria Cancer Registry Statistics. In http://www.cancervic.org.au/about-our-research/registry-statistics).