Article Text

Download PDFPDF

Risk of 12-month mortality among hospital inpatients using the surprise question and SPICT criteria: a prospective study
  1. Alison M Mudge1,2,
  2. Carol Douglas3,
  3. Xanthe Sansome4,
  4. Michael Tresillian1,
  5. Stephen Murray1,
  6. Simon Finnigan4,5 and
  7. Cheryl Ruth Blaber3
  1. 1Department of Internal Medicine and Aged Care, Royal Brisbane and Women’s Hospital, Herston, Queensland, Australia
  2. 2Faculty of Medicine, University of Queensland, Brisbane, Queensland, Australia
  3. 3Palliative and Supportive Care Services, Royal Brisbane and Women’s Hospital, Herston, Queensland, Australia
  4. 4Internal Medicine Research Unit, Royal Brisbane and Women’s Hospital, Herston, Queensland, Australia
  5. 5UQ Centre for Clinical Research, University of Queensland, Brisbane, Queensland, Australia
  1. Correspondence to Dr Alison M Mudge, Department of Internal Medicine and Aged Care, Royal Brisbane and Women’s Hospital, Herston, QLD 4029, Australia; Alison.Mudge{at}health.qld.gov.au

Abstract

Objectives People with serious life-limiting disease benefit from advance care planning, but require active identification. This study applied the Gold Standards Framework Proactive Identification Guidance (GSF-PIG) to a general hospital population to describe high-risk patients and explore prognostic performance for 12-month mortality.

Methods Prospective cohort study conducted in a metropolitan teaching hospital in Australia. Hospital inpatients on a single day aged 18 years and older were eligible, excluding maternity and neonatal, mental health and day treatment patients. Data sources included medical record and structured questions for medical and nursing staff. High-risk was predefined as positive response to the surprise question (SQ) plus two or more SPICT indicators of general deterioration. Descriptive variables included demographics, frailty and functional measures, treating team, advance care planning documentation and hospital utilisation. Primary outcome for prognostic performance was 12-month mortality.

Results We identified 540 eligible inpatients on the study day and 513 had complete data (mean age 60, 54% male, 30% living alone, 19% elective admissions). Of these, 191 (37%) were high-risk; they were older, frailer, more dependent and had been in hospital longer than low-risk participants. Within 12 months, 92 participants (18%) died (72/191(38%) high-risk versus 20/322(6%) low-risk, P<0.001), providing sensitivity 78%, specificity 72%, positive predictive value 38% and negative predictive value 94%. SQ alone provided higher sensitivity, adding advanced disease indicators improved specificity.

Conclusions The GSF-PIG approach identified a large minority of hospital inpatients who might benefit from advance care planning. Future studies are needed to investigate the feasibility, cost and impact of screening in hospitals.

  • advance care planning
  • death
  • prognosis
  • decision support techniques
  • hospitals
View Full Text

Statistics from Altmetric.com

Introduction

Identifying people who may be nearing end of life and providing them with opportunities to discuss their treatment goals and preferences can improve the quality of death and avoid futile care.1 2 However, advance care planning is still not widely implemented, particularly in non-malignant disease.3 Although there are several reasons for poor uptake by both patients and health professionals,3 one important reason is uncertainty about how to identify those nearing end of life and whose responsibility this is.4–6 As most people will use hospital services in the final year of life,7 8 hospital staff may have an important role in initiating advance care planning discussions and simple tools to predict dying applicable in the general hospital setting would help to identify patients with whom to consider these discussions.

The Gold Standards Framework Proactive Identification Guidance (formerly Prognostic Indicator Guidance, GSF-PIG)9 recommends a sequential multistep approach combining clinician judgement such as the surprise question (SQ: ‘Would you be surprised if this patient died in the next 12 months?’)10 plus general and advanced disease indicators such as the Supportive and Palliative Care Indicators Tool (SPICT, http://www.spict.org.uk)11 to identify patients likely to be in their final year of life. Although there are previous studies using the advanced care indictors,12 13 there has never been validation of this multistep approach in a general hospital setting.

This study aimed to validate a 2-step prognostic tool consisting of the SQ plus two or more general indicators from the SPICT tool in predicting 12-month mortality in an adult general hospital population. We used the SPICT March 2014 version; this differs minimally from the 2017 version which has been updated by consensus processes. We chose the 2-step approach because the general indicators are relevant and applicable to diverse hospital patients and can be applied by a range of clinical staff without requiring specialist assessment or judgement. Specific aims were to estimate the proportions of high-risk patients using the 2-step tool, describe their characteristics and care and measure prognostic performance for 12-month mortality. We also explored the prognostic performance of a 1-step (SQ only) and 3-step (SQ plus two or more general indicators, plus one or more SPICT advanced disease indicators) approach, compared with the 2-step approach, to identify the strengths and weaknesses of different screening approaches in a hospital setting.

Methods

Design and setting

This observational cohort study was undertaken in a large metropolitan teaching hospital in Brisbane, Australia which provides a wide range of medical, surgical and oncology services, including a specialist palliative care consultation service. We used cross-sectional sampling of all adult inpatients on a single day, applying the 2-step prediction tool based on data available on the survey day and then obtaining data on mortality for all participants over the following 12 months. We also undertook complete chart review of all identified high-risk patients to describe their characteristics, advanced disease indicators, care processes and documented advance care planning during their admission.

Participants

Eligible participants were all adult admitted inpatients on the survey day, excluding maternity and neonatal ward, mental health unit and day treatment admissions (eg, chemotherapy, in-centre dialysis, geriatric day hospital). Participants were identified from ward lists generated at 09:00 am and assigned a unique identifier. No contact was made with participants, with data obtained from the medical record, nursing and medical staff providing care to the participants and the state-wide death register.

Access to medical record and mortality data was approved through the Public Health Act. The study was funded by Metro North Hospitals and Health Service Innovation funding, and study oversight was through the End of Life Care subcommittee of the hospital Safety and Quality Committee.

Outcomes and measures

The prognostic tool used in this study was guided by the GSF-PIG.9 We predefined patients as high-risk if the senior nurse and/or doctor caring for patient responded ‘no’ to the surprise question (‘SQ+’)14and if they met two or more SPICT general indicators of deteriorating health outlined in table 1, obtained by structured review of the medical record.

Table 1

General indicators of deteriorating health based on Supportive and Palliative Indicators Tool (March 2014 version) (content reproduced with permission of the developers)

The primary outcome was all-cause mortality within 12 months, obtained by matching participant identifiers against deaths recorded in the Queensland death register. Matching was performed by the Queensland Health Statistics Branch in August 2016, providing information about the date, place and cause of death.

We collected information about age, mode of admission (elective, via emergency department or inter-hospital transfer), usual residence (home alone, home with others, residential care or other), use of domiciliary services and treating team from the medical record. Functional status on the audit day was reported by the most senior registered nurse (eg, nurse unit manager or shift co-ordinator) familiar with each patient, using the Australia-modified Karnofsky performance scale (AKPS).15 This scale has been validated in hospital and community settings and is widely used in palliative care clinical assessments. Scores range from 10 to 100, with 100 representing highest function and each decile identified by a brief written description. Frailty was reported by the most senior available member of the medical team (consultant or registrar if available or junior house officer if no more senior staff available) caring for each participant, using the 9-item Clinical Frailty Score (CFS-9). This score provides nine possible levels of frailty, ranging from fully independent (lowest score) to bed bound and includes both a brief written description and a pictorial description.16 It is strongly predictive of 12-month mortality in patients hospitalised on geriatric wards17 and critical illness settings18 and can be applied by junior medical staff without training.19

Initial data were collected by a team of 18 trained surveyors who reviewed the medical record using a structured paper-based data collection tool, identified the most senior available nursing and medical staff and asked the SQ and obtained AKPS and CFS scores. Surveyors were experienced clinical staff (mainly ward nurses or safety and quality unit staff) who volunteered and received 1 hour of group training from the project manager on the morning of the study based on a written data abstraction manual. The project team were available in person and by telephone to answer questions throughout the day, to support inter-rater reliability. All clinical service directors and nurse unit managers were informed of the study in the weeks preceding the survey date, but questions were not revealed to preserve spontaneity of verbal responses.

Detailed medical record review was undertaken for high-risk participants following discharge from hospital by a clinically qualified project manager and two senior medical registrars. Data were collected on advanced disease SPICT criteria, diagnoses and comorbidities (defined according to Charlson comorbidity index), investigations, procedures and medications, length of stay and discharge destination. Objectively documented advance care planning processes including presence of a written advance healthcare directive (AHD), hospital acute resuscitation plan (ARP) document or palliative care service referral note were recorded. Data were collected using a structured abstraction tool including specification of data sources and definitions. Duplicate extraction of approximately 10% of cases and discussion and cross-checking with the lead investigator supported reliability.

Analysis

Participant characteristics were described using summary statistics and participants classified into high-risk or low-risk using the 2-step predictor tool. We examined agreement between medical and nursing staff on the SQ using the kappa statistic. Characteristics were compared between high-risk and low-risk groups using Χ2 testing for categorical variables and independent t-test or Mann-Whitney U test for continuous variables. The accuracy of the tool in predicting 12-month mortality was described using sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) and overall accuracy (percentage of participant correctly classified), with 95% CIs calculated using MedCalc software (https://www.medcalc.org/calc/diagnostic_test.php). We also described prognostic performance of a 1-step tool (SQ only) and a 3-step tool (SQ plus two or more general indicators plus any advanced disease indicators) and compared the performance of SQ reported by medical and by nursing staff.

Results

The study was conducted on 23 April 2015. Of 932 inpatients, 392 did not meet inclusion criteria, 16 did not have chart data accessible on the study day and 11 were from outside Queensland and could not be followed up, leaving 513 participants (figure 1). Participant characteristics are shown in table 2. Mean age was 60 years (range 19–102 years) and 467 (91%) lived in the community alone or with others, with 68 (13%) receiving home services. Most (314, 61%) were admitted via the emergency department and 20% were transferred from other hospitals. Participants had been in hospital for median 6 days at the time of survey (IQR 2–15 days).

Table 2

Participant characteristics

No medical officer was available to report SQ or CFS-9 for 42 (8%) participants, and no nurse response to SQ was available for one participant, but all participants had a SQ response from at least one informant. Nursing staff answered SQ+ for 216/512 (42.1%) participants, medical staff for 185/471 (39.3%) participants and one or both informants for 269/513 (52.4%). Agreement between medical and nursing staff was moderate (kappa=0.49).

Complete data were available for all SPICT general indicators except weight loss, missing on 12 (2%) participants; these participants were treated as negative on this item. The prevalence of individual SPICT items is shown in table 2, with 144 (28%) meeting no criteria, 116 (23%) meeting one and 253 (49%) meeting two or more.

Using our predefined 2-step criteria, 191 participants (37%) were considered at high risk of death in the next 12 months. Characteristics of high-risk and low-risk participants are compared in table 3. The high-risk group was older, more likely non-elective admission, more likely to have SPICT general indicators, more frail, poorer functional status and in hospital longer on the day of survey (median 10 vs 5 days). The highest proportions of high-risk patients were in general medicine and geriatric services (table 4), with substantial numbers of high-risk patients also identified in oncology, surgical specialty and medical specialty services. Nine high-risk participants (5%) had an AHD, 85 (44%) had an ARP document and 48 (25%) had a previous or new referral to the palliative care service. Forty (47%) of the participants with an ARP were documented as not having capacity to contribute at the time of planning. Ninety per cent (n=171) of high-risk participants had had at least one hospital admission in the previous 12 months and their median length of stay during the index admission was 24 days (IQR 11–50 days).

Table 3

Characteristics of high-risk and low-risk participants

Table 4

Number of participants and number and percentage identified as high-risk by the 2-step tool, by hospital unit type

In the following 12 months, 92 (18%) participants died. Twenty-seven (29%) deaths occurred at the same hospital, 36 (39%) in other hospitals and 29 (32%) were not in hospital. High-risk participants had a significantly greater risk of death than low-risk participants (72/191 (38%) vs 20/322 (6%), p<0.001). Sensitivity of the 2-step tool was 78%, specificity 72%, PPV 38% and NPV 94% (table 5).

Table 5

Prognostic performance of the three risk screening approaches for 12-month mortality

In comparison, using our predefined first-step criterion of SQ+ by either discipline, 83/269 identified participants died (sensitivity 90%, specificity 56%, PPV 31%, NPV 96%, overall accuracy 62%, table 5). Nurses and doctors showed similar performance. Assessing participants with both a nursing and medical respondent (n=470), 70/194 participants identified as SQ+ by nursing staff died within 12 months (sensitivity 80%, specificity 68%, PPV 36%, NPV 93% and overall accuracy 70%) and 71/185 identified as SQ+ by medical staff died (sensitivity 81%, specificity 70%, PPV 38%, NPV 94%,  overall accuracy 72%).

In 124 (65%) high-risk participants identified using the 2-step tool there were also advanced SPICT criteria, most commonly frailty, advanced cancer, advanced neurological disease or advanced cardiovascular disease. In this subgroup, mortality was significantly higher than those meeting general criteria alone (60/124 (48%) vs 12/67 (18%), p=0.001). The predictive performance of a 1-step, 2-step of 3-step approach for 1-year mortality is summarised in table 5 and figure 2. The 1-step approach (SQ+ alone) had the highest sensitivity and would only fail to identify 10% of those who died in the following 12 months, but would identify 52% of inpatients as potentially high risk, most (69%) of whom would not die within 12 months. The 3-step approach incorporating the advanced criteria would only identify about one quarter of patients as high risk, of whom almost half would die, but would fail to identify more than one third of patients who subsequently died.

Figure 2

The percentage of general hospital participants who would be identified as high-risk (‘flag+’) and low-risk (‘flag–’) by three different screening approaches and their predicted mortality at 12 months. The 1-step tool is the surprise question alone; the 2-step tool is the surprise question plus two or more indicators of general deterioration and the 3-step tool also includes disease-specific indicators of advanced disease.

Discussion

In this observational study in a large urban hospital, 37% of inpatients were assessed as high risk of dying in 12 months using a predefined 2-step combination of the SQ and two or more SPICT general indicators. These simple measures were feasible to collect from existing documentation and brief interview with bedside medical and nursing staff, making them potentially practical items for a hospital-based screening tool. High-risk patients were older, frailer and more functionally impaired and were most prevalent in general medical and geriatric services. Overall 18% of hospital inpatients died within 12 months, including 38% of those identified as high-risk.

We explored using 1-step or 3-step identification, compared with our 2-step tool. Using the SQ (by either medical or nursing staff) alone (1-step) would miss fewer patients who died in the next 12 months (higher sensitivity) but flag over half of inpatients as high risk, two-thirds of whom would still be alive at 12 months (figure 2). Although current evidence suggests more benefits than harm in discussing values and goals of care even in those patients who do not die within 12 months (‘false positives’),1 20 initiating advance care planning in this very large group would require substantial resources. Adding the advanced SPICT criteria to the 2-step tool would reduce the number of patients flagged at risk, with greater specificity and overall accuracy, but would miss about one-third of patients who died. This tool also required a detailed clinical assessment (adjudicated by senior medical registrars from chart information), which may make it impractical as a screening tool in a general hospital population.

The proposed predictive tool had moderate sensitivity (78%) and specificity (72%) for mortality, similar to the multi-item CARING tool in medical inpatients.21 We are not aware of any other study validating a 2-step combination of SQ plus general deterioration criteria. A large Spanish study22 reported a similar multistep tool (‘NEPCAL’) including the SQ, general and advanced indicators of disease and did not find significantly better performance of this complex tool compared with the SQ alone, where they identified a PPV of 31% and NPV of 92%, similar to our SQ findings. However, their study was conducted in an advanced chronic disease population in primary and hospital care, rather than a general hospital setting. They also found a level of agreement on SQ+ between medical and nursing staff similar to our study (kappa 0.48).23 Similarly, investigators comparing SQ alone or SQ plus several other clinical predictors in 207 older emergency department patients found sensitivity 77%, specificity 56%, PPV 32% and NPV 90% with both approaches, with a similar 12-month mortality to our study (21%); physicians identified 49% of patients at risk using SQ+ alone. In our hospital inpatient study, physicians’ predictive performance was very similar, although they identified somewhat fewer SQ+ (39%), which is comparable to another inpatient study.24 Studies using the SQ alone in various settings have recently been reviewed, with pooled sensitivity of 67% and specificity of 80% noted in one review10 and pooled accuracy of 74.8% in a second.14 In contrast to the latter review, which suggested greater accuracy of medical SQ prediction compared with nursing, we found very similar performance between disciplines in this inpatient setting, but using a combination of nursing and medical assessment increased sensitivity and decreased specificity.25 Although involving medical and nursing staff might make screening more complex, it aligns with a multidisciplinary approach to assessment and decision making.

Two point prevalence studies have used advanced SPICT criteria alone, assessed by specialist palliative care staff,13 26 to identify high risk patients. A New Zealand study26 identified 20% of participants as high risk, with 67% 12-month mortality in high-risk patients. They reported 63% sensitivity and 92% specificity, very similar to our 3-step findings (table 5). An Australian hospital study13 identified 27% at high risk, with 50% 12-month mortality in the high-risk group (similar to our findings) but did not report deaths in the low-risk group to allow calculation of performance measures.

The proportion of patients identified by the 1-step, 2-step or 3-step tool (24%–49% of inpatients) is likely to exceed the capacity of even the most well-resourced specialist palliative care or advance care planning service, highlighting the need to train all hospital staff in advance care planning, especially in general medicine and geriatrics. Importantly, high-risk participants had substantial contact with the treating hospital; 90% had had at least one admission in the previous 12 months and length of stay was protracted. Although this should provide opportunities for identifying, assessing and planning future care of these patients, less than half had an ARP completed during the study admission, one quarter had existing or new palliative care referral and only 5% had an AHD, suggesting that these opportunities are not always recognised and/or acted on. The high proportion of patients who required assisted decision making for their ARP illustrates the complexity of implementing advance care planning in high risk inpatients, who are elderly and frail with complex medical illnesses and high dependency on others. Similar hospital studies have shown ARP documented in 29%–35%, palliative care referral in 8%–32% and AHD use in 0%–1% of high-risk hospital patients.12 13 24 In our study, most deaths (68%) occurred in hospital, but more than half of these were in a different hospital, highlighting the need to hand over prognostic and planning information between care settings.

Our study adds to the literature identifying hospital inpatients who may benefit from discussion of goals of care and advance care planning. It is the first validation of the recommended GSF-PIG stepwise approach reported in hospital patients. The 2-step approach was simple for non-specialist staff to assess and presented a good trade-off between the acknowledged limited specificity of the SQ alone,10 and the lower sensitivity and greater complexity of advanced disease indicators. Strengths of the study include use of an existing consensus framework9 and tools; direct involvement of clinicians involved in care rather than specialist staff, which reflects clinical practice; small amounts of missing data and comparison of three related approaches within a consistent framework. We also recognise some limitations. The study is from a single site, although findings are similar to contemporary studies from other Australian and New Zealand hospitals suggesting findings are likely to be generalisable to similar health systems.13 26 The general criteria were collected by study assessors rather than staff, but assessors were clinical staff volunteers who required minimal training and items were simple to collect from the clinical record. We did not record the individual training level of medical and nursing staff completing the SQ, but our methods likely reflect the real-world availability of senior staff for making these assessments in clinical application. We did not directly involve consumers or their families. The cross-sectional recruitment means that assessments were performed on average 6 days into admission, when staff had more information for their clinical judgements than may be available at admission.

In summary, this study validates a method to identify general hospital patients likely to be in the last 12 months of life. Hospitals choosing the optimal approach to screening in practice will need to balance the costs, complexity and clinician acceptability of implementing screening approaches; the risk of failing to identify patients who might benefit from advance care planning; and the resources available for advance care planning. Future studies need to examine feasibility, acceptability, cost and impact of prospective use of validated screening approaches to improve advance care planning for hospital patients.

Acknowledgments

We acknowledge the support of all Divisions of the Royal Brisbane and Women’s Hospital and thank the staff who volunteered their time for initial data collection.

References

View Abstract

Footnotes

  • Contributors CD, CRB, AMM and XS conceived the study. AMM and MT drafted the article. XS, SM, MT and SF made substantial contributions to acquisition and analysis of data. All authors contributed to critical revision of the article and approved the version to be published.

  • Funding The study was funded by a SEED Innovation grant from the Metro North Hospitals and Health Service. Funders had no role in study conduct, data analysis, publication drafting or the decision to submit.

  • Competing interests None declared.

  • Patient consent Not required.

  • Ethics approval Ethical approval was received from the RBWH Human Research Ethics Committee.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data sharing statement Permissions for data sharing have not been granted.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.