Article Text
PDF
Abstract
Background Predicting how long a patient with far advanced cancer has to live is a significant part of hospice and palliative care. Various prognostic models have been developed, but have not been fully compared in South Korea.
Objectives We aimed to compare the accuracy of the Prognosis in Palliative Care Study (PiPS), Palliative Prognostic Index (PPI), Palliative Prognostic Score (PaP) and Objective Prognostic Score (OPS) for patients with far advanced cancer in a palliative care unit in South Korea.
Methods This prospective study included patients with far advanced cancer who were admitted to a single palliative care unit at the National University Hospital. Variables for calculating the prognostic models were recorded by a palliative care physician. The survival rate was estimated using the Kaplan-Meier method. The sensitivity, specificity, positive predictive value and negative predictive value of each model were calculated.
Results A total of 160 patients participated. There was a significant difference in survival rates across all groups, each categorised through the five prognostic models. The overall accuracy (OA) of the prognostic models ranged between 54.5% and 77.6%. The OA of clinicians’ predictions of survival ranged between 61.9% and 81.3%.
Conclusion The PiPS, PPI, PaP and OPS were successfully validated in a palliative care unit of South Korea. There was no difference in accuracy between the prognostic models, and OA tended to be lower than in previous studies.
- hospice care
- clinical assessment
- clinical decisions
- communication
- end of life care
- terminal care
Data availability statement
No data are available. All data will be deleted after the study is completed according to the direction of the Research Ethics Committee of Pusan National University Hospital.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Key messages
What was already known?
Through previous studies, various prognostic models for patients with terminal cancer have been developed and validated.
It is unclear whether various prognostic models are suitable for patients with terminal cancer in Korean Palliative Care Unit (PCU), which tend to have a short survival time.
What are the new findings?
Even among patients in the palliative ward in Korea with a short survival time, the survival period of each group of the prognostic model showed a significant difference.
There was no difference in accuracy between the prognostic models. And the accuracy of the prognostic models was lower than that of other studies, and the positive predictive value of the group predicting a long survival period for each model was particularly low.
Key messages
What is their significance?
Clinical
For patients in countries with a system that is usually referred to as a palliative care ward late, it may be helpful to predict short-term survival using an existing prognostic model.
However, the accuracy of predicting long-term survival in these countries remains unclear.
Research
This is the first study to validate and compare various prognostic models in South Korean patients with terminal cancer with short survival periods.
It is thought to be the first study to suggest that the short survival period of the study population can affect the accuracy of the prognostic model.
Introduction
Predicting how long a patient with far advanced cancer has to live is a significant part of palliative care for both clinicians and patients.1 Accurate prognosis prediction is needed to provide detailed information to patients and caregivers, to be able to give effective treatment directions and for caregivers to spend meaningful time with patients during their remaining time.2 3 Therefore, clinicians have implemented various methods to accurately predict prognosis; clinician predictions of survival (CPS) is the most widely used method. However, CPS is not too accurate, as clinicians tend to over-estimate survival.4 5 To find more accurate prediction methods, studies on variables related to patients’ prognosis were conducted. The studies reported that clinical performance status, symptoms, clinical signs and biological parameters are associated with prognosis.2 6 7 Based on these reports, various prognostic models, such as the Palliative Prognostic Index (PPI), Palliative Prognostic Score (PaP), Prognosis in Palliative Care Study (PiPS) and Objective Prognostic Score (OPS), have been developed in order to more efficiently predict survival and have since been validated and used in the clinical field.8–11
Patients with cancer in South Korea are in many cases referred to palliative wards belatedly compared with other countries. In previous studies done outside South Korea that validated the prognostic models, the median survival time of subjects ranged between 33 and 55 days.9 12 13 However, the median survival time of patients in palliative care units in South Korea was reported to range between 14 and 18 days in several studies.14–16 There have been several studies in South Korea that have validated a single prognostic model, but the number of studies that applied various models to validate and compare their accuracy remain insufficient.14 17–21 The purpose of this study is to compare the accuracy of various prognostic models for predicting survival time in patients with far advanced cancer admitted to palliative care units in South Korea.
Method
Participants
This study was prospectively conducted with patients who voluntarily participated. The participants consisted of patients with far advanced cancer aged ≥19 years who were admitted to a palliative care unit in a university hospital in South Korea. Informed consent was obtained from all patients or caregivers. In accordance with the Act on Decisions on Life-Sustaining Treatment for Patients in Hospice and Palliative Care or at the End of Life, patients with far advanced cancer are defined as patients suffering from far advanced cancer who are expected to die within a few months, as predicted by the doctor in charge and at least one medical specialist in the relevant field, due to there being no possibility of fundamental recovery and a gradual worsening of symptoms despite proactive treatment.22 The size of the sample was calculated through the PS power and sample size programme. Because the accuracy of each prognostic model applied to the palliative care unit in the previous study was between 69% and 77.4%, in this study, the accuracy was set to 70% and the clinical margin to 15%.13 The sample size was calculated for equality comparison of the ratio between the two groups by setting the power to 0.8 and type I error to 5%; calculations were made on at least 160 people. The subject registration period lasted from 26 April 2016 to 23 July 2020. Patients were evaluated within the first week of admission to the palliative care unit and all variables were recorded by the palliative care physician. All patients received follow-ups until their date of death or discharge. Data from laboratory tests were obtained based on blood tests conducted within 1 week after the date of admission.
Prognostic models
Information for calculating the PiPS model, PPI, OPS and PaP was collected. To calculate the PiPS-A model, we collected information on the following variables: diagnoses, bone metastasis, liver metastasis, distant metastasis, abbreviated mental test score, pulse rate, anorexia, dyspnoea, dysphasia, weight loss, Eastern Cooperative Oncology Group (ECOG) performance status and global health status. To calculate the PiPS-B model, we collected information on the following variables: diagnoses, bone metastasis, distant metastasis, abbreviated mental test score, pulse rate, anorexia, fatigue, ECOG performance status and global health status with laboratory data, including leucocyte count, platelet count, uraemia, alanine aminotransferase levels, alkaline phosphatase levels and albumin and C-reactive protein levels. The patients were then categorised into three groups according to prediction of prognosis: ‘Days’ (0–13 days), ‘Weeks’ (14–55 days) and ‘Months’ (>55 days).10 For the PPI, information on the following was collected: performance status in palliative care using the Palliative Performance Scale, oral intake, oedema, dyspnoea at rest and delirium.8 The patients were also categorised into three groups according to prediction of prognosis: : ‘<21 days’, ‘21–41 days’, and ‘≥42 days’. OPS consists of two symptoms (anorexia and resting dyspnoea), one performance score (ECOG) and four types of laboratory data (total leucocyte count, serum total bilirubin, serum creatinine and lactate dehydrogenase). OPS ranged from 0 to 8 points. Patients were classified into two groups: ‘<21 days’ and ‘≥21 days’.11 For PaP, the following information was collected: anorexia, dyspnoea, performance status using Karnofsky Performance Scale, total white blood cells, lymphopenia and the physician’s survival prediction, measured in weeks.9 The patients were categorised into three groups according to survival probability in a month period >70%, 30%–70% and <30%. Actual length of survival was defined as the period from the date of assessment to date of death. CPS were categorised into three groups in the same manner as PiPS, based on the clinician’s predicted value entered in PaP. Patients who were discharged or transferred to another hospital were regarded as censored data.
Statistical analysis
The overall survival rate and the survival rate specific for each group were estimated using the Kaplan-Meier method, and the survival rates for each group were also compared using the log-rank test; the sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) of each model were calculated as well. For PiPS-A, PiPS-B, PPI and CPS, the prognostic prediction results were categorised into three groups: short-term survival, medium-term survival and long-term survival. Sensitivity, specificity, PPV, NPV overall accuracy (OA) and absolute agreement were calculated for these individual groups. The OA was calculated by dividing the total sum of true positives and true negatives for each group of the prognostic model by the total number of subjects. The absolute agreement was calculated by dividing the number of true positives for the prognostic model by the total number of subjects. For PaP, the actual 30-day survival rate for each group was calculated independently, without calculating the previously mentioned values. This is because the outcome of the prognosis prediction of PaP was calculated as the probability of survival rather than survival time. Therefore, it is difficult to compare accuracy with other models through values such as OA or absolute agreement mentioned above. The rate of survival for each group was compared using the Cox proportional hazards regression model. The area under the receiver operating characteristic curve (AUROC) was calculated to determine the accuracy of the PPI, PaP and OPS to predict survival time within 21, 30 and 42 days. AUROC for PiPS-A and PiPS-B could not be calculated because only the estimated survival time can be showed through the web page, not the scores. The significance level of the statistical analysis was set to p<0.05, and the software IBM SPSS V.22.0.0 was used for all statistical analysis.
Results
General characteristics of the study subjects
A total of 160 patients participated, of which 126 died during the study and 34 were discharged or transferred to another hospital. Table 1 shows the general characteristics of the study subjects. Of the 160 patients, 98 were men (61.3%) and 62 (38.8%) women. The median age of patients was 65 years old, and among their conditions, six represented haematological cancer cases and the other 154 solid cancer cases. In patients with solid cancer, lung cancer was the most prevalent, with a total of 30 cases (18%), followed by colon cancer, with a total of 23 cases (14.4%), liver cancer, with 23 cases (13.8%) and stomach cancer, with a total of 22 cases (13.2%). There were 82 cases (51.3%) of distant metastasis, 21 cases (13.1%) of liver metastasis and 29 cases (18.1%) of bone metastasis. The median time of survival was 18 days (95% CI, 13.1 to 22.8); 67 patients (41.9%) died within 14 days and 21 patients (13.1%) survived more than 55 days.
General characteristics of patients (n=160)
Survival analysis for each group of prognostic models
For all models, survival time was analysed for each group using the Kaplan-Meier survival curve (figure 1, table 2). First, the median survival time for each predicted group calculated according to PiPS-A was 8 days (95% CI, 5 to 12 days) in the ‘Days’ group, 21 days (95% CI, 15 to 25 days) in the ‘Weeks’ group and 40 days (95% CI, 28 to 52 days) in the ‘Months’ group. Second, the median survival time for each predicted group calculated according to PiPS-B was 6 days (95% CI, 3 to 9 days) in the ‘Days’ group, 16 days (95% CI, 11 to 21 days) in the ‘Weeks’ group and 44 days (95% CI, 33 to 55 days) in the ‘Months’ group. Third, in PPI, the median survival time for each predicted group was 9 days (95% CI, 6 to 12 days) in the ‘<21 days’ group, 16 days (95% CI, 8 to 24 days) in the ‘21–41 days’ group and 42 days (95% CI, 30 to 50 days) in the ‘>42 days’ group. Fourth, in OPS, the median survival time for each predicted group was 7 days (95% CI, 3 to 11 days) in the ‘<21 days’ group and 28 days (95 % CI, 18 to 38 days) in the ‘≥21 days’ group. Fifth, in PaP, the median survival time for each predicted group was 8 days (95% CI, 5 to 11 days) in ‘<30%’ group, 25 days (95% CI, 17 to 33 days) in ‘30%–70%’ group and 64 days (95% CI, 29 to 100 days) in the ‘>70%’ group. Finally, according to the CPS, the median survival time for each group was 10 days (95% CI, 7 to 13 days) in the ‘Days’ group, 22 days (95% CI, 18 to 27 days) in the ‘Weeks’ group and 43 days (95% CI, 6 to 80 days) in the ‘Months’ group. Through the log-rank test, survival rates were compared between groups within all models; there were significant differences in all groups (log-rank test p<0.001 for all) (figure 1, table 2).
Actual survival time of the five prognostic tools and clinical prediction of survival
Survival experience of each group of patients identified by prognostic models and CPS of table 2 in the testing series (Kaplan-Meier analysis). All groups predicted for short-term survival categorised by each prognostic model showed a significantly shorter survival time than the group predicted for long-term survival. P<0.001 by log-rank test. CPS, Clinical Prediction of Survival; OPS, Objective Prognostic Score; PaP, Palliative Prognostic Score; PiPS, Prognosis in Palliative Care Study; PPI, Palliative Prognostic Index.
Accuracy of prognostic models
The sensitivity, specificity, PPV, NPV, OA and absolute agreement of the PiPS-A, PiPS-B, PPI and OPS models are displayed in table 3.
Accuracy of four prognostic tools and clinical prediction of survival
For predictions of probability for short survival (<14 days or <21 days), the PPV of all prognostic models was >70%. For predictions of probability for long survival (>55 days or ≥42 days), the PPV of all prognostic models ranged between 33.3% and 42.5%. The OA of the following groups, PiPS-A, PiPS-B, PPI and OPS, ranged between 61.2% and 75.4%, 54.5%–77.6%, 66.4%–74.6% and 67.6%, respectively. The absolute agreement of PiPS-A, PiPS-B, PPI and OPS was 51.5%, 47.8%, 54.5% and 67.6%, respectively; the OA of CPS ranged between 61.9% and 81.3% and the absolute agreement was 54.1%. The absolute agreement of OPS was higher than that of the other three models. This is probably explained by the fact that PiPS-A, PiPS-B and PPI categorise patients into three groups and OPS categorises patients into two groups. Arithmetically, absolute agreement will be lower than more prognostic categories there are. Therefore, it is inappropriate to compare absolute agreement of OPS directly with other three prognostic models. In PaP, in which the predicted prognosis is categorised as a 30-day survival rate rather than length of survival, the 30-day survival rate for each group was analysed instead of accuracy (table 4).
Survival probability of Palliative Prognostic Score
True 30-day survival rate for the ‘<30%’ group was 15.9%, 42.4% for the ‘30%–70%’ group and 69.4% for the ‘>70%’ group. The Cox proportional hazards regression model was conducted using the ‘>70%’ group as a reference, which resulted in the HR for death being 4.57 (95% CI, 2.51 to 8.33) for the ‘<30%’ group (p<0.001) and 2.06 (95% CI, 1.13 to 3.74) for the ‘30%–70%’ group (p=0.018). The AUROC values for PPI, PaP and OPS were 0.783, 0.745 and 0.745, respectively, for 21 days survival; were 0.761, 0.765 and 0.724, respectively, for 30 days survival; were 0.808, 0.833 and 0.782, respectively, for 42 days survival.
Compared with previous studies, the prognostic models tended to be less accurate, especially when predicting long survival periods. Table 5 presents a comparison with previous studies conducted to develop or validate the prognosis prediction model.
Positive predictive value and median survival of the studies
Discussion
This is a prospective study in South Korea to investigate the accuracy of various prognostic models for patients with far advanced cancer. The most relevant finding of this study is that there was a significant difference in survival rates across all groups, each categorised through the five prognostic models, which means that these models can be applied to predict the prognosis in patients with far advanced cancer in South Korea. On the other hand, the predictive models adequately predicted survival time, but were no more accurate as CPS.
Multiple studies have suggested that CPS accuracy ranges between 20% and 60%, and physicians tend to overestimate length of survival among patients with far advanced cancer.4 23 Prognostic models were created to aid physicians in predicting prognosis in patients with far advanced cancer,7 and Morita et al revealed that prognostic models can contribute to physicians’ ability to predict survival of terminally ill patients with cancer.24 However, in this study, the accuracy of the predictions of various prognostic models and that of CPS was similar. Taking this into consideration, the accuracy of the prognostic models in this study was lower compared with previous studies.
In this study, the OA of each group of PiPS-A ranged between 61.2% and 75.4% and that of PiPS-B ranged 54.5% and 77.6%. Based on the tables presented in previous study that developed PiPS-A and PiPS-B, we calculated the OA for each group.10 The OA of each group of PiPS-A ranged between 63.6% and 83.6%, and in PiPS-B, it ranged between 63.9% and 85.8%. In the validation study, the OA of each group of PiPS-A ranged between 73.7% and 80.7%, and in PiPS-B, it ranged between 77.4% and 81.1%.13 In the prognostic models, the validation study OA of each group of PPI ranged between 66.7% and 79%.12 In the study that developed the model, the OA of OPS was 75%.11 Compared with previous studies, the OA of each prognostic model presented in this study were lower.
The present study shows that the PPV of each group predicting a longer survival period (>55 days or ≥42 days) is much lower than that of the shorter survival period groups (<14 days or <21 days) in all prognostic models (table 3). The median survival time of the subjects in this study was 18 days; the proportion of the long survival group (>55 days—15.7%, ≥42 days—20.9%) was much lower than that of the short survival group (<14 days—50%, <21 days—61.2%). It can be assumed that the difference in these proportions influenced the difference in the PPV between groups.25 Compared with the present study, the subjects of previous studies had a longer median survival time and a higher PPV (table 5).10 11 13 Therefore, it can be concluded that the short median survival time in South Korea contributed to lowering the accuracy of the prognostic model, resulting in a similar accuracy to that of CPS.
Limitations
First, this comparison study was conducted in a single National University Hospital. Additionally, only patients in palliative care units were included as subjects. Therefore, it cannot be generalised to all patients with far advanced cancer who receive palliative care from different facilities in South Korea. Multicentre studies should be conducted with patients participating in different settings. Second, because the prognostic prediction results calculated by each model were not the same, the accuracy of these models could not be directly compared, so the superiority and inferiority between the indicators could not be compared either. Additionally, absolute agreement was used as one of the values for comparing the accuracy of the prognostic models. Absolute agreement is a value to show how accurately the survival period predicted by the prognostic model coincides with the actual survival period. However, due to the structural nature of the calculation formula, the value of absolute agreement decreases as the number of groups categorised by the prognostic model increases, and increases as the number of groups decreases. Therefore, it is not suitable for comparing the accuracy of models with different number of categorised groups. Third, Because the number of subjects was small, we could not analyse the accuracy of prognostic factors for each type of cancer. A large-scale study is needed to validate and compare the prognostic models by each type of cancer.
Despite these limitations, this study is the first one to prospectively compare the accuracy between the validated prognostic models and CPS for Korean patients in palliative care units, which could be regarded as a major strength. In addition, considering that prognostic models predict the life expectancy in different forms, it is not appropriate to compare the accuracy of these models with single analysis method. Therefore, we tried to compare accuracy using various analysis methods such as OA, absolute agreement and AUROC.
Conclusion
In conclusion, previously developed prognostic models were compared in Korean patients in palliative care units. All prognostic models significantly predicted survival time, and there was no difference in accuracy. However, since the total accuracy tends to be low especially in long survival groups, and the reason cannot be clarified through this study. Therefore, it is necessary to conduct multicentre study including patients with far advanced cancer in various settings.
Data availability statement
No data are available. All data will be deleted after the study is completed according to the direction of the Research Ethics Committee of Pusan National University Hospital.
Ethics statements
Ethics approval
This study was conducted with the approval of the Research Ethics Committee of Pusan National University Hospital (IRB 1604-002040).
References
Footnotes
Contributors We certify that no portion of this manuscript has been previously published, and we agree to transfer copyright to BMJ Supportive & Palliative Care in the event that this manuscript is accepted for publication. None of all of authors had a personal or financial conflict of interest.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.