Evaluation methods

In terms of methods of evaluation, the literature highlights a broad range of methods, including self-devised questionnaires, qualitative interviews and the use of data collection instruments such as grief inventories and depression scales. Much research on bereavement focuses on long-term interventions and there is next to no evidence to support practices in the immediate aftermath of death. In many cases, information is collected at a single time point—on completion of the programme or intervention. Across both adult and children’s bereavement services, formal evaluation appears to be ad hoc and may not provide the quality of information required for robust service development. An examination of how bereavement programmes were evaluated identified 38 studies that reported specifically on approaches to bereavement service evaluation.19 Outcomes were generally related to use of crisis services, participation in ongoing bereavement support groups and identification of those who would benefit from services. The included studies primarily used questionnaires or surveys for data collection. More detail on these approaches is provided within the themes below.

Patient-centred/Family-centred care

This theme contains studies that reported on how individuals experience end of life and bereavement care interventions, including families and loved ones. The interventions include actions to support memorialisation, such as creation of mementoes (letters, pictures), hand and lip prints or locks of hair and provision of comfort items such as twinned knitted hearts, one of which remains with the deceased person while the other is kept by loved ones. The interactions that take place in the context of end of life and bereavement care are a key consideration in terms of how they contribute to the overall experience. This relates not only to the kinds of therapeutic relationships required when caring for patients and families but also to how staff work together to provide compassionate care. This theme includes the interpersonal aspects of interventions and how they are evaluated.

The studies report on family experiences, both directly and as a proxy for patient experience. In the UK, families’ experiences of end-of-life care have been evaluated using satisfaction surveys, with one of the most prominent modes of data collection in the UK being the ‘Views of Informal Carers—Evaluation of Services’ (‘VOICES’) survey.20–22 This retrospective quantitative survey includes data relating to demographics, the quality of clinical care, covering issues such as compassionate communication and symptom control. There are also free-text questions to gather descriptive data. The survey is recommended as best practice in enhancing hospital-based end-of-life care,23 and has been widely used on institutional, Trust and national levels to support and inform service development. However, while a profile of the quality of end-of-life care is essential, the VOICES questionnaire provides minimal information about what works, for whom and in what circumstances.

Efstathiou et al 24 described nine bereavement interventions within the literature, including personal mementos, condolence letters, debriefs for staff and story-telling. The quality of evidence of effectiveness is considered weak and not generalisable; the success (or otherwise) of such interventions appears to be highly dependent on the role and way such interventions are delivered. There is some evidence that acts of remembrance and memorialisation are valued by bereaved family members.25–29 These studies have primarily used qualitative methods to find out families’ perceptions of having received mementoes (such as ECGs, locks of hair or handprints) following the death of a loved one.

There appears to be a clear role for hospital-based bereavement care, particularly in the support of families of children.30 Much of the current research about hospital-based bereavement care is derived from the intensive care unit (ICU) and critical care unit settings, where deaths can be unexpected and sudden. A scoping review of memory-making in adult ICU31 identified seven studies presenting research on this intervention, using both qualitative and quantitative methods. The acceptability and comfort offered by the interventions was largely ascertained by proxy, for example, by asking nursing staff for their opinions, rather than directly from families.

Several studies examining family perceptions of care made use of surveys and questionnaires for the purpose of evaluation.32–34 Some questionnaires had been developed specifically to evaluate a single service, while other studies made use of existing, validated tools such as the ‘Satisfaction With Bereavement Experience Questionnaire’ and the ‘Quality of Dying and Death in the Intensive Care Unit survey’.35 One of the challenges in using quantitative instruments for the purposes of evaluating services is that they do not capture the complexity inherent in this form of care. For example, of 52 studies identified in a review by Garstang et al 36 on parents’ needs following sudden death of a child, fewer than a quarter used qualitative methods.

While quantitative instruments such as surveys play an important role in evaluation, avoiding some of the ethical complexity of more iterative research approaches, they can limit the level of detail obtained, in particular the generation of new insights into how an intervention might work in particular situations. A further consideration is the variable reliability in different survey instruments in different populations. Evaluations may be prone to bias as users of such services are a self-selecting group, so findings of such studies may not indicate the value of interventions for people who decline or are not afforded access to services, and who may be highly vulnerable to the adverse impact of grief.13

The use of qualitative methods provides a more nuanced insight into these kinds of individual interventions. In the field of paediatric intensive care, Bloomer et al 37–39 describe how observations of practice reveal several ways in which nurses support families in the period leading up to a child’s death. Again, respecting the child as a person and collecting and creating mementoes appears to be an important part of nursing care, and appears to empower parents and families to be a part of the process rather than just onlookers.

There is little research that focuses specifically on the interpersonal nature of care in the context of specific interventions. One small Australian study by Clark et al 11 highlights the ramifications of the interpersonal encounters between them and the healthcare providers. The study combined the use of the Quality of Death and Dying tool with semistructured interviews to evaluate experiences of bereavement in the acute hospital settings. Two studies sought to describe the role of staff in providing immediate care following a death. Raymond et al undertook a systematic review40 and suggested that nurses provide vital patient-centred and family-centred care at the end of life and that this continues into the bereavement period. Naef et al 41 described ways in which health professionals in Switzerland provide immediate support following a death in acute and psychiatric care settings, via a cross-sectional online questionnaire (n=196). Roles included enabling viewing of the deceased, sending condolence cards, providing information on funerals and other support, and referring onwards. Nurses provided more immediate care, while physicians were involved in screening for bereavement risk factors. Notably, participants observed that such aspects of their roles are rarely evaluated and there is a need for support and education. The value of volunteers in the provision of hospital end-of-life support was the subject of a literature review by Bloomer and Walshe,42 with the emphasis being on ‘being with’ the dying person in a non-specialist capacity. The psychosocial aspect of the role was considered to be particularly valuable, alongside the clinical care provided by the core healthcare team.

Several studies report on interventions that sustain or maintain personhood in the face of dying. A systematic review of 21 studies by Kochen et al 43 summarises a range of interventions to support families when a child dies. The studies point to the importance of acknowledging and recognising a child who has died through the creation of keepsakes by hospital staff. In general, the literature in this area suggests that this is appreciated by bereaved loved ones. However, the authors conclude that the interventions lacked an empirical evidence base and suggested that one approach to evaluating them is to do so against theoretical concepts such as theories of grief and loss.

Institutional approaches

Many of the interventions described in the theme ‘Person-centred/Family-centred care’ were implemented on an individual ward level, rather than across entire institutions. This theme addresses changes across institutions, which were perceived to impact on the quality of end of life and bereavement support.

The ‘VOICES’ survey reports on the quality of the hospital environment and repeatedly, the importance of privacy, dignity and sensitive care after death are highlighted.21 Coombs et al’s44 scoping review of family experiences of end-of-life care in ICU identified the impact of flexibility in terms of family perceptions of support, including hospital parking, visiting hours and privacy. Particularly in critical care of children, the importance of the hospital environment is emphasised, such as in the grounded theory study by Butler et al.45 Moss et al 46 note that three existing randomised controlled trials of bereavement interventions omitted key bereavement outcomes, including communication and connectedness, finding balance, and relationships and social functioning.

Where service developments take place across entire institutions, it is important to obtain information about staff perceptions. Walker and Deacon47 attest to the impact on staff of caring for the suddenly bereaved. Through a series of interviews with staff, they identified that staff find this role simultaneously important and a source of unrest. Participants in the study identified the context of care as a key influencing factor in staff’s perceptions of this aspect of their role; this includes attitudes to death and dying, professional support from colleagues, and the culture of care on that ward or shift. Participants indicated that care provision was less challenging when departmental guidelines about visiting times and numbers of visitors were relaxed. Participants identified a number of aspects of this type of care that they perceived to be rewarding, including ‘person-centred moments’, informal debriefs with colleagues and the ability to establish priorities of care with families and members of the multidisciplinary team. Bristowe et al 48 identified tangible changes in practice from the views of relatives when new models of care such as the Amber care Bundle were put into practice, notably in relation to the effectiveness of open and honest communication from clinicians.

A review by Harrop et al 49 appraised bereavement support during the COVID-19 pandemic, identifying that organisational culture and prioritisation of bereavement care across the entirety of the institution is highly valued by staff in those clinical settings where death occurs.

Bristowe et al 50 examined healthcare professionals’ views of a complex intervention to improve care towards the end of life (the “AMBER” care bundle) and although the intervention itself relates to decision-making in relation to escalation of treatment and advance care planning, both of which fall somewhat outside the remit of the review and fit more with the role of the specialist palliative care team, this is a useful study in terms of looking at approaches to evaluation for complex interventions relating to the end of life. Understanding perceptions can help to shape implementation plans and education to ensure consistent use of a service or model. One of the findings of Bristowe’s evaluation was that the implementation of the AMBER care bundle appeared to have an impact on staff through serving a symbolic purpose—it indicated that the patients in receipt of the intervention were at a particular point on their illness trajectory (in this case, at a point where recovery is uncertain) and the fact of being identified as an appropriate patient for the use of the bundle led to staff awareness of the wider context of their situation becoming apparent. This idea of symbolic relevance may be an important element of large-scale changes in institutions.

Walsh et al 51 undertook an evaluation of a whole-hospital approach bereavement services through secondary analysis of qualitative data from focus groups with bereaved people and staff (n=21). The approach they evaluated took place in one Irish hospital and included appointment of a dedicated bereavement coordinator, whose role included staff support and training across the institution. Improved practices in a number of areas were identified, such as the establishment of ‘memory bags’, closer working between services such as organ transplantation, ward and mortuary staff and bereavement care coordination. Staff reported a sense of pride in the development of the bereavement services. The central role of management and leadership was also identified as contributing to the success of the service changes.

Using a case-study approach to evaluation can help to make sense of large-scale innovations. Della Penna et al 52 undertook a quality improvement study of an inpatient palliative care service across a large healthcare organisation in the USA. Through interviews with national, regional and local leaders, a number of factors were identified that were thought to contribute to the widespread uptake of the innovation across the organisation. Alongside evidence of improved patient satisfaction (as identified through surveys), the impact of leadership and support on creating a change in culture appears to be pivotal.53

Infrastructure and support systems

This theme addresses interventions that appear to act on service infrastructure and support systems. In contrast to the ‘institutions’ theme above, they propose ways in which support systems can be strengthened through service development.

Observational research suggests that acute hospital staff may need support after patient deaths, particularly around sudden or unexpected deaths38; a need for education alongside the intervention has been established in other evaluations of end-of-life care innovations. The need for support for staff is well established38 54 55 particularly in those clinical settings where there are more unexpected or traumatic deaths.47 56 There are times when staff need immediate advice and support in relation to unfamiliar or particularly challenging end-of-life situations. Role-modelling and expertise-in-action may therefore also be an important contextual factor in the acceptability and action of the individuals supporting patients and their loved ones.38 There is little research on interventions to support staff in the immediate aftermath. A small, anonymised survey on a ‘sacred pause’ in a single ICU in the USA suggested taking time to reflect on deaths permits closure and builds resilience within the team.57

Shariff et al.’s narrative overview of ICU nurses55 focused on the needs of the ‘suddenly bereaved’, and the particular challenges that this poses both to relatives and to the staff providing care. An ethos of caring support between staff was seen as fundamental to being able to enact this difficult part of the job. Contextual factors that impeded the provision of care included manpower resources, the conflicting demands of other patients and the brevity of the encounter with the person who has died and their families. Where death had not been discussed, or there was conflicting information provided by members of the multidisciplinary team, staff reported finding bereavement care more difficult as death felt more like a failure and was unanticipated. The literature identified in Shariff et al’s review55 further supported the importance of small acts of humanity—making a cup of tea, providing physical and temporal space, remembrance and gentle conversation—on the experiences of the suddenly bereaved.

Transdisciplinary working is generally considered to be an important component of the success of services around the end of life, given that it is a time of such uncertainty and unpredictability, with patients often requiring input from a variety of services in a variety of care settings.58 Efstathiou et al 59 undertook a realist evaluation of a coordinated end-of-life care service. They concluded that the service’s acceptability and success was partly related to the way in which expectations about pre-existing services and traditional roles were challenged. However, the authors also considered that challenging traditional boundaries brought with it the need to establish and maintain clear and realistic expectations, as well as investment in recruiting, educating and developing the workforce.

Research that incorporates observation of service provision has had somewhat more success in capturing the kinds of mechanisms and contexts of care at this time of life. Woodthorpe and Komaromy60 explored the role of mortuary staff in hospital-based bereavement care services through undertaking an ethnographic study of one mortuary team. The study included observation and interviews, and participants were keen to describe the barriers and facilitators to the provision of mortuary services. They challenged existing perceptions that the dead person’s body should be kept hidden, suggesting instead that it is important to promote visibility of death and dying to counter stigma. Becoming more integrated into multidisciplinary teams was a common wish expressed by participants, as well as recognition for their ‘bridging’ role between clinical care of the living person, and care of the dead person. This study also describes the use of interventions such as enabling viewing of the deceased person34 but did not provide information on how this is valued by the bereaved. A qualitative approach, such as that taken by Chapple and Ziebland,15 is able to provide insight into relatives’ perceptions of being able to view the body of a deceased loved one following traumatic death.

Structural features of organisations and institutions have not been well evaluated in terms of their impact on end-of-life care and bereavement. It is difficult to evaluate cost-effectiveness, not least because of the lack of clarity about outcome measures.61 One example identified in this review was a realist evaluation of a single-point coordinated end-of-life care service,59 which suggested that several mechanisms contributed to success: the presence of coordinated services, the recruitment and development of the workforce, articulation of clear roles and the management of expectations. One approach to evaluate effectiveness of an intervention is to investigate the impact of an intervention on a particular outcome. In a stepped-wedge cluster trial by Aoun et al,62 researchers investigated whether involvement of palliative care before death had an impact on whether the patient achieved their preferred place of death, and how carers perceived their level of grief. These findings in end-of-life care are mirrored in a recent narrative review on bereavement by Harrop et al,49 examining the relative success of elements of systems approaches to mass bereavement. The latter highlighted that those characteristics of successful system-level responses were that they were proactive and centrally organised, with an emphasis on psychosocial support. It is perhaps unsurprising that the elements of success in end of life and bereavement care show such overlap.