Article Text

Identification and operationalisation of indicators to monitor successful uptake of advance care planning policies: a modified Delphi study
  1. Konrad Fassbender1,
  2. Patricia Biondo2,
  3. Jayna Holroyd-Leduc3,4,
  4. Alexei Potapov1,
  5. Tracy Lynn Wityk Martin5,
  6. Eric Wasylenko2,6,
  7. Neil Angus Hagen2 and
  8. Jessica Simon2,4
  1. 1Department of Oncology, University of Alberta Faculty of Medicine and Dentistry, Edmonton, Alberta, Canada
  2. 2Department of Oncology, University of Calgary Cumming School of Medicine, Calgary, Alberta, Canada
  3. 3Department of Medicine, University of Calgary Cumming School of Medicine, Calgary, Alberta, Canada
  4. 4Department of Community Health Sciences, University of Calgary Cumming School of Medicine, Calgary, Alberta, Canada
  5. 5Palliative / End of Life Care, Alberta Health Services, Edmonton, Alberta, Canada
  6. 6John Dossetor Health Ethics Centre, University of Alberta, Edmonton, Alberta, Canada
  1. Correspondence to Dr Konrad Fassbender, Department of Oncology, University of Alberta Faculty of Medicine and Dentistry, Edmonton, Canada; konrad.fassbender{at}


Background In 2014, the province of Alberta, Canada implemented a province-wide policy and procedures for advance care planning (ACP) and goals of care designation (GCD) across its complex, integrated public healthcare system. This study was conducted to identify and operationalise performance indicators for ACP/GCD to monitor policy implementation success and sustainment of ACP/GCD practice change.

Methods A systematic review and environmental scan was conducted to identify potential indicators of ACP/GCD uptake (n=132). A purposive sample of ACP/GCD stakeholders was invited to participate in a modified Delphi study to evaluate, reduce and refine these indicators through a combination of face-to-face meetings and online surveys.

Results An evidence-informed Donabedian by Institute of Medicine (IOM) framework was adopted as an organising matrix for the indicators in an initial face-to-face meeting. Three online survey rounds reduced and refined the 132 indicators to 18. A final face-to-face meeting operationalised the indicators into a measurable format. Nine indicators, covering 11 of the 18 Donabedian×IOM domains, were operationalised.

Conclusions Nine ACP/GCD evidence-informed indicators mapping to 11 of 18 Donabedian×IOM domains were endorsed, and have been operationalised into an online ACP/GCD dashboard. The indicators provide a characterisation of ACP/GCD uptake that could be generalised to other healthcare settings, measuring aspects related to ACP/GCD documentation, patient satisfaction and agreement between medical orders and care received. The final nine indicators reflect the stakeholders’ expressed intent to strike a balance between comprehensiveness and feasibility within a large provincial healthcare system.

  • service evaluation
  • methodological research

Statistics from

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.


  • Twitter @K_Fassbender

  • Contributors NAH and KF conceived the study. All authors (KF, PB, JH-L, AP, TLWM, EW, NAH and JS) contributed substantially to the design of the study. PB and AP contributed substantially to data acquisition and analysis. All authors contributed substantially to interpretation of data. KF, PB and JS drafted the article. All authors revised the article for important intellectual content, approved the version to be published and participated sufficiently in the work to take public responsibility for appropriate portions of the content. KF is responsible for the overall content as guarantor.

  • Funding This work was funded by Alberta Innovates Collaborative Research and Innovation Opportunities (CRIO) Programme Grant #201201157: 'Advance Care Planning and Goals of Care Alberta: A Population Based Knowledge Translation Intervention Study').

  • Competing interests JS is the Physician Consultant for Advance Care Planning and Goals of Care, Calgary Zone, Alberta Health Services. TLWM. was the Provincial Lead for Palliative End of Life Care Practice Development, Alberta Health Services at the time of this work.

  • Patient consent for publication Not required.

  • Ethics approval This study was reviewed and approved by the University of Alberta Health Research Ethics Board (ID: Pro00052297) in February 2015. Written, informed consent was obtained from panellists who participated in face-to-face meetings; implied consent was obtained from panellists who completed online surveys.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data availability statement The final indicator set is available for use in the main document. The original longlist (n=132) of potential ACP/GCD indicators is available from the authors upon request. The supplementary online table catalogues the 18 indicators resulting from round 2b. The Standards Manual outlining approaches to measurement (data sources, target populations, data collection instruments) and detailed characterisation of the indicators (e.g., measurement setting, level of measurement, numerator, denominator) is available at