Article Text
PDF
Abstract
Background The increase in the numbers of patients requiring palliative care input prior to death, and a global economic situation where few countries are able to invest further in specialist palliative care services, has meant an increased focus upon ‘generalist palliative care provision’. The goal of the present review is to ascertain what questionnaire tools exist to measure the perceived competence of generalists in palliative care provision.
Method A systematic review of both qualitative and quantitative literature was undertaken. Medline, Medline in Progress, PubMed and CINAHL databases as well as hand searches of Palliative Medicine, International Journal of Palliative Nursing and the Journal of Palliative Care were conducted for the period 1990–2010. A checklist adapted from Hawker et al (Appraising the evidence: reviewing disparate data systematically. Qual Health Res 2002;12:1284–99) was used to select and assess data.
Results 19 of the 1361 articles met the inclusion criteria. Overall, a lack of validation and a focus upon the physical aspects of symptom management was apparent. No single validated questionnaire to measure perceived competence in palliative care management among health professionals involved in generalist palliative care management could be identified.
Conclusion The rising prominence paid to generalist care provision points to an urgent need for further development of comprehensive and validated perceived competence measurement tools.
Statistics from Altmetric.com
Background
Given the context of ageing populations and a rise in chronic diseases, it is unsurprising that the demand for palliative care provision is predicted to rise significantly over the coming decades in most developed countries.1 This increase in the numbers of patients requiring palliative care input prior to death, and a global economic situation where few countries are able to invest further in specialist palliative care services, has meant an increased focus upon ‘generalist palliative care provision’. Generalist palliative care has been defined as that ‘provided for those affected by life-limiting illness as an integral part of standard clinical practice by any healthcare professional who is not part of a specialist palliative care team.’2
The End of Life Care Strategy for England identifies two groups of health professionals who do not work in specialist palliative care settings or have accredited training in specialist palliative care but who do have a role to play in palliative and end of life care management.3 The first group consists of those who ‘frequently deal with end of life care as part of their role’, including general practitioners (GPs) and secondary care doctors and ‘their teams’. The second group comprises health and social care staff who ‘infrequently have to deal with end of life’ (although the extent to which this applies to care home staff as stated by the document is debatable). This latter group is seen as requiring ‘a good basic grounding in the principles and practice of end of life care and being enabled to know when to refer or seek expert advice or information’, while the former has the ‘greatest potential training need’, particularly in relation to ‘communication skills, assessment, advanced care planning and symptom management as they relate to end of life care’.
Recommendations for a need for increased education and training for generalists in palliative care provision are supported by previous research particularly in the area of non-cancer palliative care.4,–,9 However, findings from recent research indicate that there is a need to focus on the perceived competence in palliative care provision among generalists and on the extent to which palliative care is perceived to be a part of generalists' core workload.10 A large qualitative study conducted in both England and New Zealand found that many health professionals working in non-specialist palliative care settings, and particularly in acute hospitals, do not perceive palliative care provision as their responsibility.10 A lack of clarity about whether the responsibility for palliative care provision for older patients should lie with generalists working within primary or secondary care settings was also identified.11 It seems likely that training and education initiatives that do not address these issues are unlikely to be successful in the long term.
To build the much needed evidence base in this field, there is a need to move beyond exploratory qualitative studies to conduct large-scale surveys that capture generalist competence in, and perceived responsibility for, palliative care provision as well as measure the effects of education and training interventions.5 Robust validated tools with the ability to be used in multiple generalist settings would be particularly valuable in identifying the potential diversity among generalists in relation to education and training needs.
The critical literature review reported in this paper was conducted to identify questionnaire tools that have been developed to measure the perceived competence of generalists to provide palliative care.
Aim
To ascertain what questionnaire tools exist to measure the perceived competence of generalists in palliative care provision.
Methods
A systematic review of both qualitative and quantitative literature was undertaken to examine the evidence relevant to the measurement of competence in palliative care provision. The review was conducted in the following stages: (1) search strategy, (2) inclusion criteria, (3) assessment of relevance, (4) data extraction and appraisal and (5) data synthesis.
Search strategy
The goal of this research was to conduct a systematic and comprehensive review of the published evidence related to the selection/inclusion criteria. A list of keywords was developed by consensus among the reviewers, and relevant databases were searched. Medline, Medline in Progress, PubMed and CINAHL databases were searched for literature published between 1990 and 2010. Relevant articles were also located through hand searches of the following journals: Palliative Medicine, International Journal of Palliative Nursing and the Journal of Palliative Care. In addition, relevant references from bibliographies were examined and citation indices were followed up.
Inclusion criteria
Inclusion criteria were developed in consultation between the reviewers. Literature selected for review had to include an assessment of competence using a questionnaire. The topic of the research had to be perceived competence and the research participants must have been generalists in palliative care provision. Further selection criteria limited the scope of the review to peer-reviewed articles published in English between 1990 and December 2010. Both qualitative and quantitative research was eligible for inclusion. Keywords included ‘palliative care’, ‘terminal care’, ‘end of life care’, ‘hospice care’, ‘competency’ (and associated words/synonyms) and ‘generalist’ (and associated words/synonyms).
Assessment of relevance
Study assessment proceeded through a systematic sifting process that involved examining the title, abstract and body of the paper. The two reviewers (MG) and (RB) independently assessed each paper. Disagreements between the reviewers were resolved through discussion. It was anticipated that an assessment protocol that could take account of the diverse nature of the research approaches would be required. A method developed by Hawker et al was deemed most appropriate for systematically and objectively reviewing both qualitative and quantitative research papers.12
Data extraction, appraisal and synthesis
A checklist adapted from Hawker et al was used to select and assess data on the abstract and title, introduction and aims, methods and data, sampling, data analysis, bias, results, transferability or generalisability, and implications and usefulness. The diverse nature of the included research prevented/precluded the undertaking of a statistical analysis of study results.12 Although Hawker et al recommend grading empirical studies when reviewing disparate data systematically, methodological quality scores were not applied within this review as assigning such values to studies is arguably an ‘arbitrary and unscientific process’.13 Hence, studies were not excluded on the basis of quality assessment (figure 1).
Flow chart of the included literature.
Results
Nineteen of the 1361 articles met the inclusion criteria. Excluded articles (n=1342) were not relevant to the identified objectives. Of the included articles, 10 assessed doctors, 3 looked solely at nurses and 2 included both doctors and nurses. Eight articles were based on studies conducted in the UK, five in the USA and two in Canada. Two studies from Norway, one from Austria and one from The Netherlands accounted for the remainder of the articles reviewed. Review results are presented below as both a summary and an evaluation in relation to each research objective. The screening process identified 19 citations reporting on the development or the use of 22 questionnaire tools.
Overview of the questionnaires
One instrument was developed in the early 1980s.14 15 Four instruments were developed in the 1990s.16,–,19 The remaining tools were developed from 2000 to 2010.20,–,27
The majority of the studies reviewed (10) assessed the perceived competence of doctors, while 3 measured the competence of nurses. Two articles used questionnaires with both doctors and nurses, and four questionnaires were used to gauge the self-assessed competence of students.
Newly qualified doctors in the UK were assessed by Charlton and Smith and GP registrars by Charlton et al.14 15 Dryden and Addicott surveyed all practitioners who were attending a pilot study day.23 Primary care physicians (n=1000) were randomly selected from a list of registered physicians in a study conducted by Farber et al.24 Residents (house officers/senior house officers in the UK and several Commonwealth countries) were assessed in three studies: internal medicine residents (n=282) from university and community-based programmes, residents (n=30) in internal medicine in The Netherlands and family medicine residents (n=36).22 26 27 Three studies assessed students: all students (n=25) who took part in a 2-year course on palliative care, fourth-year medical students (n=216) from Liverpool University and fourth-year medical undergraduates undertaking a 2-week programme in palliative medicine.25 28 29
Nurses were also assessed in three studies. Nurses (n=172) from 10 hospices and a community nursing service were evaluated for the effectiveness of their communication skills.19 Slatten et al assessed clinical nurse specialists who had completed a postgraduate programme in palliative care in Norway in one of the two university colleges.30 In a survey conducted by Shipman et al, 374 district nurses were assessed from eight cancer networks over a 2-year period.31 Weissman et al assessed a range of participants: third-year and fourth-year medical students, internal medicine interns and internal medicine residents.18
Study designs were descriptive14 16 18 20,–,22 24 30 and evaluative.15 17 19 23 25 27,–,29 31 32 The evaluative studies dealt particularly with measuring skills and knowledge development as the result of education programmes.
The questionnaires varied in length from those with fewer questions with more detailed scenarios14 15 to those with numerous items. Questionnaires with less than 30 items were most often used: 8,25 14,19 22 1732 and 23.20 28 29
The majority of the questionnaires were designed for self-administration. By contrast, the Objective Structured Clinical Examinations used by Pereira et al was observer-rated.27
The following tables present a description of the measures used in the research articles (table 1) proceeding from questionnaires incorporating one or more of the most widely used and tested scales (table 2) to more recent author-developed tools (table 3).
Summary of the included research articles
Characteristics of modified instruments to measure competence in palliative care provision
Characteristics of newly developed instruments to measure competence in palliative care provision
Content
Instruments differed in the number and aspects of palliative care competence assessed. Theoretical knowledge and proficiency were assessed by a number of the measures.17 21 24 27 30 31 Scales measuring the participant's perceptions of competence in palliative care delivery were included in several of the questionnaires. Mulder et al, for example, assessed perceived competence over 18 situations of palliative care.26 Areas of measurement included communication regarding euthanasia, organ donation, stopping treatment and the patients' fears. Perceived competence was often expressed as a measure of the level of confidence.15 22 23 25 32 Dryden and Addicott provided participants with six statements based on which they rank their competence.23 For example, ‘I feel able to support patients who are in pain’ and ‘I can identify and help manage common symptoms at the end of life’. A few of the questionnaires paired measures of competence or perceived competence with measures of anxiety in dealing with common issues in palliative care (eg, symptom management, depression).15 27 Measures of attitudes/opinions regarding palliative care issues were also found in a number of the questionnaires.20 21 27 32 Burge et al surveyed family medicine residents for both attitudes and knowledge of palliative care.21 The attitudes section of the survey assessed communication, preparation for death, relieving suffering, cultural factors and opioid medication use. The 50-question instrument had 25 of the 50 items allocated to attitudes and 25 for knowledge. The research by Bradley et al sought specifically to develop a valid and reliable assessment instrument of clinician attitudes about care for patients with a terminal illness.20 Some of the questionnaires used existing scales that had been modified for completeness or to improve applicability.15 17 28 32 The remaining questionnaires used measures developed by the researchers based on a review of the literature, various palliative care education programme guidelines as well as other existing instruments.
Structure
Both numerical and verbal scales were represented in the sample. The most common format was a Likert scale. Of the 19 articles, 17 incorporated some form of a Likert scale in their questionnaire. A 1–5 Likert scale was often used.17 22 27 30 32 Charlton and Smith asked participants to rank overall confidence on a 6-point Likert scale from 0 (no confidence) to 5 (complete confidence).14 Wilkinson et al provided a 10-point Likert scale from 1 (not at all confident) to 10 (very confident).19 Dryden and Addicott used a Likert scale from 1 (very little knowledge) to 10 (extremely knowledgeable) while Shipman et al used a Likert scale from 1 (very confident) to 4 (not at all confident).23 Dichotomous yes/no items were used in knowledge subscales as well as to ascertain education received and education desired.17 31
Open-ended items were also used in a number of questionnaires. This tended to be one or two questions asking for further recommendations or comments at the end of the measurement tools. Two studies provided space for comments upon conclusion of the questionnaire.14 15 Such comments were analysed through a coded theme approach.
Psychometric properties
Of the 19 research papers analysed, 8 did not report validation information for the measures used in the questionnaires. Of the remaining articles, the amount of detail provided with respect to validation varied. Becker et al, for example, mentioned that the measures were pretested only for face and content validity.16 By contrast, the palliative care quiz for nursing and thanatophobia scales were rigorously validated by Hughes et al.17 Reliability data most often referred to internal consistency as measured by Cronbach's α.20,–,22 27,–,30 32 Both Hughes et al and Pereira et al assessed the reliability of their knowledge scales using the Kuder–Richardson (KR-20) formula for dichotomous variables.17 27 Validated instruments had moderate-to-good psychometric properties in relation to reliability as judged by the magnitude of the coefficients (range 0.63–0.96). Reported inter-item correlations for the measures however varied widely (0.01–0.93). Test–retest reliability was reported only for the measures used by Hughes et al.17 A few of the included research articles confirmed the structural validity of measures through the use of factor analyses.20 22 28 Five of the research articles assessed the face and content validity of their measures.16 24 27 28 32
Practicality
Of the 19 studies, 7 reported that more than 70% of respondents completed the study questionnaire.17 21,–,23 26 27 32 The majority of the studies reported response rates ranging from 24% to 69%. Becker et al, for example, reported a response rate of 30%.16 Both Becker et al and Slatten et al (response rate 50.6%) attributed problems with response to questionnaire length.16 30 Slatten (personal communication) reported that the instrument has since been reduced from 176 items to 26 items and renamed the Nurse's Clinical Core Competence in Palliative Care.33 Weissman et al reported only on the number of completed questionnaires and not on the sample size.18 Bradley et al and Landmark et al did not report their response rates.20 25 Some of the studies involving longitudinal designs demonstrated decreases in completions over time. For example, Hughes et al reported a 99% response rate for their precourse questionnaire; 72% completed the questionnaire at the end of the course and 51% completed the questionnaire after 1 year.17
Discussion
This paper presented findings of a critical review of the published research literatures to identify questionnaire tools developed to measure perceived competence in palliative care provision among generalist clinicians. While strengths can be identified in many of the tools identified, overall a lack of validation and a focus upon the physical aspects of symptom management was apparent. No validated questionnaire to measure perceived competence in palliative care management among health professionals involved in generalist palliative care management could be identified. Moreover, with the exception of the research by Bradley et al, the issue of perceived responsibility for palliative care provision was also not addressed.20
It is evident across the studies reviewed that questionnaires both played an integral role and were extensively used in assessing perceived competence in palliative care. However, although the studies tended to have an overall theme of palliative care competence, the research projects incorporated questionnaires that were very different in design and specific focus. These differences hinder comparisons of results across studies. It is apparent from the review results that questionnaire design and validation, which was the foundation of data collection in all of the studies, was often neglected. The validity of a questionnaire is grounded in its reliability. In other words, without reliability there can be no validity. Yet many of the research projects reviewed introduced their own instruments to improve upon tools used in previous studies leading to issues of incompatibility. Additionally, lack of validation increased the likelihood of bias due to situational effects present at the time of administration.34 Ultimately, a study is only as good as the data collected. As stated in a review by Olsen, ‘the use of an invalid instrument is simply a waste of time and money. The adequate preparation of questionnaires is thus essential for the quality of data’ (p. 2).35
This review used comprehensive search, retrieval and review strategies, although searches were limited to the English language and the grey literature was not searched. No previous review on this topic could be identified. It should also be noted that this research review included measures of perceived competence. Critiques of this type of assessment argue that self-reported competence or confidence does not accurately reflect true performance.36 37 Overall, self-assessment is by its very nature subjective and context dependent. Therefore, self-reported abilities will necessarily vary from actual abilities.38 39
The rising prominence paid to generalist palliative care provision at a policy level in many developed countries points to an urgent need to develop the evidence base in this area. Comprehensive and validated tools to measure perceived competence among generalists will be the key to future research activities, and the findings of this review indicate the need for further development work in this area.
Acknowledgments
Taima Campbell of Auckland District Health Board, Auckland, New Zealand, served as an advisor for the project and critically reviewed the study proposal.
References
Footnotes
-
Funding The funding for this research was provided by the University of Auckland, Auckland, New Zealand. The sponsor did not have any involvement in the planning, execution or write-up of the research. Additionally the funder played no role in drafting the manuscript.
-
Competing interests None.
-
Provenance and peer review Not commissioned; externally peer reviewed.