The Diploma in Palliative Medicine was established in 1991 and included communication skills as a major part of the curriculum. In order to assess the efficacy of doctors' communication skills in the Diploma examination, an assessment tool was developed based on the modified Prevara Score. Simulated (actor) patients are used for the consultation in the examination; the doctors' performance is assessed independently by the examiner and by the actor. This provides an opportunity to consider the methods used for describing agreement between raters. There was high correlation between examiners' and actors' scores and high acceptability of the scoring method. However, satisfactory agreement in terms of the mean differences between scores and their standard deviation between examiners' and actors' scores was not achieved. We have found the simulated patient interview to be a useful teaching and assessment tool. The good correlation between the observer's (examiner) and the recipient's (actor-patient) perception of the doctor's interviewing skills provides evidence of the validity of the assessment. However the actor-patients' ratings tended to be higher and the two groups of assessors could not be used interchangeably. We conclude that actor-patients are of value in teaching and in assessing the communication skills of doctors but produce different scores to clinical examiners.