In North Somerset, a need was identified to improve self-management and increase support for patients with fibromyalgia. To address this need a 7 week patient education course was developed and tested. Voluntary Action North Somerset (VANS), North Somerset Clinical Commissioning Group and Healthwatch North Somerset formed the steering group which guided the project and the evaluation. An evidence review was conducted to inform the planned service design, however it identified a limited and weak evidence base. As such, the steering group was clear with all stakeholders that this was a pilot project and that all feedback and learning would be used to inform future work.
Due to the innovative nature of the pilot, the steering group identified the need for an evaluation from the start of the project, although the scope of the evaluation evolved over time. The purpose of the evaluation was to produce a report for Healthwatch North Somerset and North Somerset CCG describing the impact of the pilot, lessons learnt, and recommendations for going forward.
Originally the team planned to use the Patient Activation Measure (PAM) as an evaluation tool. However, due to the requirement of a licence to use the PAM, alternative methods were adopted. The main tool used was a validated, public domain available, free tool known as MYCaW (Measure Yourself Concerns and Wellbeing questionnaire). The rest of the evaluation data was collected via questionnaires that were bespoke to the project. These included a participant health and course views questionnaire and an NHS service use questionnaire. The evaluation methods were only finalised in the last month of the project when an independent evaluator was recruited.
Mixed methods data was collected using a pre-post design. All evaluation of the PIC course took place via written questionnaire responses. The qualitative data was analysed using thematic analysis. This method generated themes in the data regarding all 13 participants’ concerns, which were slotted into categories such as ‘peer support’ and ‘improving confidence and self-esteem’.
The thematic analysis of MYCaW was presented in a table and a word cloud. The quantitative MYCaW data was presented in a histogram showing scores before PIC, after PIC, and 3 months after PIC. The NHS service use questionnaire findings were also presented in a table. The findings were interpreted as showing that the course was fairly well received by the participants and no negative reactions to the course were recorded. However, the number of participants was small (13 in total), meaning that the results lack reliability. The report has been shared with all stakeholders. The PIC steering group hope that by sharing the report with North Somerset CCG they may generate interest to secure future funding.
The take home message from this evaluation is around the importance of planning an evaluation from the very start of a new service. Pinning down the aims and objectives of an evaluation before a service is up and running can help to ensure that the right data is collected at the right time. The PIC pilot has pioneered a new way of approaching patient control and it is hoped that these methods will be replicated and modified to be used with other long-term health conditions in the future.