Arts & Health: Economics and the dangers of Randomised Control Trials
Couple of really interesting presentations and discussions at the ESRC funded Arts, Health & Wellbeing Research Network meeting in London. Unfortunately this was the last event in the series, but I’m sure that a longer term programme will emerge, especially when it was noted that there have been some 200 participants of which approximately 50 have been Phd students.
The first interesting area was a presentation by David McDaid on health economics and how that field interacts with research and decision making. We know that decisions about healthcare are made on the basis of efficacy and cost, but McDaid unpacked some of the basics for us. He highlighted that when looking at the economics of any decision about healthcare we need to understand:
- the cost of inaction;
- the cost of action (and here he pointed out that understanding project or programme costs is very difficult and arts organisations can be quite opaque about their costs. Costs also need to include in-kind costs absorbed by partners.);
- The cost effectiveness of the action in comparison to other potential actions;
- levers for maximising value (ie how to maximise the money spent by working on uptake and participation).
In terms of the cost of inaction he highlighted three areas to consider:
- Cost of every visit to the GP (which in the figures he showed was about £45 per appointment);
- The higher cost of attending Accident and Emergency;
- The even higher cost of hospitalisation.
- In parallel with this are the informal care costs (ie how much is the family bearing factored at hours times the minimum wage), and the out of pocket costs for the individual for treatments or lost earnings.
Some really interesting challenges emerged in response to McDaid’s framing of the economics through the example of ‘arts on prescription vs. individual therapy sessions.’ Firstly, why these are presented as alternatives when in many cases they might be complimentary? Secondly why reduced contact is presumed to be good when there are circumstances where greater contact with healthcare workers is the good outcome. To which David responded, “These are all good points, but the model of decision-making in healthcare economics is simplistic.”
On the back of McDaid’s presentation were two evaluated project case studies, one using reading with people with chronic pain, and the other using arts on prescription for people with depression and anxiety. Both were really significant, but looking at them through the lens McDaid had provided, you’d note:
- the need to focus on efficiency of delivery, maximise participation, understand operational finances and share models;
- be prepared to scale up from projects to programmes.
The afternoon presentations focused on the perceived weaknesses of two Randomised Control Trials recently published. Without trying to rehearse the details, some interesting points emerged which suggest that using Randomised Control Trial (the gold standard for evidence in healthcare decision-making) to prove the value of arts interventions is not something to be undertaken lightly.
Arts interventions need to be understood as “complex interventions” as defined by the Medical Research Council and interestingly this means that any “complex intervention” should,
- Start with a theory;
- define which ingredient(s) essential;
- include process analysis.
In terms of theory, healthcare is looking for the ‘theory of change’ that the study is going to test, but that ‘theory of change’ requires the practitioners (not just the researchers) to be very clear about their practice, and to deliver that practice in a consistent way in relation to the theory. (This clearly links to the ongoing development of a qualification in participatory practice by the ArtWorks programme in Scotland, jointly funded by the Paul Hamlyn Foundation and Creative Scotland.)
Perhaps one of the characteristics of any theory of change in the arts is the fundamentally voluntary nature of participation in that change. It’s one of the problems pointed out with Randomised Control Trials – people get selected to participate in the arts randomly. This is slightly problematic, particularly when you’re asking someone to engage in creative activity or even singing.
The point about defining essential ingredients is important – process-based work is often about context, empowerment and empathy as well as specific activity, but it’s extremely difficult to study more than one factor.
Finally the process analysis is important, on one level because that might a way to balance the attempt to define the essential ingredient, but also because timing and pattern are important in experiential work, e.g. participants are often interviewed for the baseline and then interviewed at the end of the programme, precisely the point where they might be feeling a sense of loss of an activity that had been enjoyed. Even a Randomised Control Trial is subject to such factors: not only what questions are you asking, but when are you asking them.
There was reference made to another Randomised Control Trial focused on singing, to be published imminently, which was ‘successful.’ It will be interesting to understand how this was constructed. But going back to McDaid’s point, scale may be critical because at least one really good, well evaluated, project was unable to engage with the Clinical Commissioning process simply because it’s too small (and most arts & health organisations are small even in the cultural sector, let alone in relation to healthcare).
The understanding of “complex interventions” in the medical literature bears further scrutiny and some references were suggested including Marchal (2013) (and Yin (2009) on “systematic case theory”).
Theo Stickley started the morning by offering an imagined scenario around the trajectory from the materialist understanding of healthcare that characterised the 20th Century through a transition to an holistic understanding of healthcare that could characterise the 21st Century. Whilst it’s important that the practitioners delivering arts in healthcare are well trained and professional, that they can articulate clearly their theory of change, as well as the economics of their interventions, the belief that the Randomised Control Trial is the way to prove the value might genuinely jeopardise what makes the arts distinctive from other interventions. We must become more articulate about the characteristics and values of our artforms and forms of intervention to ensure that they have equal status with the economics and the criteria imposed by the methods of research.
Only one speaker said, “and the art produced in this project was good too. We were proud of it.”
Presentations can be found here.
leave a comment