Social policy and austerity outcomes

It has been argued that sociologists should ‘catch up’ with behavioural scientists and get more involved in trials of social policy, engaging in a culture of ‘experimental government’. I would be the first to agree that we could develop a more interventional sociology. And trials – though often criticized by members of our discipline – are one possible way of doing that.

Trials, by appearing scientific, have some clear advantages over qualitative or observational studies, in terms of convincing policy makers. They create space for different ways of implementing policy to be recognized and tested. When done and reported with appropriate modesty, they can tell us something with a great deal of certainty.

But these positives are balanced by some problems in practice. Carrying out a controlled trial of a complex intervention is expensive and might take longer than policy makers are prepared to wait. If the findings contradict policies already enacted, the policies are seldom reversed. Indeed trials do not always end a policy debate, because there is often disagreement about the meaning of the results. Successful trials tell us that it’s possible to do something in one place, but don’t end discussion about whether that is possible or desirable in another.

In health care these problems are familiar. And now that trials are dominant in medicine a good deal of energy goes into upstream debates on the choice of outcomes to be used in an experiment, and downstream debates on how widely the results should be applied. Trials often fail to end controversies. Similarly ‘scientific’ approaches seem unlikely to remove the politics from social policy, but rather to displace and relocate some of those debates. To focus on outcomes – it’s a big assumption that governments will necessarily have the same ‘goods’ in mind as academic researchers or others, or that we could all agree what success would look like.

Let’s take as an example the reform of the benefit system, one of the major social policy debates of recent years. Though there are increasingly social policy experiments, still many of these debates focus on data showing some change after the policy is introduced, treating policy change itself as a natural experiment.

Changes in housing benefit, incapacity benefit and disability living allowance or the latest tax credit cuts or future modifications of sickness benefit assessment could all be studied with trials or with data from before or after. While a controlled trial would help ensure that knowledge was created about the effects of those policies rather than unrelated changes over time, that wouldn’t end the scope for disagreement.

First the parameters of the experiment, and the appropriate outcomes, would need to be agreed. If the case is being made that these reforms are necessary to rein in public spending then an evaluation might focus on levels of spending, but given the ways in which ‘need’ may be moved around the welfare system you would have to cast the net wide, much wider than any single government department (see arguments that pressure on NHS is increasing with cuts to social care).

The Grim Reaper outside Kiln House Jobcentre

Iain Duncan Smith argued at the Conservative conference a fortnight ago that ‘we want people to work because it’s best for them… all our reforms have a simple principle at the heart of them: to restore lives’. This is a much broader claim for welfare reform than reducing government expenditure alone. One could of course look at employment figures, but how to test the claim that employment is good for the people encouraged into work?

David Cameron might like to capture some of these policy effects in measurements of personal wellbeing among people affected but this is not that widely used. In health a Quality of Life measure is often used to capture psychological factors and the impact on daily activities as well as clinical parameters. But what measures should be used to count the stress of someone with disability going for yet another assessment or taking on a job for which they turn out to be unfit? Or the anxiety experienced by people managing with reduced benefits? Or the physical and emotional impoverishments of children who have less variety in the food they eat, fewer books and toys in the house, fewer trips to family or friends?

Meanwhile, debates about the effects of past benefit change are currently being pursued through less nuanced measures, and often through arguments about the number of deaths that might be attributed to changes to the benefit system. What does this tell us about the politics of social policy?

From the perspective of campaigners deaths or clinical harms caused by policy are newsworthy in the way that quality of life and experience is not.

Campaigners’ strategies vary but one approach is to simply collect together reports of the deaths of people that they claim are linked to ‘welfare reform’, see for example Calum’s List, and seek to publicise those. Other dossiers, like the latest Hunt file around 7 day working proceed by amassing ‘cases’ of people who say there were affected. Such lists represent additional data work to that done currently by government (though those collating the Hunt file ask people not only to respond online to their question but also to report them as near misses in hospital data).

Campaigners have also sought to force the publication of the government’s own data on deaths among benefit recipients (for examples see campaigns by Maggie Zolobajluk or Mike Sivier). They won this fight in August this year resulting in news reports about more than 2300 people who died after being found fit for work and losing benefits.

The Department for Work and Pensions argues that these deaths are not necessarily the ‘effect’ of policy change. If their scope could be agreed randomised controlled trials usually offer clearer evidence about causation, but in this case a single trial might not include one death, as these are thankfully rare events among the millions affected by benefits. Even if they were, I doubt that the argument would be over. Viewed from a government perspective small numbers of people suffering stress may appear only a small ‘harm’. Perhaps this is outweighed by the benefits to society of a ‘fairer’ welfare system, or the improved quality of life of people who find work?

But it is also possible to argue that any deaths represent a serious moral failure in social policy, which has put very vulnerable groups under additional stress, with tragic results for a few people. The significance of the numbers is hotly debated – is this a lot or a little? Neither natural or controlled experiments can resolve arguments about the overall balance of goods and bads from policy change.

Social scientists may also disagree about what is a ‘good’ outcome as well as how best to intervene in these debates. Some, like Adam Fletcher, might get involved by working to complicate and improve the evaluations made by government departments. Others, like Danny Taggart and Carl Walker, will look for ways to resist and challenge policies. I for one welcome the chance to debate these important questions and find ways to keep the politics alive in social policy.