Small Data: DWP leads the way on statistical complaints

If government departments do questionable things with statistics you can send them to the headmaster, writes Anthony Reuben.

In this case, the headmaster is the UK Statistics Authority (UKSA) and its chair Sir Andrew Dilnot.

Sometimes people complain about the use of a statistic and sometimes the UKSA undertakes its own investigations. It then writes letters with its conclusions, which are published on its website.

I've spent the last few days reading all the letters it has sent out since it was established in 2008, because that's the sort of thing I do.

I concentrated on letters that were critical or suggested changes were needed.

Since the last General Election in 2010 there have been 47 such letters, and 16 of those have been about the Department for Work and Pensions (DWP).

If you go back to 2008 it's 16 out of 59.

To put that into context, the department in second place was the Department of Health with five. The Office for National Statistics, which produces about one third of official statistics, had three.

There was a range of actions being criticised.

In April this year, for example, the UKSA criticised the DWP for saying in a press release that more than 50% of decisions on disability living allowance are made on the basis of the claim form alone without any additional corroborating medical evidence, when the figure should actually have been 10%.

In March it criticised employment minister Esther McVey for telling the House of Commons that unemployment had fallen 400,000 since the general election, when it had actually only fallen by 7,000.

So what's going on? The UKSA said in a statement: "The proportion of concerns raised with us about the DWP reflects a range of factors, including the salience of the policy initiatives undertaken by the department; the range of interests affected by the delivery of the department's policies; the complexity of the systems the department implements; and the department's statistical practices."

So in other words, people are more likely to complain about DWP errors because they are in controversial areas. But is that really enough to account for the gulf between the DWP and other departments that produce considerably more statistics?

The department itself said: "Since May 2010 the DWP has led the way in openness and transparency of statistical releases by publishing more than 770 releases and datasets.

"Great care is taken to get things right, and in this time the UKSA has only written direct to DWP Ministers on two occasions about issues raised with it on DWP statistics. DWP has responded to these points and taken on board UKSA's suggestions."

It's worth mentioning that the UKSA does not generally write to the minister - it writes to the person who has complained and sends a copy to the minister. Misusing statistics again? You decide.

Subscribe to the BBC News Magazine's email newsletter to get articles sent to your inbox.