Too good to be forgotten - why institutional memory matters

Many organisations are at risk of losing their memory because workers switch jobs too quickly, without passing on what they've learned. This "corporate amnesia" can have serious consequences, writes Phil Tinline.

Britons today change job more frequently than workers in much of the rest of Europe and, as the economy improves, that churn rate increases. Young workers move jobs much often than older workers did.

But there's a hidden cost.

Each time someone leaves their job, a chunk of the organisation's memory leaves too. How, then, do you run complex systems, see through long-term projects, or avoid past mistakes?

Short-term contracts and outsourcing reduce the appetite for learning company or product history. And when job losses land, even more knowledge is lost.

In 2012, one institution found that, as City firms poached its bright young employees, its staff turnover was hitting 28% - faster, apparently, than McDonald's.

And for Her Majesty's Treasury, after its experiences during the financial crisis, this was rather scary.

The Treasury's outgoing permanent secretary, Sir Nicholas Macpherson, says its young staff are highly knowledgeable but the short time they spend in their posts means they often miss out on the "folk memory".

This, he says, became very clear to him and to then Chancellor of the Exchequer Alistair Darling during the course of the 2008-09 crisis, when they realised "the vast majority of Treasury staff had never been through even a recession, let alone a banking crisis".

When the run on one of Britain's biggest banks, Northern Rock, struck in 2007, Sir Nicholas says, the Treasury "might have been able to stop the run, but we were all starting from first principles".

"We hadn't had a run on a bank of the sort we have with Northern Rock really since before the First World War," he points out. However, "there are really good books written about banking crises and I guess if we'd all read those, totally [absorbed] the knowledge which had been within them, maybe we could have acted more quickly".

To tackle the risk of corporate amnesia, the Treasury has started working with Dr Jon Davis and his team at the Policy Institute, based at King's College London. A King's doctoral student has been given privileged access to official records to conduct a unique study of the Treasury under Alistair Darling.

And Treasury officials now compete to attend seminars on recent Treasury history conducted by Dr Davis as part of an MA course.

These have been addressed by Lord Terry Burns, permanent secretary to the Treasury in the 1990s, former cabinet minister and ex-Treasury official David Willetts, former Chief Secretary to the Treasury Sir Danny Alexander and ex-shadow chancellor Ed Balls, as well as former Chancellors Lord Lawson and Alistair Darling.

The nuclear weapons sector of public policy is also troubled by corporate amnesia.

For years after the end of the Cold War, policy and expertise became less of a priority; collective memory leeched away as more and more people retired.

David Jarvis spent 36 years in the Royal Navy, largely focused on the nuclear programme. Towards the end of his career, he says, it was hard to identify the people who were going to replace his generation.

So he set up the Project on Nuclear Issues (PONI) to bring together the sixty-somethings and the twenty-somethings working in the nuclear sector today.

But it's one thing to get these people in a room - how do you provoke a real exchange of ideas and knowledge? This, after all, is a very hierarchical, intimidating field.

So PONI has used some fairly revolutionary tactics - staging plays and war game exercises to get those who have actually commanded nuclear submarines and their junior colleagues talking.

And Mr Jarvis has now pointedly handed over the reins of PONI to Andrea Berger, an expert on nuclear non-proliferation - who is 27.

Strikingly, the Ministry of Defence gave its blessing to Mr Jarvis's scheme, but no money; Mr Jarvis raised that himself. Now PONI is up and running, it has, he says, decided to provide some funding.

Corporate amnesia is an issue in the private sector too.

Margaret Heffernan, former chief executive of various internet businesses and author of Wilful Blindness, argues that a culture of casual contracts can have disastrous consequences. She cites the 2010 explosion at BP's Deepwater Horizon oil rig.

"What's really horrifying," she says, is that, when you look back five years earlier to another explosion at BP's Texas City refinery, "you find many of the same causes.

"There were exhaustive investigations as to its causes, [but] nobody did very much to encode the memory."

Very competitive organisations such as banks, she contends, must allow employees to discuss and learn from mistakes.

But there are signs that, like the public sector, business is addressing this issue.

David Nieper, a clothing company in Derbyshire, has attracted press attention for its policies to encourage older workers to pass on valuable expertise through phased retirement and mentoring.

In 2008, a rather different company, American Express, introduced a similar scheme. Even as far back as 2000, the Harvard Business Review was encouraging companies to employ a "chief memory officer".

And in the wake of the 2008 financial crash that so spooked the Treasury, politicians across the spectrum have begun to reassert the value of institutions and institutional memory.

But problems remain.

As chair of the Commons Public Accounts Committee, Dame Margaret Hodge listened to accounts of botched government-commissioned projects.

At a recent lecture for the Strand Group at King's College London, she criticised the tendency for government employees to stay in their jobs for relatively short spans - often as a result of internal rotation - and called for civil servants to see projects through.

And then there is the problem of preserving the institutional memory in the age of email and texts.

Last year an official review of government digital records concluded that existing systems had not worked well. Most departments, it said, "have a mass of digital data stored on shared drives that is poorly organised and indexed".

Historian Prof Sir David Cannadine, who was involved in the 2009 review of the "30-year rule" on the release of official papers, goes further.

Speaking at the British Academy in March, he said poor preservation of electronic records from the late 1980s to the early 2000s meant that often, when you looked for documents, "there's nothing there".

Analysis: Corporate Amnesia is broadcast on BBC Radio 4 at 20:30 GMT on Monday, 21 March. If you miss it, you can catch up later online.