Why did Pope Benedict XVI resign?

Benedict XVI shocked the world in February when he became the first pope to resign in almost 600 years. But attention shifted quickly to the succession, and the election of the new Pope, Francis. Amid the drama, one question was never fully answered - why did Benedict quit?

Pope Benedict's official resignation statement offered his waning physical and mental powers as the explanation, but it's long been suspected there was more to it. And my inquiries have confirmed that.

I went to visit the Nigerian Cardinal, Francis Arinze at his apartment overlooking St Peter's. He's one of the most senior figures in the church and knows the Vatican like the back of his hand. He was even, for a short time in March of this year, mooted as a possible successor to Pope Benedict. And he was one of the select handful of senior church officials who were in the Pope's Apostolic Palace when he broke the news to them personally.

I raised the subject of the scandals that had preceded the Pope's bombshell decision and, in particular the Vatileaks affair in which the Pope's butler, Paolo Gabriele, had leaked confidential documents exposing Vatican power struggles. Could that have been a factor in his resignation? His answer was unexpected.

"It is legitimate for a person to speculate and say 'Maybe,' because some of his documents were taken secretly. It could be one of the reasons," he told me.

"Maybe he was so pained that his own butler leaked out so many letters that a journalist was able to write a book. It can be one of the reasons. I don't expect him to be enjoying that event."

In the Vatican, young ambitious members of the church are advised to "hear a lot, see everything and say nothing". That such a senior figure should essentially countenance a departure from the official line is significant.

Essentially, Pope Benedict was a teaching Pope, a theologian and intellectual. "His idea of hell would be to be sent on a one-week management training seminar," one insider told me. His misfortune was to accede to the papacy at a time that there was a power vacuum, in which a number of middle-ranking members of the Roman curia, the Church's civil service, had turned into "little Borgias" as another clerical official put it.

Don't take my word for it, this assessment comes from the highest source - the current leader of the Church. And Pope Francis does not mince his words. "The court is the leprosy of the papacy," he has said. He has described the curia as "narcissistic" and "self-referential". This is what Joseph Ratzinger had to deal with.

Over a period of time dating back to final years of Pope John Paul II, the heart of the HQ of the Roman Church had become dominated by infighting cliques. This was what the Pope's butler, Paolo Gabriele said he wanted to expose by photocopying and leaking all those documents.

But Gabriele said his relationship with Pope Benedict was like "father and son". So why did he act in a way that was sure to embarrass a man he was clearly close to?

"He said he had seen many ugly things inside the Vatican. At a certain point he couldn't take it any more," says his lawyer Cristiana Arru, clutching her rosary beads, in only her second ever public interview. "And so he looked for a way out. He says he saw lies being told. He thought that the Pope was being kept in the dark regarding key events."

Gabriele was found guilty of "aggravated theft" and spent three months in custody before being pardoned by the Pope. But that was not the end of it. The Church's leader set up an inquiry into the whole affair.

Three Cardinals produced a 300-page report. It was meant to be kept under lock and key, but a leading Italian daily claimed it had been briefed on its contents. The result? More embarrassing leaks, this time with claims of a network of gay priests exerting "inappropriate influence" inside the Vatican.

The headaches continued to mount for the German Pope. In many journalistic endeavours, "follow the money" is good advice for getting to grips with what is really going on, and it applies to the Vatican too. One of the most eyebrow-raising stories we encountered involved an annual Nativity scene in St Peter's Square.

For years, deals were struck in which the Vatican paid several times the market rate. When a whistleblower tried to reform the system, officials in the papal court persuaded a hapless Pope Benedict to promote him to a role 4,000 miles from Rome.

Similar antics occurred at the Vatican Bank, for years a source of unwelcome headlines for the Catholic Church. It was set up to help religious orders and foundations transfer much-needed money to far-flung parts of the world. But when a sizeable proportion of the transactions are in cash and are being sent to politically unstable parts of the planet, it does not take a genius to see what might go wrong.

It appears that bank officials took key decisions without always informing the Pope. When the board ousted its reforming president, Ettore Gotti Tedeschi (conveniently, on the day that the news of the Gabriele's arrest was getting saturation news coverage), the Pope did not find out until it was too late. He was "very surprised" in the later words of his private secretary. Gotti Tedeschi was an Opus Dei member and thought to be close to the Pope, but in the end this did not protect him.

Did all this prove too much for the ageing Pope Benedict?

Examine the precise words of the papal press spokesman, Father Federico Lombardi: "The Church needed someone with more physical and spiritual energy who would be able to overcome the problems and challenges of governing the church in this ever-changing modern world." Maybe that is as near as you are ever going to get from a senior official that the church had become ungovernable and needed someone else at the helm to stop the rot.

This is a church that now has a huge opportunity to move on and face up to the challenges of the 21st Century. Often seen as remote, its leadership is now canvassing the views of ordinary Catholics on hot-button issues such as contraception and gay marriage. Reform has come on the back of scandal. This is a development that has not gone unnoticed by Cardinal Arinze.

"What you have to remember," he says, "is that God often writes straight on crooked lines."

Mark Dowd presents The Report on BBC Radio 4 on Thursday 28 November at 20:00 GMT

Follow @BBCNewsMagazine on Twitter and on Facebook