Can a computer really recruit the best staff?

It's a difficult question - what exactly is an organisation?

It is one on which huge amounts of time, money and energy has been spent, as experts grapple with the ideals of management.

The answers are complex - legal, structural, philosophical, cultural.

A company is an entity composed of hundreds - or thousands - of individuals. A beehive given a sense of purpose... by what?

The products or services it produces? The customers who depend up on it? The hierarchies it embraces? The ethos it follows? The legal personality it gives itself by being a limited company?

Well, recently I stumbled upon a fascinating way of thinking about what an organisation is, during the making of a BBC Radio 4 In Business programme about the way computer algorithms are beginning to loom large in the way organisations go about recruiting their staff.

Based in London, Bill Nowacki lives in a world of data. His title is managing director of decision science at the huge accountancy group KPMG. His work tries to apply analysing data - the explosion of information now called Big Data - to improving the way businesses work.

More and more people working in companies leave many traces of what they do behind, he says. These are data trails which can be harvested to improve the performance of individuals... and the organisation as a whole.

It sounds unremarkable, until you start to think about it. In the connected, digitised world of work, we now produce vast amounts of tiny, specific pieces of information about our daily, hourly, minute-by-minute behaviour.

How we use our computers and phones logs our corporate activity, what we say on social networks does the same thing. When we use a keycard to get from department to department, that's generating data about our movement across the workspace. Bill Nowaki calls all this data "artefacts".

And - he says - we have probably no idea how much information about ourselves we are leaving behind - all of it measurable data about what seem to be pretty trivial aspects of our work performance.

What time we start work, for example, is there somewhere on the system every time we log in... at work or at home or in the train?

Immediately the hackles rise - this is Big Brother, isn't it? Yes, it could be, says Mr Nowacki.

But it would also be a great aid to improving someone's performance... if an organisation uses the data of its most successful people to give tips and hints to the laggards.

As for recruitment, well the sites such as the business social network LinkedIn are now producing huge numbers of potential recruits for every job. To winnow out suitable candidates, data generated by traceable behaviour trends may become very useful indeed to create a shortlist of candidates from hundreds of applicants.

And this is when the concept of what the organisation really is becomes significant. Any established organisation will have a group of very successful people in it - employees who fit and perform outstandingly well.

They are already there, and every day they generate hundreds of bits of data about the way they go about performing so well - productive salespeople, for example.

So one way of recruiting is to use number-crunching computer power to assess the traits of the outstanding people a company already employs... and then shortlist potential new recruits by comparing them with established corporate high performers.

But doesn't that lead to companies hiring only people who most resemble what the company is like here and now? Doesn't the use of Big Data tend to drive out vital diversity?

Not necessarily, says Bill Nowacki, because of the subtlety of the analysis process. At KPMG they've built a model which incorporates 10,000 different data points generated by a single individual. That's millions of bits of data about a group of individuals in a big firm.

Number crunch those intelligently, and important signals may emerge. Bill Nowaki calls that "training" the algorithm by reviewing the data generated by previous recruits and comparing that with the current results - who stayed, who was promoted, who performed well.

You see what's emerging here? A new complex model of an organisation viewed through the Big Data prism that the people who work for it generate every day. The very practical aspects of their working life, obviously, but also the relationships and interests they mention in their social networking.

It is as though we are becoming able to peer into the inner working of the business beehive... and contemplate (and perhaps begin to understand) the dances of the worker bees deep in the heart of the organisation.

If the algorithms are so good at eliminating the uncertainties of recruitment, it's conceivable they ought to be applied up the corporate hierarchy... to help determine who the leaders ought to be, based on the key performance indicators data they have generated all through their onwards and upwards career.

All this may lead old-fashioned people to point out that one thing seems to be missing in this new model of the organisation - the subtle human skills and characteristics that may not show up on any of those data points. The things that used to be called interpersonal skills.

Well in fact this personal interaction seems to cause real trouble when it comes to recruiting.

Study after study demonstrates a huge bias in the recruiting process... even in organisations which say they are committed to eliminating discrimination. White middle-aged men have a tendency to hire other white middle-aged men, whatever they intend. Robotised recruitment is blind to that sort of human influence.

And if these algorithms really can "learn" from experience, then maybe they will come to be able to scan personality traits, just as they can already assess how the measurable behaviour of candidates compares to that of already successful members of the team.

It looks as if Big Data is beginning to generate a whole new map of an organisation, and (probably) whole new theories about how an organisation works.

As it happens, I also have a little organisation theory of my own. It goes like this. Many businesses have an organisation chart - a sort of map of how things work, who reports to who.

This reporting seems to be the defining characteristic of how something works - management status is defined by how many "reports" someone has (that is, how many workers come under his or her administration). And by how close the manager is to the perceived centre of power... how many levels between him or her and the big boss.

But there's another organisation map, quite a different one from the power hierarchy the bosses live and die by.

Just ask the people who work in the power pyramid who they need to get their job done. This simple question will produce an organisational map of relationships utterly different from the way the top people think their business operates.

It reveals just how distant are the top people from the workers... and how remote is the leadership those top people assume is making an impact.

The difference is a revealing one. Like the 10,000 data points, it describes an organisation not as it ought to be, but as it really is. For better or for worse, this is the real workplace, not the one the top people think they have created.

An accompanying radio report on this story is available to listen to via podcast. It will also be aired on the BBC Radio 4's In Business programme on Sunday, 8 May at 21:30 BST.