BIDI: The BBC Internet Distribution Infrastructure explained

In his previous blog post, Product Manager Federico Benedetto described how BBC on demand content reaches our audiences. In particular through Radix, the BBC system that delivers media to the Content Distribution Networks (CDNs), which are in turn responsible for distributing it to the final users across the UK and internationally. Federico now turns to another piece of the puzzle in the BBC's media distribution stack: the “BBC Internet Distribution Infrastructure”, or BIDI for short.

BIDI is the BBC’s own content distribution platform and is responsible for delivering media directly to the BBC end users in the UK. As depicted in the diagram below, the clients are always presented with a ‘CDN mix’ from which they can fetch the media they are interested in. BIDI is part of this mix, alongside the commercial CDNs.

This set-up allows us to provide a better overall quality of service, by enabling clients to move between different media distribution platforms if they encounter problems when streaming from any particular one.

BIDI consists of several distributed HTTP caches, referred to as Caching Cells and grouped in ‘Batteries’. The content is served directly from these to the end users.

In addition, there is a Control Plane that manages the fleet of physical caches, and a Matchmaker service, which steers the incoming traffic to the best available point of presence. Both these applications are hosted in the cloud.

The BIDI caches are based on the same software stack as Radix. In particular they are based on a distribution of the Nginx web server called OpenResty. We also use ExaBGP for network related tasks. For more information on why we are using Nginx, I suggest reading Alistair Wooldrige's blog post. If you are interested in knowing more about how BIDI works, there will be more specific posts from the OTG Media Distribution team in the coming months.

BIDI is currently in use on selected iPlayer platforms only, but chances are that if you are using iPlayer on iOS devices or in your desktop’s browser, you may have already been using it without realising.

Why BIDI?

The BBC owns and operates a complex network and server infrastructure, which is crucial to the functioning of all the BBC online services.

We have also been an active member of the London Internet Exchange (“LINX”) for many years, and we have a long history of working with Internet technologies.

For an overview of the Internet history at the BBC, I highly recommend reading this blog post from our Chief Scientist, and explore this iWonder guide on the 20 years of the BBC on the web.

BIDI builds upon this rich heritage, and marks an important milestone for the development of an Internet first BBC. By building and owning the infrastructure that allows us to directly reach our audiences, we are making sure that we are preparing for a future when distribution over the Internet will become an even more essential part of media delivery than it is today.

BIDI also allows us to work in partnership with the UK Internet Service Providers community and interconnect with them in locations other than London. By doing so we hope to contribute to the development of a regional and resilient architecture of the British Internet. BIDI is in fact delivering traffic via London based peering, but we also have a presence at IX Manchester. Moreover, BIDI servers can also be deployed within 3rd party networks, to reduce potential traffic congestion over specific network paths.

Quality of service is another important driver for our efforts. By managing our Internet streaming technologies and controlling the distribution costs, we can easily monitor the performance of our streams, take immediate action when we notice a problem, and enable BBC products to access higher bitrates and better quality streams. Deploying our servers near the edges of networks (i.e. closer to the end users) also improves quality by reducing the network hops. Therefore packet round-trip time (RTT) will also be reduced.

By relying on a mix of commercial CDNs and our decentralised infrastructure we can provide useful mitigations when dealing with problems over specific network paths or suppliers, and we can offer a better streaming performance to our audiences and highly available media endpoints.

From R&D to production

BIDI is a great example of a successful transfer of technology from research to production.

Initially conceived jointly by BBC Research and Development (R&D) and the BBC Online Technology Group (OTG), the first functioning prototype was developed by R&D. The development of the BIDI prototype was a fundamental step which demonstrated that the concept of building our own Internet distribution platform was a feasible and valuable goal.

The prototype was further developed by R&D to a point that it was ready to be used for production traffic. A live trial was therefore launched, with the goal of learning as much as possible from serving traffic to a very small set of real users. By doing so, the model was finally proved outside of a lab, on real networks, with real users and content.

Following the success of the BIDI trial, the OTG Media Distribution team, initially supported by the R&D team, took over the development of BIDI. Over the last year, we have managed to increase the BIDI traffic share and capacity, while transforming BIDI from a prototype to a fully supported, production quality platform.

Today BIDI runs on a very different software stack from the original R&D prototype, but the key concepts and ideas haven't changed.

The fruitful collaboration between production and research is still ongoing. Lessons learnt from operating BIDI at scale are now feeding into the R&D roadmap, which itself is experimenting with new concepts and technologies that will pave the way for future BIDI developments.

What's next for BIDI?

Despite being a fully supported platform, BIDI is still at the early stages of its development. We are constantly building new features, improving the algorithms behind the Control Plane decisions and tuning the performance and throughput that we can achieve with our servers.

We are not planning on distributing all the BBC traffic via BIDI, as using a mix of CDNs helps us achieve our quality of service goals. Despite this, there is a vast growth potential for BIDI which we will soon tap into, by increasing our network capacity, our partnerships with the ISP community and the range of supported clients and protocols.

In particular, we are currently working on supporting HTTPS distribution, as mentioned by Paul Tweedy in his blog post.

In the near future we will also add support for MPEG DASH streams, so that we will be able to deliver media to the new HTML5 player and we will soon be supporting iPlayer on TV devices and set top boxes.

Want to find out more?

If you like the sound of what I have just described and you would like to play a part in it you can apply here. If you would like to know more about BIDI, feel free to write to us at bidi@bbc.co.uk or get in touch with the OTG Media Distribution group engineering manager Umesh Telang.