Modern Data Center Processing Chips: The Foundation of Next Generation Data Centers

In June 2021, Intel announced its vision for the Infrastructure Processing Unit (IPU), describing it as follows: “The IPU is a new category of technologies and is one of the strategic pillars of our cloud strategy. It expands upon our SmartNIC capabilities and is designed to address the complexity and inefficiencies in the modern data center. At Intel, we are dedicated to creating solutions and innovating alongside our customer and partners — the IPU exemplifies this collaboration”. Intel took the time to stress that this innovation was a collaboration and stated that it was working with hyperscalers on this technology, but it stopped short of giving further details. Intel’s announcement came just weeks before Marvell announced their Octeon 10 Data Processing Units (DPU), within two months of NVIDIA’s announcement of a similar product, a new family of BlueField DPUs and the DOCA SDK that exposes a programmable Application Programming Interface (API) for use with them. It is worthy of note to mention that all three data center chips mentioned above are based on Arm technology.

In October 2021, Intel and Google jointly announced that they have collaborated to create a new category of chip for the cloud computing market, named the Mount Evans chip and described as an IPU. We can be forgiven for thinking that Google’s input at this stage facilitates the optimization of the chip for Google’s own purposes and exclusively for Google’s data centers. However, the announcement went on to say that they were collaborating on a set of software tools that will be released for free in hopes of making Intel’s version of the chip a broader industry standard, to be used beyond Google’s proprietary data centers. The intention is that the resulting chip, and Google influenced ecosystem, will be sold by Intel into all datacenters through existing sales channels.

With the October announcement, Intel was completing the introduction of this new technology and bringing the complete product, with developer ecosystem, to the market, but what problem are these devices solving?

The traditional data center was built for traditional workloads that ran through the Central Processing Unit (CPU), and to make those workloads run faster you either increased the number and clock speed of the CPU or looked to eliminate bottlenecks or inefficiencies that were causing the CPU to fall idle whilst waiting for other tasks to complete. An example of improving an inefficiency would be moving a database from disk into memory to make the read and write calls faster.

Modern, parallel workloads such as Artificial Intelligence (AI) engines and Machine Learning (ML) are best served by heterogenous compute, where hardware accelerators such as Graphical Processing Units (GPU) have workload assigned to them that they are optimized to work on, leaving the CPU to work on the general processing tasks that it has been designed to do. This means that all the processing required to complete a task is running on hardware that has been optimized to run it, which increases efficiency and reduces the burden on the CPU. These modern, distributed, and compute intensive workloads cannot be effectively served by increasing capacity as was the case in the traditional data center. To improve performance effectively for these workloads, they will benefit from being further split and presented to silicon that is optimized for the task at hand.

Add to this the virtualized nature cloud data centers, where tasks such as data transfer, storage management, or ensuring the end-to-end security of a transaction are currently abstracted to the hypervisor layer, are further burdening the CPU. The benefit to managing these tasks on dedicated silicon are clear.

With three marketable products all entering this space within a few months of each other it is clear where the industry thinks this technology is heading. It is good to see a healthy number of products in this space, as this will allow enterprise customers to choose which technology is best for their data centers. Often it is the ecosystem that allows the flexibility of these devices to be exploited, which is the differentiating factor, so Intel and Google’s announcement means that they understand the value of this, and they are putting in the groundwork to ensure they can compete.

More importantly, this announcement highlights the fact that Google will be using Intel IPU’s in its next generation data centers. Google is backing the Intel IPU and building a Google-centric ecosystem that it feels will give it a competitive edge as it tries to make ground on Amazon Web Services (AWS) and Microsoft in the hyperscaler competitive landscape. Equally, Intel is likely to be customizing its IPU with experience gained from the collaboration it has entered with Google. Intel now has a heavyweight partner with a real-life data center deployment that brings the data center processor concepts of distribution, orchestration, and infrastructure task-specialized silicon out of the High Performance Computing (HPC) data centers and into the cloud data centers, then beyond into the enterprise.

Whilst this is a commercial endeavor between Google and Intel and is not necessarily an endorsement of Intel’s technology being any better than Marvell or NVIDIA’s, what it does do is fix the data/infrastructure processing unit as a feature in enterprise data centers and provide a commercial proof of concept to anyone looking to design their next generation data center.