What is an IPU (Infrastructure Processing Unit)?

Intel has released an IPU (Infrastructure Processing Unit) that, much like a SmartNIC, aims to enhance processing, networking, and storage by freeing up a computer's CPU.

In this blog, you'll learn what an IPU is, what it does, and why it's important. We'll also introduce you to an IPU that we are currently working with for a networking platform.

What is an IPU?

Released by Intel in 2021, an IPU, or infrastructure processing unit, is a programmable networking device designed to enable cloud and communication service providers to reduce overhead and free up performance for CPUs. 

Much like SmartNICs, IPUs aim to address complexities and inefficiencies in data centers that result from information overload. 

The primary goal of an IPU is enable customers to better utilize resources with a secure, programmable, and stable solution that enables them to balance processing and storage. 

How does an IPU work?

An IPU is a programmable network device that intelligently manages system-level infrastructure resources by securely accelerating those functions in a data center. 

It allows cloud operators to shift to a fully virtualized storage and network architecture while maintaining a high degree of performance, predictability, and control.

An IPU has dedicated functionality to accelerate modern applications that are built using a microservice-based architecture in the data center.

As a result, a cloud provider can securely manage infrastructure functions while enabling its customer to entirely control the functions of the CPU and system memory. 

What are the main features of an IPU?

There are four main features of an IPU: 

1. Highly intelligent infrastructure acceleration 
2. System-level security, control, and isolation
3. Common software frameworks 
4. Programmable hardware and software, built to the customer's needs

Why are IPUs important?

An IPU has the ability to:

1. Accelerate infrastructure functions, including storage virtualization, network virtualization, and security with dedicated protocol accelerators. 
2. Free up the CPU by shifting storage and network virtualization functions that were previously done in software on the CPU to the IPU. 
3. Improve data center utilization by allowing for flexible workload placement.
4. Enable cloud service providers to customize infrastructure function deployments at the speed of software. 

On a larger scale, evolving data centers will require a new intelligence architecture where large-scale distributed compute systems work together seamlessly connected as a single platform. 

This will help resolve today's challenges of stranded resources, congested data flow, and incompatible platform security. 

Within this new architecture, there will be three categories of compute: the CPU for general purpose computing, the XPU (cross-platform unit) for application-specific or workload-specific acceleration, and the IPU for infrastructure acceleration.

All three categories will be connected through programmable networks to efficiently utilize data center resources.

Let's look at an IPU that we are currently working with: the Intel IPU SoC (system-on-a-chip). 

Intel's IPU SoC

Currently, Trenton Systems is incorporating Intel's IPU SoC into a networking platform we are creating for government and military applications.

The Intel IPU SoC has three key features: 

1. Hyperscale ready: It is co-designed with a top cloud provider with integrated learnings from multiple generations of Intel FPGA (Field Programmable Gate Array) SmartNICs to deliver high performance under real workloads with security and isolation from the ground up.
2. Technology Innovation: It has a highly programmable packet processing engine, NVMe storage interface scaled up from Intel Optane Technology, next generation reliable transport, advanced crypto, and compression acceleration.
3. Software: It has Software/Hardware/Accel co-design, P4 usage Barefoot, leverage, and extended Data Plane Development Kit (DPDK) and Storage Performance Development Kit (SPDK). 

Final Thoughts 

An IPU is a critical component of a computing architecture which seeks to offload tasks from a host computer's processor to enhance networking, processing, and storage.

With infrastructure accelerations, system-level security, and customizable hardware and software, IPUs can help deliver safe, accelerated AI/ML/DL workloads and enhance big data analytics to deliver actionable insights in real-time. 

Through completely virtualizing critical capabilities, IPUs allow users and data center administrators to avoid overload through enhanced predictability, performance, and control.