A tour of the Intel Sapphire Rapids QCT System Overload Cabin


QCT QuantaGrid D54Q 2U Liquid Cooled Intel Innovation 2022 2

At Intel Innovation 2022, QCT showcased its new 4th Gen Intel Xeon Scalable servers (codenamed Sapphire Rapids.) Some vendors at the show brought a machine or two. Others just showed product videos. QCT, on the other hand, had a huge range of next-gen systems, so I wanted to show off some of the interesting designs on the stand.

A tour of the Intel Sapphire Rapids QCT System Overload Cabin

The first eye-catching solution was the QCT 42U liquid cooling rack solution. At the show, the company was using the QCT D54Q-2U in liquid-cooled and air-cooled configurations.

2022 Intel Innovation QCT Liquid Cooling Solution 4
2022 Intel Innovation QCT Liquid Cooling Solution 4

The servers are liquid-cooled via direct-on-chip cold plates, and then the rear door heat exchanger is used to more efficiently remove heat from the systems. I ended up taking over 100 photos from this device, so we’ll be doing a deeper dive in the next week. We focused on liquid cooling last year, more on STH, so this was an awesome demo.

2022 Intel Innovation QCT Liquid Cooling Solution 19
2022 Intel Innovation QCT Liquid Cooling Solution 19

QCT also opened the liquid-cooled server in its booth. You can see the blocks here on each CPU. There are then hot and cold tubes that go to the back of the chassis and then connect to the rest of the rack.

QCT QuantaGrid D54Q 2U Liquid Cooled Intel Innovation 2022 2
QCT QuantaGrid D54Q 2U Liquid Cooled Intel Innovation 2022 2

In these servers, QCT always uses air to cool memory, storage and networking. It looks like high power accelerators can be added to liquid cooling loops if needed. The net impact is that the fans run at a much lower RPM and therefore use much less power than if they were running at full speed.

QCT QuantaGrid D54Q 2U Liquid Cooled Intel Innovation 2022 3
QCT QuantaGrid D54Q 2U Liquid Cooled Intel Innovation 2022 3

Here is the air shroud in the QuantaGrid D54Q-2U air-cooled QCT.

QCT QuantaGrid D54Q 2U Intel Innovation 2022 3
QCT QuantaGrid D54Q 2U Intel Innovation 2022 3

It was just interesting to see an air-vs-liquid-cooled solution on the show floor, especially with the next-gen platforms.

For the 1U market, QCT had a dense supply. The QCT QuantaGrid D54X-1U is the 1U server offering with sixteen DIMMs per CPU.

QCT QuantaGrid D54X 1U Intel Innovation 2022 6
QCT QuantaGrid D54X 1U Intel Innovation 2022 6

One interesting aspect of the company’s new motherboards is the rear I/O layout. QCT has two OCP NIC 3.0 slots, one on each side. In the middle there is another module which has the BMC and the rear I/O.

QCT QuantaGrid D54X 1U Intel Innovation 2022 4
QCT QuantaGrid D54X 1U Intel Innovation 2022 4

Above the OCP network card and rear I/O there are two full-height risers. We were told that other configurations were also available.

QCT QuantaGrid D54X 1U Intel Innovation 2022 2
QCT QuantaGrid D54X 1U Intel Innovation 2022 2

What is clear, however, is that the system is packed. If you are using Skylake or Cascade Lake servers, this build will be a >2:1 consolidation ratio opportunity on those servers.

On the topic of 1U servers, we have the QCT QuantaGrid D54T-1U. This is a platform that many of our readers have seen previous versions of and many have also used. This time, however, there is a big change.

QCT QuantaGrid D54T 1U Intel Innovation 2022 4
QCT QuantaGrid D54T 1U Intel Innovation 2022 4

The front of the system has a sliding tray (more on that shortly) and the cool QCT feature of having space for four cache SSDs below.

On the back of the system we can see slots for expansion cards, but also room for things like OCP NIC 3.0 cards.

QCT QuantaGrid D54T 1U Intel Innovation 2022 1
QCT QuantaGrid D54T 1U Intel Innovation 2022 1

Looking at the tray, we can see twelve 3.5″ hard drive slots. Each tray has a cool locking mechanism and is similar to previous generations of this chassis.

QCT QuantaGrid D54T 1U Intel Innovation 2022 5
QCT QuantaGrid D54T 1U Intel Innovation 2022 5

What is different is the CPU. With the Sapphire Rapids Xeon, this platform can use a single CPU instead of requiring two CPUs just for the extra PCIe lanes. That’s a huge cost saving for a server of this class.

Next to the 1U storage server was an interesting 3U system, the QCT QuantaGrid D54U-3U. It is a GPU and accelerator platform. Here we can see the 10 2.5-inch hot-swap bays and then the hot-swappable front fans.

QCT QuantaGrid D54U 3U Intel Innovation 2022 2
QCT QuantaGrid D54U 3U Intel Innovation 2022 2

The rear can accommodate up to four power supplies, an OCP NIC 3.0, and rear I/O. There is more though.

QCT QuantaGrid D54U 3U Intel Innovation 2022 4
QCT QuantaGrid D54U 3U Intel Innovation 2022 4

Each side of the frame has two risers. These lifts use quick coupler connectors for entry and exit. There are even small cutouts in the chassis and riser cages to support bridges, for example, NVLink bridges for GPUs.

QCT QuantaGrid D54U 3U Intel Innovation 2022 3
QCT QuantaGrid D54U 3U Intel Innovation 2022 3

The middle of this system is for networking and other I/O. QCT told us that this generation’s 3U design should provide more efficient cooling. With 4 x 350W PCIe GPUs, 2x more powerful Xeon Scalable processors, and all the other system components, the decision to go with 3U was for efficiency. Additionally, even using these 3U servers, one can easily exceed 30 kW of compute in a rack, so QCT has extended the chassis size from 2U to 3U for better efficiency.

For high-end GPU and AI accelerators, there was a whopping 7U account box. where 5U was dedicated solely to cooling.

QCT QuantaGrid D74U 7U Intel Innovation 2022 2
QCT QuantaGrid D74U 7U Intel Innovation 2022 2

Here is the back of the device. If you see there is a 4U top section, then three separate 1U portions. This top 4U section is designed for things like the HGX H100 baseboard. You can think of this as the QCT version of the NVIDIA DGX H100. We don’t show this plinth because inside was a prototype OAM platform that one would use for accelerators like the Ponte Vecchio or Gaudi 2. QCT’s design accommodates different types of accelerators, although that the H100 seems to be the main configuration.

QCT QuantaGrid D74U 7U Intel Innovation 2022 3
QCT QuantaGrid D74U 7U Intel Innovation 2022 3

Below the GPU/accelerator tray is the motherboard. Here we get two Sapphire Rapids sockets each with 16 DIMMs. This motherboard plugs into the PCIe switch backplane and is removable for service.

QCT QuantaGrid D74U 7U Intel Innovation 2022 4
QCT QuantaGrid D74U 7U Intel Innovation 2022 4

Below, at the rear of the system, we can see 1U of white space. The first is a set of up to 10 OCP NIC 3.0 slots. The bottom 1U of this system is for DC power supplies.

Last words

At Intel Innovation 2022, vendors brought many next-generation Intel Xeon platforms. QCT, however, stood out with the number of different types of systems the company had on display. I thought this would be a good opportunity to show off some of the new systems, including standard 1U/2U servers, liquid-cooled servers, a 1U storage server, and GPU servers. If you want to know more about the liquid-cooled rack solution, we should have this one in about a week.

4th Generation Intel Xeon Scalable processors are still a long way off. At least Intel let vendors show off some platforms and share them with us at Innovation 2022. We’ll have an accompanying video soon with more quick views of these and other servers.

Previous iPhone proves fatal for Delhi boy who borrowed money to buy it
Next The debt repayment app that helps you pay off your mortgage faster