UPDATED 18:33 EST / NOVEMBER 02 2026
Rack-scale AI: How Dell and AMD are building infrastructure
by Mark Albertson
Dell Technologies Inc. and its partner Advanced Micro Devices Inc. face similar competitive pressures. Both tech powerhouses operate in a business climate where success is now measured across central processing units, graphics processing units, rack-scale systems, memory capacity, network bandwidth and tokens per dollar.
The pressure to compete across those measures underscores what theCUBE Research has termed “Engineering Velocity.” Since the Dell Technologies World event in May, AMD and Dell have not slowed their pace of collaborations and announcements.
AMD and Dell are working to build enterprise AI infrastructure that can keep pace with changing workloads and customer requirements.
“In AI infrastructure, the winner may not be the company with the best benchmark on keynote day,” said theCUBE Research’s Dave Vellante. “It will more likely be the company whose engineering system learns, validates and improves the entire platform faster than anyone else.”
This feature is part of SiliconANGLE Media’s exploration of the architectural shifts powering continuous, production-grade AI. Be sure to check out theCUBE’s “Dell AI Data Platform Event: From Ambition to AI at Scale” and its past coverage of this year’s Dell Technologies World. (* Disclosure below.)
Rack-scale consolidation
A cornerstone of the Dell and AMD relationship is the server giant’s PowerEdge portfolio. Dell expanded its PowerEdge server line in May and has followed that up with additional details surrounding the integration of AMD processors.
The two companies have pursued a strategy designed to facilitate server consolidation as agentic AI drives a surge in demand for general-purpose compute. Agents rely heavily on CPUs, yet data centers don’t have a lot of extra room to add servers in rooms already stacked with GPUs.
Systems powered by AMD processors will deliver 70% gen-over-gen performance gains and can fold 13 servers from Dell’s 14th PowerEdge generation into one, according to Pradyumna Narendra, product planner and senior consultant at Dell.
“For every 10 to 13 servers you replace, you get so much of that productivity gain back because now you can do that with one server,” he told theCUBE. “When you expand that across data centers, that’s like an accelerating effect that you get to capitalize from.”
Dell and AMD must also accommodate a wealth of AI use cases and the accompanying demand for compute. Dell’s three-rack-unit server requires less power for cooling than a two-unit equivalent, allowing customers to pack more computing power into each rack, according to Narendra.
“By deploying servers that are slightly larger in package, you deliver more density at a rack level, which seems sometimes counterintuitive,” he explained. “As AI accelerates, inferencing and agentic AI around it will accelerate and grow faster. As these use cases scale out, you get exponential growth in the need for compute.”
Modular deployment and validation
Beyond server consolidation, Dell and AMD have been steadily enhancing a modular solution to simplify AI deployment and scaling: the Dell AI Platform with AMD.
The on-premises enterprise AI solution integrates AMD Instinct accelerators and EPYC processors with Dell PowerEdge servers, PowerSwitch networking and PowerScale storage for model training, fine-tuning and high-throughput inference. The platform is designed to let customers begin with configurations that meet current workloads and expand compute and GPU density over time, without rearchitecting.
“What we’re finding is customers want modular solutions. They want to think about [solutions] across the whole platform,” said Varun Chhabra, senior vice president of product marketing and infrastructure solutions group at Dell, in a discussion with theCUBE. “Compute, storage, networking [and] GPU. The software framework on top of it, the models: Have they all been tested? Have they all been validated?”
Dell has integrated the AMD ROCm software stack into the platform to support training, fine-tuning and serving large language models on-premises. The modular platform lets customers begin with smaller deployments and expand on the same infrastructure, according to Chhabra.
“You can start small with these composable units of storage, compute, networking, pre-integrated with AMD Instinct and EPYC CPUs with the ROCm software as well as the open ecosystem that sits on top of it,” he said. “We test it, we validate it, and the customers can use it and once they start seeing value in it, they know they can scale in a modular way with the same frameworks.”
Leveraging benefits of PCIe
Dell announced support for AMD Instinct MI350P PCI Express GPUs in its PowerEdge XE7745 and R7725 servers beginning in July, giving enterprises another option for scaling agentic and generative AI deployments. It also represents an important boost in memory bandwidth which matters in time-to-first-token latency.
The MI350P offers 144 gigabytes of HBM3E memory, which can be a key differentiator for inference performance, according to Mike Darby, senior manager of business development, Instinct Data Center GPUs, at AMD, during an appearance on theCUBE.
“It’s a lot of memory, but it’s also HBM3E, so [it has] an amazing amount of bandwidth,” he said. “The main use case being inference: It’s all about that latency, time to first token [and] the user experience, so that your attention span doesn’t move somewhere else while you’re waiting for your response.”
AMD and Dell’s collaboration on PCIe integration has another angle worth watching. AMD’s PCIe-based GPU supports air-cooled deployments, providing users with a path to deploy AI within existing data center infrastructure.
As the tech industry has encountered resistance in recent months to the buildout of large data centers across the U.S., concerns have centered on the environmental impact and use of water-based cooling systems. Separately, Dell found that, as it deployed its next-generation PowerEdge servers, some enterprise customers were still not prepared to adopt liquid cooling within data centers.
“Working with Dell, we added a PCI card to our air-coolable roadmap for GPU Instinct so that it fit in more Dell PowerEdge servers that go to the broader enterprise,” said Robert Hormuth, corporate vice president of architecture and strategy for the Data Center Solutions Group at AMD, during a conversation with theCUBE. “Not everybody is adopting direct liquid cooling.”
Collaboration in an open market
Dell’s work with AMD this year has extended beyond joint product announcements. In June, the two companies announced a partnership with the University of Cambridge to establish a Sovereign AI Innovation Lab (SAIL) in the United Kingdom.
The collaboration is designed to help advance the next generation of AI infrastructure and AI-powered scientific breakthroughs. It builds on Cambridge’s expanded AI research infrastructure, including the Zenith AI supercomputer, powered by fifth-generation AMD EPYC processors and AMD Instinct MI355X GPU accelerators integrated into Dell infrastructure.
Dell and AMD have also collaborated in recent months on a telecom initiative with AT&T Inc. Telecom data is a heavy lift when it comes to AI, with AT&T having processed over one trillion tokens to train models. AMD has provided the high-performance hardware foundation for both training and inference workloads involving the OTel 2.0 models developed through the Open Telco AI initiative.
Dell integrated this technology with AI-optimized servers for telecom environments to create a production-grade foundation for operators across data-intensive, real-world network environments. OTel 2.0’s training infrastructure uses Dell’s PowerEdge XE9785 server configured with eight MI355X GPUs.
The alliance between Dell and AMD highlights a key development in the growing AI infrastructure market: There are plenty of opportunities for competition. Both Dell and AMD appear committed to building portfolios that offer enterprise choices in AI processing solutions.
“AMD didn’t present its new products … as ‘respectable alternatives,’” Vellante said. “It claimed outright leadership across CPUs, GPUs, rack-scale performance, memory capacity, network bandwidth and tokens per dollar. It extended the argument across enterprise AI, robotics, ultra-low-latency inference and a rapidly expanding partner ecosystem.”
(* Disclosure: TheCUBE is a paid media partner for the Dell AI Data Platform Event. Neither Dell, the sponsor of theCUBE’s event coverage, nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)
Support our mission to keep content open and free by engaging with theCUBE community. Join theCUBE’s Alumni Trust Network, where technology leaders connect, share intelligence and create opportunities.
15M+ viewers of theCUBE videos, powering conversations across AI, cloud, cybersecurity and more
11.4k+ theCUBE alumni — Connect with more than 11,400 tech and business leaders shaping the future through a unique trusted-based network
Are you an AWS customer? Support SiliconANGLE financially by buying your AWS services from our Marketplace portal page and links: https://siliconangle.com/aws-marketplace/
Founded by tech visionaries John Furrier and Dave Vellante, SiliconANGLE Media has built a dynamic ecosystem of industry-leading digital media brands that reach 15+ million elite tech professionals. Our new proprietary theCUBE AI Video Cloud is breaking ground in audience interaction, leveraging theCUBEai.com neural network to help technology companies make data-driven decisions and stay at the forefront of industry conversations.
Anthropic’s quarterly revenue jumps to $17B, losses grow ahead of IPO
42AI launches with $62.5M to predict how traders act when markets turn volatile
Palantir beats Q3 forecasts as US commercial revenue overtakes government for first time
Marvell to acquire optical component maker Lumentum in massive $115B deal
On theCUBE Pod: Stock market gets the jitters and Anthropic files IPO
Google finds 'LLMjacking' attackers running up $100K a day in fraudulent cloud charges
Anthropic’s quarterly revenue jumps to $17B, losses grow ahead of IPOAI - BY MARIA DEUTSCHER . 14 MINUTES AGO
42AI launches with $62.5M to predict how traders act when markets turn volatileAI - BY DUNCAN RILEY . 46 MINUTES AGO
Palantir beats Q3 forecasts as US commercial revenue overtakes government for first timeBIG DATA - BY DUNCAN RILEY . 1 HOUR AGO
Marvell to acquire optical component maker Lumentum in massive $115B dealINFRA - BY MARIA DEUTSCHER . 2 HOURS AGO
On theCUBE Pod: Stock market gets the jitters and Anthropic files IPOAI - BY DEVONY HOF . 7 HOURS AGO
Google finds 'LLMjacking' attackers running up $100K a day in fraudulent cloud chargesSECURITY - BY DUNCAN RILEY . 9 HOURS AGO