Hyperscalers Face a New AI Hardware Dilemma: Deploy Off-the-Shelf AI Infrastructure or Invest in Custom Solutions

NVIDIA has dominated the boom in Artificial Intelligence (AI) data center capacity over the last few years, with revenue increasing by over 4X. This juggernaut has built its market leadership and Total Addressable Market (TAM) creation strategy through “off-the-shelf” AI systems with leading compute, networking, and the ability to “optimize” solutions through CUDA-X libraries. But internal and external pressures to diversify on the data center market, especially hyperscalers, mean that some are questioning if “off-the-shelf” products will be sufficient:

• Increasingly Homogenous Solutions: With nearly all AI data center vendors deploying NVIDIA hardware, competitors are struggling to differentiate their offerings based on value. Instead, the market is moving toward a price war, with competitors looking to undercut costs to differentiate.
• Local Power Constraints: Any data center is reliant on the local power grid. As hyperscale data centers grow, governments will increasingly become opposed to providing power capacity comparable to small cities. This will constrain the size and location of data centers developed.
• Rapidly Expanding Operational Costs: Power efficiency and resource utilization continue to struggle in data centers as systems are not optimized for specific workloads. This is creating cost efficiency problems within hyperscale facilities, which will rapidly spiral out of control as AI data center capacity continues to grow at a rapid rate.

Solving these challenges may require vendors to go deeper and provide customized AI solutions for the data center market. Marvell certainly thinks so and has built its data center strategy around customization. Targeting the hyperscale market, they have developed custom compute (Auxiliary Processing Unit (XPU), Central Processing Unit (CPU), Digital Processing Unit (DPU)), switching, and interconnect, which supports both scale-up and scale-out data center architectures. In addition, they are providing further customization targeting high bandwidth memory with Micron, Samsung, and SK Hynix.

This approach has enabled them to rapidly grow their data center business from US$0.5 billion to US$1.5 billion in 1 year. This rapid growth means that in the first 3 quarters of Full Year (FY) 2025, 71% of their revenue came from the data center segment. Partnerships with Amazon Web Services )(AWS on accelerated infrastructure and Meta on custom Network Interface Controllers (NICs) for the Open Compute Project (OCP) have been key drivers of this rapid growth. Marvell does not see this market slowing down, with ambitious revenue projections for FY 2026 of <US$2.5 billion. This expansion will be supported with 80% of its Research and Development (R&D) budget spent on this market.

Marvell claims that it is not directly competing with NVIDIA, but is instead working with NVIDIA to expand the data center TAM. But from the hyperscale perspective, it is important to consider the debate between custom and “off-the-shelf” hardware and understand the role each will play in the market. Table 1 explores the relative value of these different solutions.

Custom solutions offer greater efficiency, which will be highly prized in a capacity- and power-constrained market. However, the scale of demand for infrastructure from hyperscalers like AWS, Azure, and Google means that custom solutions will not always be suitable or even possible. This means that NVIDIA will remain dominant in terms of overall market share. But the allure of custom solutions means that Marvell’s position in the market will continue to expand (as it predicts). External constraints (e.g., local power grid) will drive demand for more efficient customized solutions. Meanwhile, the necessity to diversify away from a single sourcing framework and increasing need to differentiate from competitors will mean that NVIDIA’s hyperscaler customers will look for alternative solutions.

Given the size of the AI data center TAM, Marvell is in a great position to continue to grow its business. But this is not to say that it will not face challenges, some of which will be shared by competitors—supply chain fragility and slowing demand for training as vendors transition to inference—but some are specific to its custom approach to the market:

• Customer Constrained: The scale necessary to make custom AI systems at a reasonable cost per unit, and to demand foundry capacity within high-end TSMC fabs means that Marvell can only target “hyperscaler” customers with sufficient volume demand.
• Time to Market: AI data center capacity continues to grow rapidly, and hyperscalers want to move quickly to satiate demand for training and inference. However, building custom AI solutions takes between 1 and 3 years depending on the level of customization. Shipments of NVIDIA hardware does take time, but the lag will not be as noticeable.
• Operational Scale: Building custom hardware is time and talent consuming, as Marvell’s custom solutions team needs to work closely with its partners/customers across a 1 to 3-year time frame to build customized solutions. This places a constraint on the number of customers that Marvell can support and effectively manage. Scaling custom capacity will not be easy, requiring expensive new engineers.
• Lack of Software Capabilities: NVIDIA enables domain optimization through the CUDA-X libraries, while Marvell provides very limited software level support for its clients.
• Reliance on Customer for Deployment: Unlike off-the-shelf vendors like NVIDIA, Marvell does not have an ecosystem of Original Equipment Manufacturer (OEM) partners that provide the “server” and act as a channel to market. This means that Marvell is reliant on its own talent and the customer to build the specialized AI systems before deployment in the data center.

Some of these challenges are structural and will always be significant problems due to its custom strategy. Where Marvell should focus, though, is on software. NVIDIA has shown substantial domain-specific workload performance improvements by using CUDA-X Libraries to optimize hardware for each application. This should be the next focus for Marvell—providing software optimization capabilities to further enhance the performance of the custom AI systems.