Habana Labs’ Second Generation AI Processors Strengthens Intel’s AI Strategy

In May 2022, the Habana Labs team at Intel announced the launch of Habana Gaudi 2 and Habana Greco at the inaugural Intel Vision event. Habana Gaudi 2 is the company’s second-generation Artificial Intelligence (AI) training processor, while Habana Greco is the successor to Habana Goya, the company’s AI) inference processor. This is the first product update from Habana Labs since the first-generation Habana Gaudi and Habana Goya launched back in 2019.

Habana Gaudi 2 is built on the seven nanometers (nm) process. It has tripled the number of AI-customized Tensor Processor Cores from eight to twenty-four, tripled the in-package memory to 96 GB of High Bandwidth Memory 2e (HBM2e), and integrated a media processing engine for processing compressed media. The new processor also supports Floating Point 8 (FP8), an increasingly popular format due to its performance and energy efficiency.

Similar to Gaudi 2, Greco offers upgrades in both hardware and software. The new AI inference processor has five times more memory bandwidth, double the on-chip SRAM memory, and now supports new data formats, namely Brain Floating Point 16 (BFLOAT16), FP16, Integer 8 (INT8), and INT4.

Both products are designed to elevate the heavy cost of AI further computing in the hyperscale data centers. Habana Labs claims the Gaudi 2 can train popular Deep Learning (DL) models such as ResNet-50 and BERT much faster, while Greco has managed to half its power consumption for AI inference. As such, AI developers will benefit from much lower training and inference cost.

Both AI processors form a core component in Intel’s heterogenous portfolio. The growing demand for AI training and inferencing in the late 2010s has led to the emergence of new processors optimized for these specific workloads. Sensing the need to create a more heterogeneous product portfolio, Intel acquired Israel-based Habana Labs in December 2019.

Intel has positioned Habana Labs’ solutions as a processor option for AI-intensive workloads, alongside its general-purpose Xeon Central Processing Unit (CPU) and Field Programmable Gate Array (FPGA) products. Habana Labs’ products are designed to compete against top AI training and inference chipsets from the likes of NVIDIA and Xilinx. Aside from hardware advancement, Habana Labs also invest in software stack to reduce the barrier to adoption. All Habana Labs’ AI processors are built based on Synapse AI. It is a single software stack that supports TensorFlow and PyTorch frameworks and all the reference models, kernel libraries, containers, firmware, drivers, and tools. Habana Labs has also prepared pre-trained models for popular tasks such as image detection and recognition, natural language processing, and recommendation systems.

Since the acquisition, Habana Labs has made significant inroads into the hyperscalers and supercomputing market. As detailed in ABI Research’s insight on the commercial successes of Habana Labs (IN-6347), Habana Labs’ first generation Gaudi is currently available as Amazon EC2 DL1 instances or virtual server services. Furthermore, in April 2021, the San Diego Supercomputer Center at the University of California San Diego selected Gaudi as its AI compute chipset provider for its Voyager supercomputer. Other key clients include Mobileye and Leidos.

Based on these successes, the launch of Gaudi 2 and Greco will allow Intel to solidify its foothold in the competitive cloud AI processor market. However, Intel still has its work cut out for the company if it wants to expand its current reach.

Intel has developed various toolkits to ensure seamless AI development and deployment experience, including OpenVINO to optimize and deploy DL models onto all Intel hardware and oneAPI (Application Program Interface) to create resource-intensive applications that leverage all Intel processors. However, these toolkits are not yet supporting AI processors from Habana Labs. This means developers still need to use Synapse AI for their AI development. If Intel can extend the support from OpenVINO and oneAPI to Habana Labs’ AI processors, this would unify the developer community of OpenVINO and Synapse AI and bring much more of a significant influence to Intel.

It is also commonly known that most AI projects do not make past the initial experimental phase. AI technology vendors can undoubtedly do more to educate and assist the AI design and implementation market. They must be able to showcase and communicate the business value of AI to the businesses. Intel aims to address this through Project Apollo. Announced at Intel Vision, Intel partners with Accenture to provide enterprises with more than thirty open-source AI solutions kits designed for cloud and edge environments.