FPGA Vendors Charting a Path to Find Success in Machine Learning

In the past, Field-Programmable Gate Array (FPGA) vendors have been aggressively pushing into Artificial Intelligence (AI) and Machine Learning (ML). The FPGA partnership between Microsoft and Intel demonstrated in 2018 that FPGA is capable of handling large AI inference workloads. The FPGA-fueled architecture provides ultra-high throughput that can run ResNet 50, an industry-standard deep neural network for object classification, without batching, meaning no tradeoff between high performance and low cost. This is due to FPGA’s strengths in flexible hardware. AI developers can fully customize FPGA to their needs, adding custom data paths, bit-widths, memory hierarchies, and bit manipulations. This means FPGA is fully adaptive to evolving AI algorithms.

Despite all of this, the popularity of FPGA in the ML space is definitely not on par with Graphics Processing Unit (GPU) and certain Application-Specific Integrated Circuit (ASIC) products. A key obstacle for this is the programming language preference of AI software developers. Popular ML toolkits such as cuDNN, OpenVINO, and OpenCL generally support high abstraction programming languages, such as Python and R, that are staples among data scientists and programmers these days, but programming FPGA requires developers to understand very low-level abstraction programming languages such as Verilog and VHDL. The mastery of hardware description languages is not very common among software developers.

In addition, the AI ecosystem around FPGA is still less vibrant as compared to that of Central Processing Unit (CPU) and GPU. NVIDIA and Intel, for example, have been actively promoting and acquiring startups that offer AI toolkits based on their chipsets. Such an ecosystem is critical to build a developer community around their products and services.

Understanding these challenges, various FPGA vendors are starting to offer End-to-End (E2E) full stack toolkits for AI developers that support familiar AI frameworks, such as TensorFlow and OpenCL, and high-level programming languages, such as C++ and Python. Lattice Semiconductor, a FPGA company focusing on ultra-low powered machine learning use cases, features Lattice sensAI stack, which has a neural network accelerator and compiler, model libraries, and reference designs that facilitate AI model development and deployment. Meanwhile, high performance FPGA vendor Xilinx launched Vitis AI in October 2019, featuring open-source Xilinx runtime library, the AI Engine in Versal ACAP, and core development tools such as compilers, analyzers, and debuggers, as well as AXI Interfaces and DMA Engine to communicate with CPU or GPU.

Beyond in-house software toolkits, Xilinx has been actively building an ecosystem of partners that build FPGA-based software. They can be divided into several categories:

• High Performance Networking: Algo-Logic System, Bigstream, Maxeler Technologies, Titan IC (acquired by Mellanox in March 2020)
• Big Data Analytics and Data Warehouse: BlackLynx, SumUp Analytics, Vitesse Data, Xelera Technologies
• Runtime and Compiler: CTAccel, Falcon Computing, Mipsology
• Video Processing and Codec: NGCodec, Skreens, VYUSync

In order to attract more developers, partners focusing on runtime and compiler are critical to Xilinx. France-based Mipsology is one of these startups. Its flagship software platform, Zebra, accelerates inference workloads for neural networks on FPGA. Originally designed to support only Xilinx, the startup has started to extend its support for Achronix and has plans to support Intel’s FPGA lineup in the future. Zebra supports all major AI frameworks, including TensorFlow, Caffe, Caffe2, and MXNet. Miposology claims that Zebra users don’t have to learn new languages, new frameworks, or new tools in order to use Zebra, thereby speeding up the deployment rate.

In short, Mipsology is aiming to serve the FPGA community the same way NVIDIA’s cuDNN and Intel’s OpenVINO do, being the go-to AI development platform for FPGA, facilitating ease of use and lowering the barrier to entry for AI developers. According to ABI Research’s Artificial Intelligence and Machine Learning (MD-AIML-105) Market Data, the total revenue of FPGA vendors in AI is expected to grow from US$655 million in 2019 to US$1.5 billion in 2025. No doubt the figure is dwarfed in comparison to GPU, but this is a significant market that witnesses FPGA vendors playing active role in both cloud and edge AI. As shown by the aforementioned example of Xilinx and Lattice Semiconductor, ease of use and ecosystem building are key priorities for all FPGA vendors, as these will enable them to get more developers on board.

In addition, it is important for FPGA vendors to craft their own unique set of messaging for ML. Instead of going head-to-head with market leaders like NVIIDIA and Intel, which have established strong developer communities around their products, FPGA vendors should emphasize the benefits of having different tools for different applications. Identifying use cases that can be uniquely served by FPGA, such as in wireless communications, oil and gas, and medical and industrial imaging, would enable FPGA vendors to better demonstrate their capabilities and flexibility. This is also something that Xilinx and Lattice Semiconductor have done, to a certain degree of success, by focusing on medical imaging and industrial machine vision for example.