How better FPGA resource utilization can save you time and money

Hardware accelerators, like GPUs and FPGAs, offer much higher computing power compared to typical processors. This is the reason that more and more cloud providers offer FPGAs and GPUs as computing resources. However, they are among the most expensive devices in a datacenter, thus being very important to make sure that all of the resources in a system are being shared and as fully utilized as possible.

Abstraction layers on computing systems allow the seamless deployment and easy of programming. The same applies for the utilization of hardware resources where abstraction layers allow better resource utilization and lower cost of deployment. We watched this happen with the server virtualization hypervisors, and now it happens with Kubernetes container controllers too. This is one of the main reasons that cloud providers offers virtual machines instead of bare metal servers.

Especially in the domain of FPGAs, where virtualization is not possible, efficient resource management can lead to significant savings in terms of total execution time and overall opex.

Applications like deep learning, CPU, I/O, and network bottlenecks lead to poor utilization of the hardware accelerators. This is why a resource manager that can abstract away the FPGA resources is crucial for the efficient deployment of these applications. Until now there was not any efficient resource manager that offered scalable deployment and efficient resource management. This means that system administrators had to allocate manually specific FPGA devices to specific users. This process is tedious and can lead to low resource utilization.

This is why InAccel developed the Coral resource manager that abstracts away the FPGA resources and decouples the SW developers from the FPGA developers. The SW developer just invoke the function that they want to accelerate and the Coral manager takes care the resource allocation, the configuration of the devices and the workload distribution. Essentially, InAccel Coral functions as an abstraction layer for the workloads: it abstracts and streamlines access to the underlying hardware of FPGAs.

That means that inaccel Coral allows easy deployment, scaling and automatic resource management of the FPGA resources reducing significantly the cost of ownership and improving the performance of the applications. Especially for FPGAs that cost thousands of dollars, provisioning the right number of FPGAs for your applications is extremely challenging. With the resource manager, you can reduce the total number of FPGAs that are used and you can also share the available resources among several users or threads.