OpenPOWER Summit Europe 2019

Modern big data systems are highly heterogeneous. Components are implemented in a wide variety of programming languages and frameworks. Due to implementation differences, interfaces between components are burdened by serialization overhead. The Apache Arrow project helps to overcome this burden through a language-agnostic columnar in-memory format for big data applications. It is currently being integrated in many big data analytics frameworks, such as Apache Spark & Parquet, Dask, Pandas, and others.
In this talk, we present the open-source Fletcher framework, which is an implementation of Arrow for FPGA accelerators. Through a design generation step, Fletcher takes Arrow data structures and generates specialized, high-performance and easy-to-use hardware interfaces that can connect to accelerator kernels. This allows for automating the design of data interfacing and management. In addition, using Arrow datatypes prevents serialization overhead, and provides out-of-the-box integration with over 11 high-level languages efficiently.