FPGA setup and building - Revision history

Rnane at 08:27, 30 September 2022

2022-09-30T08:27:22Z

Rnane at 12:46, 27 September 2022

2022-09-27T12:46:50Z

Gmarchiori at 20:36, 30 May 2021

2021-05-30T20:36:53Z

Gmarchiori: Removed unnecessary comment in command line.

2021-01-22T14:15:36Z

Removed unnecessary comment in command line.

Gmarchiori: Added link to paper.

2021-01-22T10:44:11Z

Added link to paper.

Gmarchiori: Changed format of the git clone command.

2021-01-21T18:03:40Z

Changed format of the git clone command.

Gmarchiori: First version of the page completed.

2021-01-21T17:47:38Z

First version of the page completed.

Gmarchiori: Initial page, mostly a placeholder.

2021-01-20T18:20:27Z

Initial page, mostly a placeholder.

New page

This page contains instructions on how to setup, compile and use the FPGA ''bitstream'' required to run the Flow simulator when compiled with the fpgaSolver enabled (ILU0-BiCGSTAB accelerated solver).

PREREQUISITES
-------------

DOWNLOADING
-----------

BUILDING
--------

USING THE BITSTREAM
-------------------

REPORTING ISSUES
----------------

Issues specific to the FPGA ''bitstream'' compilation process can be reported in the git issue tracker at:

http://github.com/OPM/FPGA/issues

@@ Line 25: / Line 25: @@
 In principle, the shell version can be changed to a different one (typically a newer one), but the bitstream generation process may fail or generate sub-performing bitstreams (e.g. in case the shell allocates the static resources differently than the one used currently), and changes may be required to the makefiles and other code in the FPGA repository.
-'''Warning''': Please note that the last opm-simulators master branch tested that worked with the FPGA accelerator was from 2021-1-15 09:00 ending with the hash code 57d158b. opm-simulators releases & the master branch after that point broke the FPGA API due to incompatibilities with the GPU code in how the data is encoded and transferred to the accelerator.
+'''Warning''': Please note that the last opm-simulators master branch tested that worked with the FPGA accelerator was from 2021-1-15 09:00 starting with the hash code 57d158b. opm-simulators releases & the master branch after that point broke the FPGA API due to incompatibilities with the GPU code in how the data is organized (i.e., reordered) and transferred to the accelerator.
 = Downloading =

@@ Line 24: / Line 24: @@
 Therefore, this shell version must also be the one installed on the Xilinx Alveo U280 FPGA board which is used to run the accelerated flow application.<br>
 In principle, the shell version can be changed to a different one (typically a newer one), but the bitstream generation process may fail or generate sub-performing bitstreams (e.g. in case the shell allocates the static resources differently than the one used currently), and changes may be required to the makefiles and other code in the FPGA repository.
 = Downloading =

@@ Line 117: / Line 117: @@
 For further information, please refer to this paper currently on Arxiv: [https://arxiv.org/abs/2101.01745 Hardware Acceleration of HPC Computational Flow Dynamics using HBM-enabled FPGAs]<br>
 This paper provides an in-depth description of the ILU0-BiCGSTAB solver kernel, including some FPGA performances gathered with a slightly older version of the Flow simulator (2020.10-rc4) and a comparison with software-only and GPU implementations.
 = Reporting issues =

@@ Line 90: / Line 90: @@
 * run flow in FPGA mode with a small dataset (e.g., SPE1CASE1.DATA found in opm-tests/spe1/): this will load the bitstream and make it available;
 * run the Xilinx programming tool to pre-load the bitstream (xbutil is part of the Xilinx XRT package):
-     xbutil program -p /path/to/bicgstab_kernel.xclbin   # if you have more than one FPGA boards on the host, add "-d<number>" to specify which FPGA must be programmed
+     xbutil program -p /path/to/bicgstab_kernel.xclbin
 and then proceed to run with the interesting datasets.<br>
 Note: if you have more than one FPGA boards on the host, add "-d<device number>" on the xbutil command line to specify which FPGA must be programmed. Flow will always use only the first Alveo U280 board in the host (e.g. the one with the lowest device number).

@@ Line 112: / Line 112: @@
 The FPGA acceleration does not support ensemble runs, because currently there is only one solver instance on the FPGA which cannot be shared among different processes.<br>
 Hence, the first flow process that starts using the FPGA won't release it until it finishes, barring the use of the FPGA by other processes for the whole run time.
 = Reporting issues =

@@ Line 28: / Line 28: @@
 Download the FPGA repository (read-only) by doing:
-     git clone git@github.com:OPM/FPGA.git
+     git clone git://github.com/OPM/FPGA.git
 = Building =

@@ Line 1: / Line 1: @@
 This page contains instructions on how to setup, compile and use the FPGA ''bitstream'' required to run the Flow simulator when compiled with the fpgaSolver enabled (ILU0-BiCGSTAB accelerated solver).
-PREREQUISITES
+= Prerequisites =
-DOWNLOADING
+Using the Flow simulator with an FPGA requires the (offline) generation of a ''bitstream'' file, which is loaded into the FPGA at runtime and configures it to make available a specific function, in this case the ILU0-BiCGSTAB solver.<br>
------------
+This generation process requires, for the kernel currently present in the FPGA repository, the installation of some Xilinx tools and an additional package to target the [https://www.xilinx.com/products/boards-and-kits/alveo/u280.html Xilinx Alveo U280 board].<br>
-BUILDING
+The bitstream generation has only been tested on Linux systems with CentOS 7 (7.4-7.9).<br>
---------
+It may however be possible to use other OS configurations (e.g. Ubuntu 16.04/18.04 as officially supported by Xilinx), but this has not been tested.
-USING THE BITSTREAM
+Please refer to [https://www.xilinx.com/html_docs/xilinx2019_2/vitis_doc/aqm1532064088764.html this page] for documentation about:
--------------------
+* a list of supported OS and minimum hardware requirements in order to compile bitstreams for the Xilinx Alveo boards (Application Acceleration Development Flow)
-REPORTING ISSUES
+The main tools required are:
-----------------
+* [https://www.xilinx.com/support/download/index.html/content/xilinx/en/downloadNav/vitis/2019-2.html Xilinx Vitis, version 2019.2]<br>'''To download the Xilinx Vitis package, a (free) registration to the Xilinx website is required.'''<br>Choose a file under "Vitis Core Development Kit", not the "Update 1" version; for example "Xilinx Unified Installer 2019.2: Linux Self Extracting Web Installer"
-Issues specific to the FPGA ''bitstream'' compilation process can be reported in the git issue tracker at:
+The additional package needed to target the Xilinx Alveo U280 board is:
      http://github.com/OPM/FPGA/issues