Picking up on the Hello AFU project, I’ve recently gone through the motions of building the Hello AFU project for an actual CAPI device and tested it out. This post documents the process I followed to build and deploy this on real hardware.
To complete this process you’ll need a few things:
* A POWER8 based machine, for me I’m using a Barreleye server
* An Alpha-Data KU3 card
* The latest HDK archive from Alpha Data’s support site, at this time that file is named ADMPCIEKU3_CAPI_HDK_REL18MAR16.zip
* A licensed version of Xilinx’s Vivado
Preparing files for the build
First off, we need to extract the HDK
In the HDK by default, there will be some AFU source files in
adku060_capi_1_1_release/Sources/afu/ we’ll jump in there and delete them, then copy over the SystemVerilog files from the hello-afu repository
cd adku060_capi_1_1_release/Sources/afu/ rm * cp ~/projects/hello-afu/*.sv .
Next, open the project file
adku060_capi_1_1_release/Sources/prj/psl_fpga.prj in a text editor to change a few lines. Remove all of the lines that start with
verilog work, then add lines to reference the source files we copied into the afu directory. Some bash-fu for that:
cd ../prj sed -i '/^verilog work/d' psl_fpga.prj for i in `ls ../afu/*.sv | cut -d'/' -f3`; do echo "verilog work \"afu/$i\"" >> psl_fpga.prj; done
That should have us setup to build our AFU in leiu of the one that comes with the HDK!
Build and flash the binfile
With our files in the right spot and our project file modified, we just need to run a few of the tcl scripts in the HDK through vivado.
vivado -mode batch -source psl_fpga.tcl -notrace vivado -mode batch -source write_bitstream.tcl -notrace
The first run here does the heavy lifting of synthesis, place and route, etc. The second command generates the actual binfile and bitfile that we can use to flash the device. The first command takes a significant amount of time on my i7-equipped laptop, about 40 minutes, the second command completed in about 9 seconds. Maybe someday we’ll have a CAPI-based accelerator for synthesis and place & route! Now that the building is complete I have my bitfile at
To flash this to your device to a card that already has the PSL working you can use the capi-flash-script utility. If your card is factory-fresh or in a bad state, you can use a JTAG programmer and Vivado’s Hardware Manager to flash directly from your laptop, or remotely via xvcserver.
Using the AFU
After I flashed my AFU, I ensured libcxl was setup on my server. Since I’m running Ubunt 16.04 I simply installed it via apt.
apt-get install -y libcxl-dev
Next I rebooted the machine so that everything is nice and fresh, as part of the PCIe reset the bitfile from the KU3’s flash chip will be flashed onto the FPGA. I can verify the card is in a good state because I have my cxl device at
I run my
test_afu binary from the
hello-afu project and boom! The same result as I get from simulation, woo-hoo!