Tutorial · accelerators

Running hls4ml

Take a trained Keras model and end up with a firmware slot on the board that runs it — conversion, precision, synthesis, the AXI interface, and the checks that catch a wrong answer before hardware does.

hls4ml helps scientists implement their machine learning algorithms on FPGAs, turning a trained Keras model into synthesizable C++. Supporting it on standard hardware enables real-time, low-power AI inference applications for experiments that would not otherwise take on an FPGA project. This page covers the rest of that path: onto KrIO’s programmable logic, addressable from Linux, and verified against its originating model.

Work through board bring-up first — this assumes a board that boots, joins the network, and loads firmware from Linux.

Two models are deployed on the board today, and the commands below are the ones that put them there. Measured results for those and two more are on the benchmarks page.

1. Environment

export XILINX=$HOME/Xilinx/2025.2        # 2022.1 or later
export PATH=$XILINX/Vitis/bin:$XILINX/Vivado/bin:$PATH
export XILINX_VITIS=$XILINX/Vitis XILINX_VIVADO=$XILINX/Vivado XILINX_HLS=$XILINX/Vitis

mamba create -y -n hls4ml python=3.10 -c conda-forge
PY=$HOME/miniforge3/envs/hls4ml/bin/python
$PY -m ensurepip --upgrade
$PY -m pip install "tensorflow-cpu>=2.19" scikit-learn matplotlib h5py pyyaml \
                   "hls4ml[keras-v3,profiling]"

Use $PY -m pip, not pip — a user-level ~/.local/bin/pip will shadow the environment’s.

On releases from 2025.1, vitis_hls has no launcher script. The binary ships; the wrapper does not. The other wrappers dispatch on argv[0], so a copy is the whole fix:

[ -x $XILINX/Vitis/bin/vitis_hls ] || cp $XILINX/Vitis/bin/v++ $XILINX/Vitis/bin/vitis_hls
vitis_hls -version

2. Start from a model that works

Before converting anything, confirm the float model scores what you expect on real data. A preprocessing mistake here looks exactly like a quantization problem three steps later, and you will spend a day on the wrong thing.

$PY -c "
import numpy as np, keras
m = keras.models.load_model('model.keras', compile=False)
X, y = np.load('x_test.npy'), np.load('y_test.npy')
print('float top-1:', (m.predict(X, verbose=0).argmax(1) == y).mean())"

Check the input scaling explicitly. Some published models take raw 0–255 pixels and some take 0–1; feeding the wrong one gives chance accuracy with no error message.

3. Profile activation ranges

Choose integer bits from measurement. A network’s first layers often span hundreds while everything after the first normalization fits in single digits, and a uniform format saturates the front silently.

$PY profile_ranges.py 2000

layer                   type                     min         max  int bits
input                   InputLayer             0.000     255.000         9
conv2d                  Conv2D              -536.047     525.993        11
batch_normalization     BatchNormalization    -8.068       9.341         5

Pin the input and the front of the network to what you measured; let the rest run at the default width.

4. Convert and check in C simulation

Fix accuracy here. It costs seconds; after synthesis it costs an hour.

$PY convert.py --reuse 16

cfg = hls4ml.utils.config_from_keras_model(model, granularity="name", backend="Vitis")
cfg["Model"]["Strategy"] = "Resource"
cfg["Model"]["ConvImplementation"] = "LineBuffer"
hls_model = hls4ml.converters.convert_from_keras_model(
    model, hls_config=cfg, backend="Vitis", output_dir="hls",
    part="xck26-sfvc784-2LV-c", clock_period=10.0, io_type="io_stream")
hls_model.compile()

io_type="io_stream" for anything convolutional; io_parallel is for small dense models.

If accuracy collapses with no error, suspect automatic precision inference. hls4ml sets every per-layer precision to auto at name granularity, and one bad inference is enough — on one model it gave the output layer ap_fixed<38,18>, which the stable-softmax table cannot index, and accuracy fell from 0.78 to 0.22. Pin precisions explicitly, or bisect:

$PY tools/precision_bisect.py --samples 1000

5. Choose reuse factors

ReuseFactor sets how many cycles each multiplier is shared across. Under io_stream the layers run as a dataflow pipeline, so one global factor makes the widest layer the bottleneck while narrow ones sit idle holding multipliers. Pick per layer so every layer takes about the same number of cycles.

Two constraints:

Keep each factor a divisor of n_in × n_out. Otherwise hls4ml emits nnet::DenseResource_rf_gt_nin, a class its own headers never define, and the generated C++ will not compile.
Read the conversion warnings. hls4ml silently substitutes factors it considers invalid, so what you asked for is not always what you got.

6. Synthesize, then check with Vivado

$PY convert.py --reuse 16 --csynth

For streaming models, run the FIFO depth optimizer — default depths dominate block RAM, and co-simulated depths cut one model’s estimate by two thirds. Convert normally and apply it as a flow; setting config['Flows'] replaces the writer flow and leaves the project unwritten:

hls4ml.model.optimizer.get_optimizer(
    "vitis:fifo_depth_optimization").configure(profiling_fifo_depth=32768)
hls_model.apply_flow("vitis:fifo_depth_optimization")

Do not trust the HLS resource report. On these designs it overstates LUT by about 2×, block RAM by about 3× and latency by nearly 2×, while getting DSP right. A design the estimate rejects may fit comfortably.

vivado -mode batch -source diag_synth.tcl -tclargs $PWD/hls
grep -E "CLB LUTs|Block RAM Tile|DSPs" hls/diag_util_flat.rpt

7. Wrap it in AXI4-Lite

A posted write costs ~7.6 ns on this board and a blocking read ~200 ns, so loading input through a register bank is cheap and polling is what actually costs.

Input	Cost to load	Against
16 features, 32 B	0.06 µs	1.1 µs of compute
32×32 RGB image, 3 KB	8 µs	929 µs of compute

Both under 1%, so neither deployed model uses DMA. Reach for it only when input loading approaches the inference time — it costs a contiguous buffer, a device-tree change and a driver.

An io_stream model needs a wrapper, since its ports are hls::stream. Three stages in one dataflow region, and only one process may touch the AXI-mapped array:

void model_axi(ap_uint<32> in[N_IN], ap_uint<16> out[N_OUT]) {
#pragma HLS INTERFACE s_axilite port = in bundle = CTRL
#pragma HLS INTERFACE s_axilite port = out bundle = CTRL
#pragma HLS INTERFACE s_axilite port = return bundle = CTRL
#pragma HLS DATAFLOW
    hls::stream<input_t> in_s;
    hls::stream<result_t> out_s;
    feed(in, in_s);
    mymodel(in_s, out_s);
    drain(out_s, out);
}

cd axi && vitis_hls -f run_hls_axi.tcl
sed -n '8,40p' $(find . -name "x*_axi_hw.h" | head -1)     # the register map

Read offsets from the generated header. Vitis packs 16-bit array elements two per register, so a ten-class output occupies five words.

8. Build the bitstream

The build must assert the PS contract and cap block-RAM inference. Without the cap, Vivado’s automatic inference asked for 37,312 RAMB18 on a design whose entire storage is 1.3 Mb, and synthesis failed outright:

set_property -name {STEPS.SYNTH_DESIGN.ARGS.MORE OPTIONS} -value {-max_bram 288} \
    -objects [get_runs synth_1]

vivado -mode batch -source vivado/build_bd.tcl
cd out && printf 'all:\n{\n\t[destination_device = pl] design.bit\n}\n' > design.bif
bootgen -image design.bif -arch zynqmp -process_bitstream bin -w

9. Deploy and verify

tar czf - -C target . | ssh petalinux@$BOARD \
  'rm -rf ~/install && mkdir -p ~/install && tar xzf - -C ~/install'
ssh -t petalinux@$BOARD 'cd ~/install && sudo sh install.sh'
ssh petalinux@$BOARD 'sudo krio-fw load mymodel'

Compare against C simulation, not just against accuracy. Accuracy can match by coincidence; bit-exactness cannot.

top-1 accuracy on hardware : 0.8755  (1751 of 2000)
agreement with C simulation: 1.0000  (2000 of 2000)
max abs score difference   : 0.00000
latency                    : 929 us per image (1076.1 images/s)

Both deployed models reproduce their C simulation exactly. If yours does not, the fault is in the wrapper or the driver — the network itself is already proven by step 4.

Known tool issues

Observed with hls4ml 1.3.0 and Vitis HLS 2025.2. Every one fails quietly.

Symptom	Cause
`vitis_hls: command not found`	Launcher script missing since 2025.1; copy `v++`
Generated C++ will not compile	`nnet::DenseResource_rf_gt_nin` is emitted but never defined — keep reuse factors divisors of `n_in × n_out`
Accuracy collapses, no error	Automatic precision inference; pin precisions
One pooling layer costs more than every convolution	Windowed pooling buffers the whole window under `io_stream`; use `GlobalAveragePooling2D` where the window covers the map
Build cannot find `build_prj.tcl`	Setting `config['Flows']` replaced the writer flow
Two `config_array_partition` errors every build	`-maximum_size` removed in 2025.2; harmless, the RTL is identical either way
Vivado demands tens of thousands of block RAMs	Automatic BRAM inference; cap it with `-max_bram`

Reference

Full command sequence: sw/hls4ml/RUNBOOK.md. Flow, drivers, readout service and benchmark suite: sw/hls4ml/.

Benchmarks 2026-08-26
Four benchmark models from the hls4ml literature mapped onto one XCK26 — what fits, what it costs, how far precision can drop, and how far the tool's estimates can be trusted.