Update QONNX parsing for 1.0

[jmitrevs]

Description

This change updates the ONNX parser and adds support for QONNX. It replaces PR https://github.com/fastmachinelearning/hls4ml/pull/832. It only supports ONNX that has been cleaned by the qonnx package, including converting convolutions to be channels-last and changing Gemm to MatMul and Add.

In QONNX Quant nodes can act on constants as well as the datapath. To make handling this easier, we explicitly put constants in the initial graph. There are also some helper nodes like MatMul and Conv that are introduced to support the explicit constant nodes. After the convert flow, no special ONNX nodes remain in the graph, though.

Generally Quant nodes that have power-of-2 scales and no zero-offset get converted to fixed data types either by setting the types of constants or adding a linear activation that is usually merged into preceding nodes. Non-power-of-2 scales result in ApplyAlpha nodes beings added to scale and unscale, with propagation across some layers. This can be further optimized and has generally been tested less.

This includes the changes from PR #855 with a few updates that will be backported and discussed there. Therefore, this PR needs to wait till that PR is merged, which is why I am making it draft.

Note: for the config_from_onnx_model I made the default granularity be "name" because that enables automatic precision inference, which you need for QONNX. The way that I did that is that I set config['Model']['Precision'] to the default (e.g. fixed<16,6>), but all the precisions filled by config['Model'] are auto. These can be overriden if, for example, the accumulator becomes too wide. In general, though, they are set by the infer_precision.py optimizer.

Binary networks are not yet supported.

More information can be found in this presentation: https://www.icloud.com/keynote/025yxvgBx8IF2m3Iso6HosqPw#QONNX_Ingestion_0p1

Type of change

• [x] New feature (non-breaking change which adds functionality)
• [x] A new research paper code implementation

Tests

The pytest, test_qonnx.py, is the main test, building some models from the QONNX model zoo

Checklist

• [x] I have read the guidelines for contributing.
• [x] I have commented my code, particularly in hard-to-understand areas.
• [ ] I have made corresponding changes to the documentation.
• [x] My changes generate no new warnings.
• [x] I have installed and run pre-commit on the files I edited or added.
• [x] I have added tests that prove my fix is effective or that my feature works.

[sei-jgwohlbier]

Hi, I'm not sure if this is the right place for this, but I'm trying to synthesize the cnv-2w2a example from the test_qonnx.py file. It has been running for nearly two days now without a sign of progressing. Have you synthesized this? Do you have an idea of how long it took? Thanks!

Initially forgot to mention that I'm running in a container. I found this, which suggests adding the --init run arg. Trying that now.

Output

****** Vivado(TM) HLS - High-Level Synthesis from C, C++ and SystemC v2019.2 (64-bit)
  **** SW Build 2708876 on Wed Nov  6 21:39:14 MST 2019
  **** IP Build 2700528 on Thu Nov  7 00:09:20 MST 2019
    ** Copyright 1986-2019 Xilinx, Inc. All Rights Reserved.

source /tools/Xilinx/Vivado/2019.2/scripts/vivado_hls/hls.tcl -notrace
INFO: Applying HLS Y2K22 patch v1.2 for IP revision
INFO: [HLS 200-10] Running '/tools/Xilinx/Vivado/2019.2/bin/unwrapped/lnx64.o/vivado_hls'
INFO: [HLS 200-10] For user 'hls4ml-user' on host 'etc-gpu-09.etc.sei.cmu.edu' (Linux_x86_64 version 4.18.0-513.24.1.el8_9.x86_64) on Fri Jun 21 14:33:16 UTC 2024
INFO: [HLS 200-10] In directory '/home/hls4ml-user/work/hls4mlprj_qonnx_cnv-2w2a_Vivado'
Sourcing Tcl script 'build_prj.tcl'
INFO: [HLS 200-10] Creating and opening project '/home/hls4ml-user/work/hls4mlprj_qonnx_cnv-2w2a_Vivado/myproject_prj'.
INFO: [HLS 200-10] Adding design file 'firmware/myproject.cpp' to the project
INFO: [HLS 200-10] Adding test bench file 'myproject_test.cpp' to the project
INFO: [HLS 200-10] Adding test bench file 'firmware/weights' to the project
INFO: [HLS 200-10] Adding test bench file 'tb_data' to the project
INFO: [HLS 200-10] Creating and opening solution '/home/hls4ml-user/work/hls4mlprj_qonnx_cnv-2w2a_Vivado/myproject_prj/solution1'.
INFO: [XFORM 203-101] Allowed max sub elements number after partition is 4096.
INFO: [XFORM 203-1161] The maximum of name length is set into 80.
INFO: [HLS 200-10] Setting target device to 'xcvu13p-flga2577-2-e'
INFO: [SYN 201-201] Setting up clock 'default' with a period of 5ns.
INFO: [SYN 201-201] Setting up clock 'default' with an uncertainty of 0.625ns.
***** C SIMULATION *****
INFO: [SIM 211-2] *************** CSIM start ***************
INFO: [SIM 211-4] CSIM will launch GCC as the compiler.
   Compiling ../../../../myproject_test.cpp in debug mode
   Compiling ../../../../firmware/myproject.cpp in debug mode
   Generating csim.exe
INFO: Unable to open input/predictions file, using default input.
-1.4662 -1.68291 -0.174961 -0.68965 -0.22011 -1.15919 -0.915391 -0.969569 -0.987628 -1.29464 
INFO: Saved inference results to file: tb_data/csim_results.log
INFO: [SIM 211-1] CSim done with 0 errors.
INFO: [SIM 211-3] *************** CSIM finish ***************
***** C SIMULATION COMPLETED IN 0h1m0s *****
***** C/RTL SYNTHESIS *****
INFO: [SCHED 204-61] Option 'relax_ii_for_timing' is enabled, will increase II to preserve clock frequency constraints.
INFO: [HLS 200-10] Analyzing design file 'firmware/myproject.cpp' ... 
WARNING: [HLS 214-113] Either use an argument of the function or declare the variable inside the dataflow loop body: firmware/myproject.cpp:69:75
WARNING: [HLS 214-113] Either use an argument of the function or declare the variable inside the dataflow loop body: firmware/myproject.cpp:69:81
WARNING: [HLS 214-113] Either use an argument of the function or declare the variable inside the dataflow loop body: firmware/myproject.cpp:73:94
WARNING: [HLS 214-113] Either use an argument of the function or declare the variable inside the dataflow loop body: firmware/myproject.cpp:73:100
WARNING: [HLS 214-113] Either use an argument of the function or declare the variable inside the dataflow loop body: firmware/myproject.cpp:77:93
WARNING: [HLS 214-113] Either use an argument of the function or declare the variable inside the dataflow loop body: firmware/myproject.cpp:77:99
WARNING: [HLS 214-113] Either use an argument of the function or declare the variable inside the dataflow loop body: firmware/myproject.cpp:81:94
WARNING: [HLS 214-113] Either use an argument of the function or declare the variable inside the dataflow loop body: firmware/myproject.cpp:81:100
WARNING: [HLS 214-113] Either use an argument of the function or declare the variable inside the dataflow loop body: firmware/myproject.cpp:85:93
WARNING: [HLS 214-113] Either use an argument of the function or declare the variable inside the dataflow loop body: firmware/myproject.cpp:85:99
WARNING: [HLS 214-113] Either use an argument of the function or declare the variable inside the dataflow loop body: firmware/myproject.cpp:93:94
WARNING: [HLS 214-113] Either use an argument of the function or declare the variable inside the dataflow loop body: firmware/myproject.cpp:93:100
WARNING: [HLS 214-113] Either use an argument of the function or declare the variable inside the dataflow loop body: firmware/myproject.cpp:97:93
WARNING: [HLS 214-113] Either use an argument of the function or declare the variable inside the dataflow loop body: firmware/myproject.cpp:97:99
WARNING: [HLS 214-113] Either use an argument of the function or declare the variable inside the dataflow loop body: firmware/myproject.cpp:101:94
WARNING: [HLS 214-113] Either use an argument of the function or declare the variable inside the dataflow loop body: firmware/myproject.cpp:101:100
WARNING: [HLS 214-113] Either use an argument of the function or declare the variable inside the dataflow loop body: firmware/myproject.cpp:105:93
WARNING: [HLS 214-113] Either use an argument of the function or declare the variable inside the dataflow loop body: firmware/myproject.cpp:105:99
WARNING: [HLS 214-113] Either use an argument of the function or declare the variable inside the dataflow loop body: firmware/myproject.cpp:113:94
WARNING: [HLS 214-113] Either use an argument of the function or declare the variable inside the dataflow loop body: firmware/myproject.cpp:113:100
WARNING: [HLS 214-113] Either use an argument of the function or declare the variable inside the dataflow loop body: firmware/myproject.cpp:117:93
WARNING: [HLS 214-113] Either use an argument of the function or declare the variable inside the dataflow loop body: firmware/myproject.cpp:117:99
WARNING: [HLS 214-113] Either use an argument of the function or declare the variable inside the dataflow loop body: firmware/myproject.cpp:121:94
WARNING: [HLS 214-113] Either use an argument of the function or declare the variable inside the dataflow loop body: firmware/myproject.cpp:121:100
WARNING: [HLS 214-113] Either use an argument of the function or declare the variable inside the dataflow loop body: firmware/myproject.cpp:125:93
WARNING: [HLS 214-113] Either use an argument of the function or declare the variable inside the dataflow loop body: firmware/myproject.cpp:125:99
WARNING: [HLS 214-113] Either use an argument of the function or declare the variable inside the dataflow loop body: firmware/myproject.cpp:130:90
WARNING: [HLS 214-113] Either use an argument of the function or declare the variable inside the dataflow loop body: firmware/myproject.cpp:130:96
WARNING: [HLS 214-113] Either use an argument of the function or declare the variable inside the dataflow loop body: firmware/myproject.cpp:134:94
WARNING: [HLS 214-113] Either use an argument of the function or declare the variable inside the dataflow loop body: firmware/myproject.cpp:134:100
WARNING: [HLS 214-113] Either use an argument of the function or declare the variable inside the dataflow loop body: firmware/myproject.cpp:138:90
WARNING: [HLS 214-113] Either use an argument of the function or declare the variable inside the dataflow loop body: firmware/myproject.cpp:138:96
WARNING: [HLS 214-113] Either use an argument of the function or declare the variable inside the dataflow loop body: firmware/myproject.cpp:142:94
WARNING: [HLS 214-113] Either use an argument of the function or declare the variable inside the dataflow loop body: firmware/myproject.cpp:142:100
WARNING: [HLS 214-113] Either use an argument of the function or declare the variable inside the dataflow loop body: firmware/myproject.cpp:146:90
WARNING: [HLS 214-113] Either use an argument of the function or declare the variable inside the dataflow loop body: firmware/myproject.cpp:146:96
WARNING: [HLS 214-113] Either use an argument of the function or declare the variable inside the dataflow loop body: firmware/myproject.cpp:148:95
WARNING: [HLS 214-113] Either use an argument of the function or declare the variable inside the dataflow loop body: firmware/myproject.cpp:148:101
WARNING: [HLS 200-471] Dataflow form checks found 38 issue(s) in file firmware/myproject.cpp
INFO: [HLS 200-111] Finished Linking Time (s): cpu = 00:13:53 ; elapsed = 00:14:19 . Memory (MB): peak = 892.973 ; gain = 463.164 ; free physical = 21561 ; free virtual = 168305
INFO: [HLS 200-111] Finished Checking Pragmas Time (s): cpu = 00:13:53 ; elapsed = 00:14:19 . Memory (MB): peak = 892.973 ; gain = 463.164 ; free physical = 21561 ; free virtual = 168305
INFO: [HLS 200-10] Starting code transformations ...