Xilinx
Refactoring of RTL/HLS component integration
With the growth of more and more RTL variants of common FINN hardware building blocks, there is a need to better organize the integration of these components. The purpose of this PR is to make this refactoring visible to the community because it will introduce significant changes in the compiler.
The FINN compiler was developed with the assumption that the hardware blocks corresponding to the neural network layers are developed based on HLS. Although we do not want to abolish this HLS implementation at this time, it has become apparent over the years that for certain modules it makes sense to implement them in RTL. This allows us greater control over the resulting hardware and we can make optimal use of FPGA resources. To make it easier for RTL and HLS components to both be integrated into the FINN compiler and for users to better make use of one variant or the other, a new custom op class hierarchy will be introduced.
The following steps will be needed for the refactoring:
- [x] Convert previous HLSCustomOp class to HWCustomOp class and backend mixin classes (HLSBackend and RTLBackend)
- [x] Introduce new hardware layer intermediate node representation for each node, e.g. FMPadding
- [x] Introduce backend specific nodes for each layer, e.g. FMPadding_hls and FMPadding_rtl
- [x] Add transformation to convert from standard and custom ONNX layers to hardware layer representation (will replace previous FINN compiler step "Conversion to HLS layers")
- [x] Add transformation to convert intermediate hardware layer into either HLS or RTL variant, based on availability (some layers will only have HLS or RTL variants) and other criteria (e.g bit width, fpga parts, ...)
- [x] Introduce new builder steps
- [x] for conversion to hardware layers
- [x] for specialization to HLS or RTL variants
- [ ] Refactor existing builder steps, documentation, tutorials/notebooks, ...
