uTensor
Tflite backend
Install
Using a virtual environment is a good idea.
git clone [email protected]:uTensor/utensor_cgen.git
cd utensor_cgen
git checkout tflite_backend
pip install -e .[dev]
Running Test
pytest -s tests/tflm
This is semi-working for:
input0 = [1,1,1,1].tranpose #set at the run time input1 = [2,4,6,8].tranpose weight = [10,20,30,40] bias = [7]
add_output = ADD(input0, input1) out = FC(add_output, weight, bias)
out == 707
Notes
Unexplained behaviors: when the first tensor in the subgraph, input0, is not set at the runtime, the tensor is then initialized to undefined values instead of values contained in FlatBuffers. See here.
@dboyliao Can use some input on the python coding style/design patterns. Keep in mind, I barely got this to work. Perhaps using registry/factory pattern for additional op supports? Imports are a mess.
The files to look at are:
- tests/tflm/tflite_export/conftest.py
- tests/tflm/tflite_export/test_write.py
- uTensor_cgen/transformer/tflite_exporter.py
There are 3 types of quantization flows in TFLM:
- None. (inputs and outputs are floating points)
- Hybrid (one of the inputs is of floating point, and quantized for others)
- Offline (everything is quantized, including activation. Activation ranged determined offline, see the API and the Colab)
Hybrid quantization seems broken atm. uTensor-cli require additional work to support offline quantization. So, no quantization is the way to go.
Useful Docs:
- Interpreter's Python binding
- Python code snippet for inferencing
- How to get the FlatBuffers TFLM Python binding
BTW, I think it's not proper to make it a tflm transformer. It's better to implement it as a backend.
Nope, it isn't. This was prototyped as a transformer as the backend was being refactored. Changes need to be migrated.
Yes, I know. I just want to leave a note so we'll not forget to fix it before merging this PR. Should not hard converting it into a backend.