basic-pitch icon indicating copy to clipboard operation
basic-pitch copied to clipboard

Add alternative model serializations for inference

Open drubinstein opened this issue 2 years ago • 0 comments

This PR is a pretty large refactor. One issue we've seen is that not all users are TensorFlow users and TensorFlow is a pretty heavy dependency. To alleviate that, this PR adds in TFLite, CoreML and ONNX serialized versions of the models. PyTorch is not included for now, but we plan to include it in the future.

Instead of TF being a default dependency, depending on the platform, a different model serialization format will be used by defualt. Instead TF is now an extra that can be installed via basic-pitch[tf].

  • On Linux, TFLite will be used via the tflite-runtime package (which only supports linux)
  • On Mac, CoreML will be used via the coremltools package (which only supports mac)
  • On Windows, ONNX will be used via the onnxruntime package (which supports all platforms)

However, if TF is detected on the system, it will be used instead. Ideally, this change will speed up initialization and install times at the cost of a slightly larger binary due to the inclusion of all the new serialized models.

In addition, related to a red herring when failing to install scipy in Python 3.11, I decided to also move from setup.cfg to pyproject.toml format in this PR. I can move that to another PR if desired.

I believe we can call this a minor bump since there are no breaking changes, but we can also call this a major bump due to the possible results changes for users.

drubinstein avatar Sep 29 '23 02:09 drubinstein