chip-n-scale-queue-arranger icon indicating copy to clipboard operation
chip-n-scale-queue-arranger copied to clipboard

Transition to Python and Serverless

Open drewbo opened this issue 6 years ago • 2 comments

For fast release, this library was originally written in Node. In order to extend it's functionality, we will be gradually re-writing in Python:

  • Better integration with the machine learning community
  • Better support for non-image inputs (numpy arrays, GeoTIFF, sentinelhub-py to start)
  • Replace kes.js with serverless to reduce our institutional maintenance burden.

We will also add easier configurability for the primary lambda function (download-and-predict) at the same time, though this functionality is largely language agnostic.

drewbo avatar Oct 07 '19 17:10 drewbo

I'd recommend integrating fast.ai and Google colab for smaller entities and students working on EO data without a big budget for AWS

moloned avatar Oct 12 '19 10:10 moloned

Thanks @moloned, we will eventually move things more in that direction and currently do our trainings using those tools:

  • This will currently support hosting models using fastai via fastai-serving
  • Google colab imposes a connection time and RAM limit which can hinder running large-scale inference
  • This should provide a relatively cost effective model. For classification models, we estimate that it costs about $10 per every million tiles/chips predicted

drewbo avatar Oct 14 '19 05:10 drewbo