inference icon indicating copy to clipboard operation
inference copied to clipboard

Add florence 2

Open MadeWithStone opened this issue 1 year ago • 1 comments

Description

This PR adds support for florence2 in workflows. It allows for a wide variety of computer vision tasks with prompts and finetuned loras. The node returns both raw outputs from the model, and parsed supervision detections.

Type of change

Please delete options that are not relevant.

  • [ ] Bug fix (non-breaking change which fixes an issue)
  • [x] New feature (non-breaking change which adds functionality)
  • [x] This change requires a documentation update

How has this change been tested, please provide a testcase or example of how you tested the change?

  • hosting the inference server on a gpu virtual machine (gcp ce) and connecting to the server using roboflow.com as the client

MadeWithStone avatar Aug 01 '24 02:08 MadeWithStone

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

:white_check_mark: MadeWithStone
:x: roboflowmaxwell
You have signed the CLA already but the status is still pending? Let us recheck it.

CLAassistant avatar Aug 01 '24 02:08 CLAassistant

Hi there, we need to push that forward, I need orientation to make some decisions:

  • @MadeWithStone - are you willing to continue the task
  • @probicheaux - what is the status of Florence 2 model on the platform, in particular:
    • what kind of pre-trained models we support
    • where Florence 2 could run - my brief analysis indicated that only on GPU server builds - if so, the block itself is not practically useful, even potentially harmful once random users got hyped, enter Workflows and see something is broken, without proper info on how to run
    • do we have plan on making good docs about this VLM?
    • do we have plan to onboard the model on the inference platform? what about CPU builds?

PawelPeczek-Roboflow avatar Aug 30 '24 08:08 PawelPeczek-Roboflow

shall we close in favour of https://github.com/roboflow/inference/pull/661

PawelPeczek-Roboflow avatar Sep 19 '24 20:09 PawelPeczek-Roboflow