lmdeploy icon indicating copy to clipboard operation
lmdeploy copied to clipboard

#1514 Update docker docs for VL api

Open vody-am opened this issue 1 year ago • 5 comments

Docs update in relation to issue https://github.com/InternLM/lmdeploy/issues/1514

Motivation

Updating documentation around Docker images for new users. It should be made clear to users what is expected, and a way to resolve issues described.

Modification

Adding a warning about dependencies and an example of how to resolve by creating a new Docker image.

vody-am avatar Apr 30 '24 15:04 vody-am

@vody-am If you are familiar with Chinese, could you please update the user guide in docs/zh_cn/serving/api_server_vl.md?

lvhan028 avatar Apr 30 '24 16:04 lvhan028

@lvhan028 Google translate has been working well, so yes I will give it a shot :joy:

vody-am avatar Apr 30 '24 16:04 vody-am

@lvhan028 while I am here -- is there anything special one has to set to use multiple GPUs? Thus far I have experimented with one container per GPU, but it seems like there should be a way to use multiple cards. Is it the --tp setting?

vody-am avatar Apr 30 '24 22:04 vody-am

Yes, by setting tp parameter

lvhan028 avatar May 01 '24 11:05 lvhan028

Hi, @vody-am there are linting errors, which are able to fixed by running the following commands:

pip install pre-commit
cd the/root/path/of/lmdeploy
pre-commit install .
pre-commit run --all-files

lvhan028 avatar May 02 '24 01:05 lvhan028