VisualGLM-6B icon indicating copy to clipboard operation
VisualGLM-6B copied to clipboard
verified publisher icon THUDM

API调用问题

Open Chen94yue opened this issue 2 years ago • 2 comments

我使用api的时候碰到这个问题:

[2023-09-13 19:47:17,004] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect)
[2023-09-13 19:47:22,544] [INFO] building VisualGLMModel model ...
[2023-09-13 19:47:22,561] [INFO] [RANK 0] > initializing model parallel with size 1
[2023-09-13 19:47:22,563] [INFO] [RANK 0] You are using model-only mode.
For torch.distributed users or loading model parallel models, set environment variables RANK, WORLD_SIZE and LOCAL_RANK.
/home/chenyue21/.local/lib/python3.11/site-packages/torch/nn/init.py:405: UserWarning: Initializing zero-element tensors is a no-op
  warnings.warn("Initializing zero-element tensors is a no-op")
[2023-09-13 19:47:38,757] [INFO] [RANK 0]  > number of parameters on model parallel rank 0: 7802193408
[2023-09-13 19:47:39,947] [INFO] [RANK 0] global rank 0 is loading checkpoint /home/chenyue21/.sat_models/visualglm-6b/1/mp_rank_00_model_states.pt
[2023-09-13 19:47:55,939] [INFO] [RANK 0] Will continue but found unexpected_keys! Check whether you are loading correct checkpoints: ['transformer.position_embeddings.weight'].
[2023-09-13 19:47:55,944] [INFO] [RANK 0] > successfully loaded /home/chenyue21/.sat_models/visualglm-6b/1/mp_rank_00_model_states.pt
[2023-09-13 19:47:56,423] [INFO] [RANK 0] > Quantizing model weight to 8 bits
[2023-09-13 19:48:03,841] [INFO] [RANK 0] > Quantized 5637144576 parameters in total.
'HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /THUDM/chatglm-6b/resolve/main/tokenizer_config.json (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7f13b420d410>, 'Connection to huggingface.co timed out. (connect timeout=10)'))' thrown while requesting HEAD https://huggingface.co/THUDM/chatglm-6b/resolve/main/tokenizer_config.json
'HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /THUDM/chatglm-6b/resolve/main/tokenization_chatglm.py (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7f13b4218f90>, 'Connection to huggingface.co timed out. (connect timeout=10)'))' thrown while requesting HEAD https://huggingface.co/THUDM/chatglm-6b/resolve/main/tokenization_chatglm.py
INFO:     Started server process [15128]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:7861 (Press CTRL+C to quit)
INFO:     11.48.73.89:46224 - "GET /metrics HTTP/1.1" 404 Not Found
INFO:     11.48.73.89:50090 - "GET /metrics HTTP/1.1" 404 Not Found
INFO:     11.48.73.89:53948 - "GET /metrics HTTP/1.1" 404 Not Found
Start to process request
描述这张图片
INFO:     11.48.66.43:59460 - "POST / HTTP/1.1" 500 Internal Server Error
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/home/chenyue21/.local/lib/python3.11/site-packages/uvicorn/protocols/http/h11_impl.py", line 408, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chenyue21/.local/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 84, in __call__
    return await self.app(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chenyue21/.local/lib/python3.11/site-packages/fastapi/applications.py", line 292, in __call__
    await super().__call__(scope, receive, send)
  File "/home/chenyue21/.local/lib/python3.11/site-packages/starlette/applications.py", line 122, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/home/chenyue21/.local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 184, in __call__
    raise exc
  File "/home/chenyue21/.local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "/home/chenyue21/.local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 79, in __call__
    raise exc
  File "/home/chenyue21/.local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "/home/chenyue21/.local/lib/python3.11/site-packages/fastapi/middleware/asyncexitstack.py", line 20, in __call__
    raise e
  File "/home/chenyue21/.local/lib/python3.11/site-packages/fastapi/middleware/asyncexitstack.py", line 17, in __call__
    await self.app(scope, receive, send)
  File "/home/chenyue21/.local/lib/python3.11/site-packages/starlette/routing.py", line 718, in __call__
    await route.handle(scope, receive, send)
  File "/home/chenyue21/.local/lib/python3.11/site-packages/starlette/routing.py", line 276, in handle
    await self.app(scope, receive, send)
  File "/home/chenyue21/.local/lib/python3.11/site-packages/starlette/routing.py", line 66, in app
    response = await func(request)
               ^^^^^^^^^^^^^^^^^^^
  File "/home/chenyue21/.local/lib/python3.11/site-packages/fastapi/routing.py", line 273, in app
    raw_response = await run_endpoint_function(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chenyue21/.local/lib/python3.11/site-packages/fastapi/routing.py", line 190, in run_endpoint_function
    return await dependant.call(**values)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chenyue21/data/VisualGLM-6B-main/api.py", line 36, in visual_glm
    answer, history, _ = chat(None, model, tokenizer, input_text, history=history, image=input_image, \
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chenyue21/data/VisualGLM-6B-main/model/chat.py", line 143, in chat
    output = filling_sequence(
             ^^^^^^^^^^^^^^^^^
  File "/home/chenyue21/.local/lib/python3.11/site-packages/sat/generation/autoregressive_sampling.py", line 123, in filling_sequence
    tokens, mems = strategy.forward(logits, tokens, mems)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chenyue21/.local/lib/python3.11/site-packages/sat/generation/sampling_strategies/base_strategy.py", line 93, in forward
    pred = torch.multinomial(probs, num_samples=1)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: probability tensor contains either `inf`, `nan` or element < 0

我打印了服务收到的问题和收到的图片都没有问题，有人可以帮忙看一下吗？

Chen94yue avatar Sep 13 '23 11:09 Chen94yue

老哥 解决了吗，我运行api.py,下载了模型，但是仍然报错，要连接hugggingface，有没有解决办法呀 [2023-09-23 09:46:27,267] [INFO] [RANK 0] > successfully loaded /home/ubuntu/.sat_models/visualglm-6b/1/mp_rank_00_model_states.pt Traceback (most recent call last): File "/home/ubuntu/.local/lib/python3.8/site-packages/transformers/utils/hub.py", line 429, in cached_file resolved_file = hf_hub_download( File "/usr/local/lib/python3.8/dist-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn return fn(*args, **kwargs) File "/usr/local/lib/python3.8/dist-packages/huggingface_hub/file_download.py", line 1291, in hf_hub_download raise LocalEntryNotFoundError( huggingface_hub.utils._errors.LocalEntryNotFoundError: Connection error, and we cannot find the requested files in the disk cache. Please try again or make sure your Internet connection is on.

lilinrestart avatar Sep 23 '23 01:09 lilinrestart

遇到了相同的问题，OSError: We couldn't connect to 'https://huggingface.co' to load this file

elesun2018 avatar Oct 12 '23 01:10 elesun2018