langgenius
Cohere.tokenize Request Results in JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Self Checks
- [X] This is only for bug report, if you would like to ask a question, please head to Discussions.
- [X] I have searched for existing issues search for existing issues, including closed ones.
- [X] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
- [X] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
- [X] Please do not modify this template :) and fill in all the required fields.
Dify version
0.6.12
Cloud or Self Hosted
Self Hosted (Docker)
Steps to reproduce
When using Dify to perform "Text Segmentation and Cleaning" on a knowledge base, a JSONDecodeError occurs during a tokenize request to the Cohere API, preventing the proper decoding of the API response. The detailed error message is as follows:
worker-1 | [2024-06-29 11:32:55,999: INFO/MainProcess] HTTP Request: POST https://api.cohere.ai/v1/tokenize "HTTP/1.1 200 OK"
worker-1 | [2024-06-29 11:32:56,130: INFO/MainProcess] HTTP Request: POST https://api.cohere.ai/v1/tokenize "HTTP/1.1 500 Internal Server Error"
worker-1 | [2024-06-29 11:32:56,131: ERROR/MainProcess] consume document failed
worker-1 | Traceback (most recent call last):
worker-1 | File "/app/api/core/model_runtime/model_providers/cohere/text_embedding/text_embedding.py", line 122, in get_num_tokens
worker-1 | response = self._tokenize(
worker-1 | File "/app/api/core/model_runtime/model_providers/cohere/text_embedding/text_embedding.py", line 146, in _tokenize
worker-1 | response = client.tokenize(
worker-1 | File "/app/api/.venv/lib/python3.10/site-packages/cohere/client.py", line 218, in tokenize
worker-1 | return super().tokenize(text=text, model=model, request_options=opts)
worker-1 | File "/app/api/.venv/lib/python3.10/site-packages/cohere/base_client.py", line 1557, in tokenize
worker-1 | typing.cast(typing.Any, construct_type(type_=typing.Any, object_=_response.json())) # type: ignore
worker-1 | File "/app/api/.venv/lib/python3.10/site-packages/httpx/_models.py", line 764, in json
worker-1 | return jsonlib.loads(self.content, **kwargs)
worker-1 | File "/usr/local/lib/python3.10/json/__init__.py", line 346, in loads
worker-1 | return _default_decoder.decode(s)
worker-1 | File "/usr/local/lib/python3.10/json/decoder.py", line 337, in decode
worker-1 | obj, end = self.raw_decode(s, idx=_w(s, 0).end())
worker-1 | File "/usr/local/lib/python3.10/json/decoder.py", line 355, in raw_decode
worker-1 | raise JSONDecodeError("Expecting value", s, err.value) from None
worker-1 | json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
worker-1 |
worker-1 | During handling of the above exception, another exception occurred:
worker-1 |
worker-1 | Traceback (most recent call last):
worker-1 | File "/app/api/core/indexing_runner.py", line 73, in run
worker-1 | self._load_segments(dataset, dataset_document, documents)
worker-1 | File "/app/api/core/indexing_runner.py", line 848, in _load_segments
worker-1 | doc_store.add_documents(documents)
worker-1 | File "/app/api/core/rag/docstore/dataset_docstore.py", line 97, in add_documents
worker-1 | tokens = embedding_model.get_text_embedding_num_tokens(
worker-1 | File "/app/api/core/model_manager.py", line 189, in get_text_embedding_num_tokens
worker-1 | return self._round_robin_invoke(
worker-1 | File "/app/api/core/model_manager.py", line 303, in _round_robin_invoke
worker-1 | return function(*args, **kwargs)
worker-1 | File "/app/api/core/model_runtime/model_providers/cohere/text_embedding/text_embedding.py", line 128, in get_num_tokens
worker-1 | raise self._transform_invoke_error(e)
worker-1 | core.model_runtime.errors.invoke.InvokeError: [cohere] Error: Expecting value: line 1 column 1 (char 0)
✔️ Expected Behavior
No response
❌ Actual Behavior
The Cohere API returns a 500 Internal Server Error
To resolve the JSONDecodeError during a tokenize request to the Cohere API when using Dify version 0.6.12, self-hosted via Docker, you should ensure that the JSON payload you are sending is correctly formatted. Here is an example of how to properly format the JSON payload for a tokenize request:
{
"name": "Dify",
"text": "Dify means Do it for you...",
"indexing_technique": "high_quality",
"process_rule": {
"rules": {
"pre_processing_rules": [
{
"id": "remove_extra_spaces",
"enabled": true
},
{
"id": "remove_urls_emails",
"enabled": true
}
],
"segmentation": {
"separator": "###",
"max_tokens": 500
}
},
"mode": "custom"
}
}
Ensure that the JSON structure is correct and that all necessary fields are included. If the JSON payload is malformed or missing required fields, it can lead to a JSONDecodeError [1].
Additionally, there are known issues or limitations with the Cohere API that could cause a 500 Internal Server Error during a tokenize request. Specifically, the InvokeServerUnavailableError is mapped to cohere.errors.internal_server_error.InternalServerError, which indicates that an internal server error from the Cohere API can result in a 500 Internal Server Error [2].
To get more detailed information about the tokenize request, you can configure the logging level to DEBUG:
import logging
# Set the logging level to DEBUG
logging.basicConfig(level=logging.DEBUG)
# Example usage
logger = logging.getLogger(__name__)
logger.debug("This is a debug message")
In the context of the provided code, you can add debug logging statements to the _tokenize method to log detailed information about the tokenize request:
def _tokenize(self, model: str, credentials: dict, text: str) -> list[str]:
"""
Tokenize text
:param model: model name
:param credentials: model credentials
:param text: text to tokenize
:return:
"""
if not text:
return []
# initialize client
client = cohere.Client(credentials.get('api_key'), base_url=credentials.get('base_url'))
logger.debug(f"Tokenizing text: {text} with model: {model}")
response = client.tokenize(
text=text,
model=model,
offline=False,
request_options=RequestOptions(max_retries=0)
)
logger.debug(f"Tokenize response: {response.token_strings}")
return response.token_strings
By setting the logging level to DEBUG and adding debug logging statements, you can get more detailed information about the tokenize request to the Cohere API [3][2].
To continue talking to Dosu, mention @dosu.
![dosubot[bot] avatar](https://avatars.githubusercontent.com/in/324583?v=4 "Published by a pub.dev verified publisher"){kind=small}
Looks like this issue is occurring within the Cohere client. Could you provide more detailed information? For example, what content are you processing when this error is triggered?
I initiated a high-quality text segmentation and cleaning process on the content extracted from https://swagger.io/specification/ using Firecrawl. After waiting for quite some time, an exception occurred. It seems that a large portion of the content was processed successfully as I can see some segmentation results, but the results are incomplete.
@laipz8200