dify How to set waiting time > 60s for OpenAI compatible model?

Self Checks

[X] I have searched for existing issues search for existing issues, including closed ones.
[X] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
[X] Pleas do not modify this template :) and fill in all the required fields.

Provide a description of the help you need

I use dify default config and returned result like this:

Mar 13 '24 03:03 PhenixZhang

Hey @PhenixZhang!👋 Great to see you diving into another challenge! How's everything going on your end?

Based on the information you've provided, it seems like you're trying to set a waiting time longer than 60 seconds for an OpenAI compatible model in the dify repository. However, the provided code snippet does not include a direct way to set a timeout or waiting time for API requests.

If the intention is to adjust the timeout for API requests to the OpenAI service, this would typically be handled in the part of your application that makes the HTTP requests, not in the configuration object for UI components. The configuration object shown is primarily for setting up UI elements and does not directly influence how the API calls are made or how long they wait for a response.

To implement a waiting time configuration, you would need to modify the part of your application that performs the API requests to OpenAI. This could involve adding a timeout parameter to your HTTP request function or configuring the HTTP client you're using to increase its default timeout. Here's a conceptual example using fetch in JavaScript, which is commonly used for making HTTP requests:

// Example function to make a request to OpenAI with a custom timeout
async function fetchWithTimeout(resource, options = {}) {
  const { timeout = 70000 } = options; // Default timeout set to 70 seconds

  const controller = new AbortController();
  const id = setTimeout(() => controller.abort(), timeout);
  const response = await fetch(resource, {
    ...options,
    signal: controller.signal  
  });
  clearTimeout(id);
  return response;
}

// Usage example
fetchWithTimeout('https://api.openai.com/v1/models', {
  method: 'GET',
  headers: {
    'Authorization': `Bearer YOUR_OPENAI_API_KEY`
  },
  timeout: 120000 // Setting custom timeout to 120 seconds
}).then(response => response.json())
  .then(data => console.log(data))
  .catch(error => console.error('Timeout or other error:', error));

This example shows how to define a fetchWithTimeout function that allows specifying a custom timeout. You would need to integrate a similar approach where your application makes the API requests to OpenAI, adjusting the timeout value as needed. Remember, the exact implementation will depend on how your application is structured and which libraries or frameworks you are using for HTTP requests.

Regarding the structure of the dify repository, specifically where the API requests to OpenAI are made, can be identified within the provided context of the file api/core/agent/agent/openai_function_call.py. The API requests to OpenAI are made in the methods should_use_agent and plan of the AutoSummarizingOpenAIFunctionCallAgent class. These methods utilize the ModelInstance.invoke_llm function to make the requests.

As for the exact error message returned in the <Response [400]> from the OpenAI API, I'm afraid I don't have enough information to provide a specific answer. The <Response [400]> indicates a client-side error, which could be due to a variety of reasons such as invalid request syntax, size too large, invalid request message framing, or deceptive request routing. Without the exact error message or more details about the request that was made, it's difficult to determine the exact cause of the error. If you could provide more details about the request or the exact error message, I might be able to provide a more specific answer.

Sources

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

Mar 13 '24 03:03 dosubot[bot]

Could you please provide more details of the issue you encountered?

Mar 13 '24 06:03 takatost

Hi, @PhenixZhang,

I'm helping the team manage our backlog and am marking this issue as stale. From what I understand, you were looking to set a waiting time greater than 60 seconds for an OpenAI compatible model in the dify project. I provided a detailed response explaining that the configuration object shown is primarily for setting up UI elements and does not directly influence how the API calls are made or how long they wait for a response. I also suggested modifying the part of the application that performs the API requests to OpenAI to implement a waiting time configuration.

Could you please confirm if this issue is still relevant to the latest version of the repository? If it is, please let the team know by commenting on the issue. Otherwise, feel free to close the issue yourself, or it will be automatically closed in 7 days. Thank you!

Mar 29 '24 16:03 dosubot[bot]