How to cancel Chat.CreateChatCompletionAsync

[ericliu0408]

Please advise how to cancel a running api call like below:

var results = await api.Chat.CreateChatCompletionAsync(new ChatRequest()
                            {
                                Model = Model.ChatGPTTurbo,
                                Temperature = 0.1,
                                MaxTokens = max_Tokens,
                                Messages = chatMessages
                            });

I know there is a class CancellationToken to do such thing, but how to embed it in the above method. I tried but to no avail.

Many thanks for your help.

[Baklap4]

I'm new to this project, but i don't see any ~~overloads~~ for adding a CancellationToken which is a shame as we then don't support canceling a running api call.

The format would be like this:

CancellationTokenSource cancellationTokenSource = new CancellationTokenSource(TimeSpan.FromSeconds(5)); // cancel after 5 seconds.
var results = await api.Chat.CreateChatCompletionAsync(new ChatRequest { //ommitted properties// }, cancellationTokenSource .Token);

the CreateChatCompletionAsync should have this body:

public Task<object> CreateChatCompletionAsync(ChatRequest _, CancellationTokenSource? cts = default) { // null if not supplied
                // Do your async stuff with `ChatRequest`
                if (cts?.IsCancellationRequested == true)
                {
                    throw new TaskCanceledException();
                }    
}

It would be nice to have this embedded within the project: So good suggestion, thanks!

I'll see if i have some time later this week to add this parameter