reverse-proxy icon indicating copy to clipboard operation
reverse-proxy copied to clipboard

YARP not able to detect UseRequestTimeouts middleware

Open jasidhu opened this issue 1 year ago • 12 comments

Describe the bug

I am trying to use .net8 request timeout middleware with Yarp proxy 2.1.0 but proxy is throwing following error.

To Reproduce

Error Message: The timeout was not applied for route 'route1', ensure IApplicationBuilder.UseRequestTimeouts() is called between IApplicationBuilder.UseRouting() and IApplicationBuilder.UseEndpoints()

System.InvalidOperationException: The timeout was not applied for route 'route1', 
ensure `IApplicationBuilder.UseRequestTimeouts()` is called between `IApplicationBuilder.UseRouting()` and `IApplicationBuilder.UseEndpoints()`.
   at Yarp.ReverseProxy.Model.ProxyPipelineInitializerMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.Routing.EndpointMiddleware.Invoke(HttpContext httpContext)
   at Microsoft.AspNetCore.Http.Timeouts.RequestTimeoutsMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddlewareImpl.Invoke(HttpContext context)

Further technical details

  • Yarp.ReverseProxy - 2.1.0
  • Local windows 11 machine

The config

"ReverseProxy": {
    "Routes": {
      "route1": {
        "ClusterId": "cluster1",
        "Timeout": "00:05:00",
        "Match": {
          "Methods": [ "GET" ],
          "Path": "/_api/v1/test/{**remainder}"
        }
      }
    },
    "Clusters": {
      "cluster1": {
        "Destinations": {
          "production": {
            "Address": "https://example.com"
          }
        }
      }
    }
  }

Pipeline:

var app = builder.Build();

app.UseHttpsRedirection();
app.UseRouting();
app.UseAuthentication();
app.UseAuthorization();
app.UseRequestTimeouts();
app.UseEndpoints(endpoints => endpoints.MapReverseProxy());

app.Run();

jasidhu avatar Feb 16 '24 00:02 jasidhu

Does TimeoutPolicy work instead? "TimeoutPolicy": "customPolicy",

Tratcher avatar Feb 16 '24 17:02 Tratcher

I am getting same error with TimeoutPolicy as well.

Does TimeoutPolicy work instead? "TimeoutPolicy": "customPolicy",

jasidhu avatar Feb 16 '24 17:02 jasidhu

@jasidhu Is by any chance the debugger is attached? The timeout middleware skips if debugger is attached and there will be no IHttpRequestTimeoutFeature.

Kahbazi avatar Feb 19 '24 08:02 Kahbazi

We are having the same issue which is keeping us from upgrading to .NET 8.0. Below is a snippet of our .NET Core MVC app's Program.cs.

builder.Services.AddRequestTimeouts();

var app = builder.Build();

if (!app.Environment.IsDevelopment())
{
	app.UseHsts();
}

app.UseRouting();
app.UseRequestTimeouts();
app.UseHttpsRedirection();
app.UseStaticFiles();
app.UseAuthentication();
app.UseAuthorization();
app.UseSystemWebAdapters();
app.MapDefaultControllerRoute();
app.UseEndpoints(_ => { });
app.MapReverseProxy();

app.Run();

--Route in appsettings.json throwing error. "ReverseProxy": { "Routes": { "reports_route": { "ClusterId": "frameworkCluster", "Timeout": "00:10:00", "Match": { "Path": "/Reports" } },

BenWissel avatar Feb 19 '24 17:02 BenWissel

As far as I can tell we will currently incorrectly flag requests in the following cases:

  • A debugger is attached to the proxy
  • A custom policy returns a null / infinite timeout (seems like a plausibly valid use case, we can detect it)
  • Misc cases we don't care about (e.g. removing existing policy from options, manually removing the timeout feature, ...)

Given a config with just a timeout set (e.g. "Timeout": "00:10:00"), this should only happen if the debugger was attached to the proxy. Is that the case for you @jasidhu, @BenWissel?

MihaZupan avatar Feb 20 '24 06:02 MihaZupan

Triage: We should improve the check here to at least avoid throwing when the debugger is attached. Moving to vNext since this is annoying when debugging and should be simple to fix.

MihaZupan avatar Feb 20 '24 06:02 MihaZupan

"Timeout": "00:05:00",

@MihaZupan Yes. It is failing only when debugger is attached.

jasidhu avatar Feb 20 '24 17:02 jasidhu

Triage: We should improve the check here to at least avoid throwing when the debugger is attached. Moving to vNext since this is annoying when debugging and should be simple to fix.

Thanks for running this down. We've been instructed internally to wait on our upgrade to .NET 8.0 until this issue in YARP is resolved.

BenWissel avatar Feb 21 '24 13:02 BenWissel

Is there any way to work on .NET 8, before the next version?

doddgu avatar Mar 18 '24 05:03 doddgu

As a workaround, you could disable request timeouts in YARP and rely on the activity timeout instead

MihaZupan avatar Mar 26 '24 11:03 MihaZupan

Triage: We should improve the check here to at least avoid throwing when the debugger is attached. Moving to vNext since this is annoying when debugging and should be simple to fix.

Thanks for running this down. We've been instructed internally to wait on our upgrade to .NET 8.0 until this issue in YARP is resolved.

Is there a release schedule for when the timeout bugs will be resolved in YARP? In our application we're having timeout issues with legacy report embedding and would also like to be able to upgrade to .NET 8 sooner rather than later.

bfwissel avatar Apr 04 '24 12:04 bfwissel

My understanding is that request timeouts are an opt-in (off by default) .NET 8 feature, where you can opt-in via the YARP config. As such, I don't see why this would be a blocker for migrating to .NET 8. You can avoid enabling the feature and use alternatives such as the existing YARP-specific activity timeout when you do want to override timeouts for the time being.

MihaZupan avatar Apr 04 '24 12:04 MihaZupan