shinyproxy-operator icon indicating copy to clipboard operation
shinyproxy-operator copied to clipboard
verified publisher icon openanalytics

Issues upgrading to 3.1.1 from 3.0.2

Open Lukeesec opened this issue 1 year ago • 3 comments

We are running in Azure using managed K8S. We've been running shinyproxy:3.0.2 + shinyproxy-operator:2.0.0 without issues. i've since tried upgrading by simply deploying into a new namespace using the sp-namespaced-apps setup.

One of the issues ran into is that the sp-shinyproxy-rs-... pods dont appear to want to cycle after configurations changes. The second issue is that two sp-shinyproxy-rs pods are spun up one with -rs-0 and the other with -rs-null.

When configuration changes occur the new sp-shinyproxy-rs-1 attempts to run and then fails. Here's the log for that too:

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::                (v3.2.6)

2024-09-06T19:11:42.293Z  INFO 1 --- [           main] .s.d.r.c.RepositoryConfigurationDelegate : Multiple Spring Data modules found, entering strict repository configuration mode
2024-09-06T19:11:42.298Z  INFO 1 --- [           main] .s.d.r.c.RepositoryConfigurationDelegate : Bootstrapping Spring Data Redis repositories in DEFAULT mode.
2024-09-06T19:11:42.409Z  INFO 1 --- [           main] .s.d.r.c.RepositoryConfigurationDelegate : Finished Spring Data repository scanning in 91 ms. Found 0 Redis repository interfaces.
2024-09-06T19:11:42.599Z  INFO 1 --- [           main] faultConfiguringBeanFactoryPostProcessor : No bean named 'errorChannel' has been explicitly defined. Therefore, a default PublishSubscribeChannel will be created.
2024-09-06T19:11:42.606Z  INFO 1 --- [           main] faultConfiguringBeanFactoryPostProcessor : No bean named 'integrationHeaderChannelRegistry' has been explicitly defined. Therefore, a default DefaultHeaderChannelRegistry will be created.
2024-09-06T19:11:43.775Z  INFO 1 --- [           main] e.o.c.stat.StatCollectorFactory          : Enabled. Sending usage statistics to micrometer.
2024-09-06T19:11:45.219Z  WARN 1 --- [           main] io.undertow.websockets.jsr               : UT026010: Buffer pool was not set on WebSocketDeploymentInfo, the default pool will be used
2024-09-06T19:11:45.294Z  INFO 1 --- [           main] io.undertow.servlet                      : Initializing Spring embedded WebApplicationContext
2024-09-06T19:11:45.295Z  INFO 1 --- [           main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 6289 ms
2024-09-06T19:11:45.516Z  INFO 1 --- [           main] e.o.c.service.IdentifierService          : ShinyProxy runtimeId:                   8lpg
2024-09-06T19:11:45.688Z  INFO 1 --- [           main] e.o.c.service.IdentifierService          : ShinyProxy instanceID (hash of config): 9989d0cce455f0153effe8235a0865c6d2ba4e68
2024-09-06T19:11:45.689Z  INFO 1 --- [           main] e.o.c.service.IdentifierService          : ShinyProxy realmId:                     shinyproxy-test-shinyproxy
2024-09-06T19:11:45.690Z  INFO 1 --- [           main] e.o.c.service.IdentifierService          : ShinyProxy version:                     1725649893637
SNIP
2024-09-06T19:11:51.679Z  INFO 1 --- [           main] o.s.b.a.w.s.WelcomePageHandlerMapping    : Adding welcome page template: index
2024-09-06T19:11:53.086Z  INFO 1 --- [           main] o.s.s.web.DefaultSecurityFilterChain     : Will secure any request with [org.springframework.security.web.session.DisableEncodeUrlFilter@517fb2c3, org.springframework.security.web.session.ForceEagerSessionCreationFilter@6c289070, org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter@69e58566, org.springframework.security.web.context.SecurityContextHolderFilter@6bbff652, org.springframework.security.web.header.HeaderWriterFilter@7ae97a58, org.springframework.web.filter.CorsFilter@2e41e0c, org.springframework.security.web.csrf.CsrfFilter@59da4992, org.springframework.security.web.authentication.logout.LogoutFilter@5a35efac, org.springframework.security.web.authentication.UsernamePasswordAuthenticationFilter@130bd00e, org.springframework.security.web.authentication.www.BasicAuthenticationFilter@32aa27a7, eu.openanalytics.containerproxy.util.AppRecoveryFilter@7cc842b0, eu.openanalytics.containerproxy.security.UserAgentFilter@2e214d39, eu.openanalytics.containerproxy.security.FixedRequestCacheAwareFilter@622e39d, org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter@6a9c848, org.springframework.security.web.authentication.AnonymousAuthenticationFilter@43a9988a, org.springframework.security.web.session.SessionManagementFilter@552b7481, org.springframework.security.web.access.ExceptionTranslationFilter@63661fc7, eu.openanalytics.shinyproxy.AuthenticationRequiredFilter@611ffa8d, org.springframework.security.web.access.intercept.AuthorizationFilter@4b02dc4e]
2024-09-06T19:11:53.886Z  INFO 1 --- [           main] o.s.l.c.support.AbstractContextSource    : Property 'userDn' not set - anonymous context will be used for read-only operations
2024-09-06T19:11:55.481Z  INFO 1 --- [           main] o.s.i.endpoint.EventDrivenConsumer       : Adding {logging-channel-adapter:_org.springframework.integration.errorLogger} as a subscriber to the 'errorChannel' channel
2024-09-06T19:11:55.482Z  INFO 1 --- [           main] o.s.i.channel.PublishSubscribeChannel    : Channel 'ContainerProxy.errorChannel' has 1 subscriber(s).
2024-09-06T19:11:55.482Z  INFO 1 --- [           main] o.s.i.endpoint.EventDrivenConsumer       : started bean '_org.springframework.integration.errorLogger'
2024-09-06T19:11:55.488Z  INFO 1 --- [           main] io.undertow                              : starting server: Undertow - 2.3.13.Final
2024-09-06T19:11:55.495Z  INFO 1 --- [           main] org.xnio                                 : XNIO version 3.8.8.Final
2024-09-06T19:11:55.503Z  INFO 1 --- [           main] org.xnio.nio                             : XNIO NIO Implementation Version 3.8.8.Final
2024-09-06T19:11:55.611Z  INFO 1 --- [           main] org.jboss.threads                        : JBoss Threads version 3.5.0.Final
2024-09-06T19:11:55.717Z  INFO 1 --- [           main] o.s.b.w.e.undertow.UndertowWebServer     : Undertow started on port 8080 (http)
2024-09-06T19:11:55.902Z  INFO 1 --- [           main] io.undertow.servlet                      : Initializing Spring embedded WebApplicationContext
2024-09-06T19:11:55.902Z  INFO 1 --- [           main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 180 ms
2024-09-06T19:11:55.912Z  INFO 1 --- [           main] o.s.b.a.e.web.EndpointLinksResolver      : Exposing 2 endpoint(s) beneath base path '/actuator'
2024-09-06T19:11:56.003Z  INFO 1 --- [           main] io.undertow                              : starting server: Undertow - 2.3.13.Final
2024-09-06T19:11:56.005Z  INFO 1 --- [           main] o.s.b.w.e.undertow.UndertowWebServer     : Undertow started on port 9090 (http)
2024-09-06T19:11:56.212Z  INFO 1 --- [           main] e.o.c.service.AppRecoveryService         : Recovery of running apps disabled
2024-09-06T19:11:56.213Z  INFO 1 --- [           main] e.o.c.util.StartupEventListener          : Started ShinyProxy 3.1.1 (ContainerProxy 1.1.1)
2024-09-06T19:11:56.383Z  INFO 1 --- [ taskExecutor-1] .o.c.s.l.r.RedisCheckLatestConfigService : This server is running the latest configuration (instanceId: 9989d0cce455f0153effe8235a0865c6d2ba4e68, version: 1725649893637), taking part in leader election.
2024-09-06T19:11:56.480Z  INFO 1 --- [ck-leadership-1] e.o.c.s.leader.redis.RedisLeaderService  : This server (runtimeId: 8lpg) is now the leader.
2024-09-06T19:12:07.214Z  INFO 1 --- [ionShutdownHook] io.undertow                              : stopping server: Undertow - 2.3.13.Final
2024-09-06T19:12:07.224Z  INFO 1 --- [ionShutdownHook] io.undertow                              : stopping server: Undertow - 2.3.13.Final
2024-09-06T19:12:07.227Z  INFO 1 --- [ck-leadership-1] e.o.c.s.leader.redis.RedisLeaderService  : This server (runtimeId: 8lpg) is no longer the leader.
2024-09-06T19:12:07.333Z  INFO 1 --- [ionShutdownHook] o.s.i.endpoint.EventDrivenConsumer       : Removing {logging-channel-adapter:_org.springframework.integration.errorLogger} as a subscriber to the 'errorChannel' channel
2024-09-06T19:12:07.333Z  INFO 1 --- [ionShutdownHook] o.s.i.channel.PublishSubscribeChannel    : Channel 'ContainerProxy.errorChannel' has 0 subscriber(s).
2024-09-06T19:12:07.333Z  INFO 1 --- [ionShutdownHook] o.s.i.endpoint.EventDrivenConsumer       : stopped bean '_org.springframework.integration.errorLogger'

operator log

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::                (v3.2.6)

2024-09-06T18:07:35.995Z  INFO 1 --- [           main] .s.d.r.c.RepositoryConfigurationDelegate : Multiple Spring Data modules found, entering strict repository configuration mode
2024-09-06T18:07:35.999Z  INFO 1 --- [           main] .s.d.r.c.RepositoryConfigurationDelegate : Bootstrapping Spring Data Redis repositories in DEFAULT mode.
2024-09-06T18:07:36.090Z  INFO 1 --- [           main] .s.d.r.c.RepositoryConfigurationDelegate : Finished Spring Data repository scanning in 79 ms. Found 0 Redis repository interfaces.
2024-09-06T18:07:36.222Z  INFO 1 --- [           main] faultConfiguringBeanFactoryPostProcessor : No bean named 'errorChannel' has been explicitly defined. Therefore, a default PublishSubscribeChannel will be created.
2024-09-06T18:07:36.288Z  INFO 1 --- [           main] faultConfiguringBeanFactoryPostProcessor : No bean named 'integrationHeaderChannelRegistry' has been explicitly defined. Therefore, a default DefaultHeaderChannelRegistry will be created.
2024-09-06T18:07:37.409Z  INFO 1 --- [           main] e.o.c.stat.StatCollectorFactory          : Enabled. Sending usage statistics to micrometer.
2024-09-06T18:07:39.124Z  WARN 1 --- [           main] io.undertow.websockets.jsr               : UT026010: Buffer pool was not set on WebSocketDeploymentInfo, the default pool will be used
2024-09-06T18:07:39.199Z  INFO 1 --- [           main] io.undertow.servlet                      : Initializing Spring embedded WebApplicationContext
2024-09-06T18:07:39.200Z  INFO 1 --- [           main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 6708 ms
2024-09-06T18:07:39.514Z  INFO 1 --- [           main] e.o.c.service.IdentifierService          : ShinyProxy runtimeId:                   lbgh
2024-09-06T18:07:39.715Z  INFO 1 --- [           main] e.o.c.service.IdentifierService          : ShinyProxy instanceID (hash of config): 55ba1652b483cd33b2433ebb35136b980289f0ee
2024-09-06T18:07:39.717Z  INFO 1 --- [           main] e.o.c.service.IdentifierService          : ShinyProxy realmId:                     shinyproxy-test-shinyproxy
2024-09-06T18:07:39.717Z  INFO 1 --- [           main] e.o.c.service.IdentifierService          : ShinyProxy version:                     1725646039375
2024-09-06T18:07:42.604Z  INFO 1 --- [           main] e.o.c.b.k.KubernetesManifestsRemover     : Kubernetes additional manifests is supported for resource [Group: '', Version: 'v1', Kind: 'Pod'] in namespace: test-APP
2024-09-06T18:07:42.610Z  INFO 1 --- [           main] e.o.c.b.k.KubernetesManifestsRemover     : Kubernetes additional manifests is supported for resource [Group: '', Version: 'v1', Kind: 'Pod'] in namespace: test-APP2
2024-09-06T18:07:42.620Z  INFO 1 --- [           main] e.o.c.b.k.KubernetesManifestsRemover     : Kubernetes additional manifests is supported for resource [Group: '', Version: 'v1', Kind: 'Pod'] in namespace: test-shinyproxy
2024-09-06T18:07:43.224Z  INFO 1 --- [           main] e.o.c.b.k.KubernetesManifestsRemover     : Kubernetes additional manifests is supported for resource [Group: '', Version: 'v1', Kind: 'Event'] in namespace: test-APP
2024-09-06T18:07:43.227Z  INFO 1 --- [           main] e.o.c.b.k.KubernetesManifestsRemover     : Kubernetes additional manifests is supported for resource [Group: '', Version: 'v1', Kind: 'Event'] in namespace: test-APP2
2024-09-06T18:07:43.240Z  INFO 1 --- [           main] e.o.c.b.k.KubernetesManifestsRemover     : Kubernetes additional manifests is supported for resource [Group: '', Version: 'v1', Kind: 'Event'] in namespace: test-shinyproxy
2024-09-06T18:07:45.412Z  INFO 1 --- [           main] o.s.boot.web.servlet.RegistrationBean    : Filter registration2 was not registered (disabled)
2024-09-06T18:07:45.711Z  INFO 1 --- [           main] o.s.b.a.w.s.WelcomePageHandlerMapping    : Adding welcome page template: index
2024-09-06T18:07:47.194Z  INFO 1 --- [           main] o.s.s.web.DefaultSecurityFilterChain     : Will secure any request with [org.springframework.security.web.session.DisableEncodeUrlFilter@39adf4e6, org.springframework.security.web.session.ForceEagerSessionCreationFilter@73299197, org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter@4ef7145f, org.springframework.security.web.context.SecurityContextHolderFilter@55854382, org.springframework.security.web.header.HeaderWriterFilter@7bf3bf2e, org.springframework.web.filter.CorsFilter@1611ce1c, org.springframework.security.web.csrf.CsrfFilter@3f13720f, org.springframework.security.web.authentication.logout.LogoutFilter@1142843c, org.springframework.security.web.authentication.UsernamePasswordAuthenticationFilter@5d7f4cbb, org.springframework.security.web.authentication.www.BasicAuthenticationFilter@726882da, eu.openanalytics.containerproxy.util.AppRecoveryFilter@106ac5f4, eu.openanalytics.containerproxy.security.UserAgentFilter@40cd02fc, eu.openanalytics.containerproxy.security.FixedRequestCacheAwareFilter@7fd7965b, org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter@77d0a492, org.springframework.security.web.authentication.AnonymousAuthenticationFilter@9742012, org.springframework.security.web.session.SessionManagementFilter@7a458c73, org.springframework.security.web.access.ExceptionTranslationFilter@672c4e24, eu.openanalytics.shinyproxy.AuthenticationRequiredFilter@118d7e0e, org.springframework.security.web.access.intercept.AuthorizationFilter@33f059ad]
2024-09-06T18:07:48.103Z  INFO 1 --- [           main] o.s.l.c.support.AbstractContextSource    : Property 'userDn' not set - anonymous context will be used for read-only operations
2024-09-06T18:07:49.827Z  INFO 1 --- [           main] o.s.i.endpoint.EventDrivenConsumer       : Adding {logging-channel-adapter:_org.springframework.integration.errorLogger} as a subscriber to the 'errorChannel' channel
2024-09-06T18:07:49.828Z  INFO 1 --- [           main] o.s.i.channel.PublishSubscribeChannel    : Channel 'ContainerProxy.errorChannel' has 1 subscriber(s).
2024-09-06T18:07:49.828Z  INFO 1 --- [           main] o.s.i.endpoint.EventDrivenConsumer       : started bean '_org.springframework.integration.errorLogger'
2024-09-06T18:07:49.834Z  INFO 1 --- [           main] io.undertow                              : starting server: Undertow - 2.3.13.Final
2024-09-06T18:07:49.898Z  INFO 1 --- [           main] org.xnio                                 : XNIO version 3.8.8.Final
2024-09-06T18:07:49.908Z  INFO 1 --- [           main] org.xnio.nio                             : XNIO NIO Implementation Version 3.8.8.Final
2024-09-06T18:07:50.021Z  INFO 1 --- [           main] org.jboss.threads                        : JBoss Threads version 3.5.0.Final
2024-09-06T18:07:50.108Z  INFO 1 --- [           main] o.s.b.w.e.undertow.UndertowWebServer     : Undertow started on port 8080 (http)
2024-09-06T18:07:50.301Z  INFO 1 --- [           main] io.undertow.servlet                      : Initializing Spring embedded WebApplicationContext
2024-09-06T18:07:50.301Z  INFO 1 --- [           main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 188 ms
2024-09-06T18:07:50.309Z  INFO 1 --- [           main] o.s.b.a.e.web.EndpointLinksResolver      : Exposing 3 endpoint(s) beneath base path '/actuator'
2024-09-06T18:07:50.400Z  INFO 1 --- [           main] io.undertow                              : starting server: Undertow - 2.3.13.Final
2024-09-06T18:07:50.402Z  INFO 1 --- [           main] o.s.b.w.e.undertow.UndertowWebServer     : Undertow started on port 9090 (http)
2024-09-06T18:07:50.606Z  INFO 1 --- [           main] e.o.c.service.AppRecoveryService         : Recovery of running apps disabled
2024-09-06T18:07:50.606Z  INFO 1 --- [           main] e.o.c.util.StartupEventListener          : Started ShinyProxy 3.1.1 (ContainerProxy 1.1.1)
2024-09-06T18:07:50.790Z  INFO 1 --- [ taskExecutor-1] .o.c.s.l.r.RedisCheckLatestConfigService : This server is running the latest configuration (instanceId: 55ba1652b483cd33b2433ebb35136b980289f0ee, version: 1725646039375), taking part in leader election.
2024-09-06T18:07:50.903Z  INFO 1 --- [ck-leadership-1] e.o.c.s.leader.redis.RedisLeaderService  : This server (runtimeId: lbgh) is now the leader.
2024-09-06T18:08:29.805Z  INFO 1 --- [  XNIO-2 task-2] io.undertow.servlet                      : Initializing Spring DispatcherServlet 'dispatcherServletRegistration'
2024-09-06T18:08:29.805Z  INFO 1 --- [  XNIO-2 task-2] o.s.web.servlet.DispatcherServlet        : Initializing Servlet 'dispatcherServletRegistration'
2024-09-06T18:08:29.806Z  INFO 1 --- [  XNIO-2 task-2] o.s.web.servlet.DispatcherServlet        : Completed initialization in 1 ms
2024-09-06T18:09:10.632Z  INFO 1 --- [GlobalEventLoop] .o.c.s.l.r.RedisCheckLatestConfigService : This server is no longer running the latest configuration (instanceId: 55ba1652b483cd33b2433ebb35136b980289f0ee, version: 1725646039375), no longer taking part in leader election.
2024-09-06T18:09:35.634Z  INFO 1 --- [ck-leadership-1] e.o.c.s.leader.redis.RedisLeaderService  : This server (runtimeId: lbgh) is no longer the leader.

Existing two pods rs-null and rs-0

sp-rs-null


  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::                (v3.2.6)

2024-09-06T18:07:35.995Z  INFO 1 --- [           main] .s.d.r.c.RepositoryConfigurationDelegate : Multiple Spring Data modules found, entering strict repository configuration mode
2024-09-06T18:07:35.999Z  INFO 1 --- [           main] .s.d.r.c.RepositoryConfigurationDelegate : Bootstrapping Spring Data Redis repositories in DEFAULT mode.
2024-09-06T18:07:36.090Z  INFO 1 --- [           main] .s.d.r.c.RepositoryConfigurationDelegate : Finished Spring Data repository scanning in 79 ms. Found 0 Redis repository interfaces.
2024-09-06T18:07:36.222Z  INFO 1 --- [           main] faultConfiguringBeanFactoryPostProcessor : No bean named 'errorChannel' has been explicitly defined. Therefore, a default PublishSubscribeChannel will be created.
2024-09-06T18:07:36.288Z  INFO 1 --- [           main] faultConfiguringBeanFactoryPostProcessor : No bean named 'integrationHeaderChannelRegistry' has been explicitly defined. Therefore, a default DefaultHeaderChannelRegistry will be created.
2024-09-06T18:07:37.409Z  INFO 1 --- [           main] e.o.c.stat.StatCollectorFactory          : Enabled. Sending usage statistics to micrometer.
2024-09-06T18:07:39.124Z  WARN 1 --- [           main] io.undertow.websockets.jsr               : UT026010: Buffer pool was not set on WebSocketDeploymentInfo, the default pool will be used
2024-09-06T18:07:39.199Z  INFO 1 --- [           main] io.undertow.servlet                      : Initializing Spring embedded WebApplicationContext
2024-09-06T18:07:39.200Z  INFO 1 --- [           main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 6708 ms
2024-09-06T18:07:39.514Z  INFO 1 --- [           main] e.o.c.service.IdentifierService          : ShinyProxy runtimeId:                   lbgh
2024-09-06T18:07:39.715Z  INFO 1 --- [           main] e.o.c.service.IdentifierService          : ShinyProxy instanceID (hash of config): 55ba1652b483cd33b2433ebb35136b980289f0ee
2024-09-06T18:07:39.717Z  INFO 1 --- [           main] e.o.c.service.IdentifierService          : ShinyProxy realmId:                     shinyproxy-test-shinyproxy
2024-09-06T18:07:39.717Z  INFO 1 --- [           main] e.o.c.service.IdentifierService          : ShinyProxy version:                     1725646039375
SNIP
2024-09-06T18:07:45.412Z  INFO 1 --- [           main] o.s.boot.web.servlet.RegistrationBean    : Filter registration2 was not registered (disabled)
2024-09-06T18:07:45.711Z  INFO 1 --- [           main] o.s.b.a.w.s.WelcomePageHandlerMapping    : Adding welcome page template: index
boot-3.2.6.jar!/:3.2.6]
	at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:754) ~[spring-boot-3.2.6.jar!/:3.2.6]
	at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:456) ~[spring-boot-3.2.6.jar!/:3.2.6]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:335) ~[spring-boot-3.2.6.jar!/:3.2.6]
	at eu.openanalytics.containerproxy.ContainerProxyApplication.main(ContainerProxyApplication.java:135) ~[containerproxy-1.1.1.jar!/:1.1.1]
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[na:na]
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
	at java.base/java.lang.reflect.Method.invoke(Method.java:568) ~[na:na]
	at org.springframework.boot.loader.launch.Launcher.launch(Launcher.java:91) ~[shinyproxy.jar:3.1.1]
	at org.springframework.boot.loader.launch.Launcher.launch(Launcher.java:53) ~[shinyproxy.jar:3.1.1]
	at org.springframework.boot.loader.launch.JarLauncher.main(JarLauncher.java:58) ~[shinyproxy.jar:3.1.1]

2024-09-06T18:07:47.194Z  INFO 1 --- [           main] o.s.s.web.DefaultSecurityFilterChain     : Will secure any request with [org.springframework.security.web.session.DisableEncodeUrlFilter@39adf4e6, org.springframework.security.web.session.ForceEagerSessionCreationFilter@73299197, org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter@4ef7145f, org.springframework.security.web.context.SecurityContextHolderFilter@55854382, org.springframework.security.web.header.HeaderWriterFilter@7bf3bf2e, org.springframework.web.filter.CorsFilter@1611ce1c, org.springframework.security.web.csrf.CsrfFilter@3f13720f, org.springframework.security.web.authentication.logout.LogoutFilter@1142843c, org.springframework.security.web.authentication.UsernamePasswordAuthenticationFilter@5d7f4cbb, org.springframework.security.web.authentication.www.BasicAuthenticationFilter@726882da, eu.openanalytics.containerproxy.util.AppRecoveryFilter@106ac5f4, eu.openanalytics.containerproxy.security.UserAgentFilter@40cd02fc, eu.openanalytics.containerproxy.security.FixedRequestCacheAwareFilter@7fd7965b, org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter@77d0a492, org.springframework.security.web.authentication.AnonymousAuthenticationFilter@9742012, org.springframework.security.web.session.SessionManagementFilter@7a458c73, org.springframework.security.web.access.ExceptionTranslationFilter@672c4e24, eu.openanalytics.shinyproxy.AuthenticationRequiredFilter@118d7e0e, org.springframework.security.web.access.intercept.AuthorizationFilter@33f059ad]
2024-09-06T18:07:48.103Z  INFO 1 --- [           main] o.s.l.c.support.AbstractContextSource    : Property 'userDn' not set - anonymous context will be used for read-only operations
2024-09-06T18:07:49.827Z  INFO 1 --- [           main] o.s.i.endpoint.EventDrivenConsumer       : Adding {logging-channel-adapter:_org.springframework.integration.errorLogger} as a subscriber to the 'errorChannel' channel
2024-09-06T18:07:49.828Z  INFO 1 --- [           main] o.s.i.channel.PublishSubscribeChannel    : Channel 'ContainerProxy.errorChannel' has 1 subscriber(s).
2024-09-06T18:07:49.828Z  INFO 1 --- [           main] o.s.i.endpoint.EventDrivenConsumer       : started bean '_org.springframework.integration.errorLogger'
2024-09-06T18:07:49.834Z  INFO 1 --- [           main] io.undertow                              : starting server: Undertow - 2.3.13.Final
2024-09-06T18:07:49.898Z  INFO 1 --- [           main] org.xnio                                 : XNIO version 3.8.8.Final
2024-09-06T18:07:49.908Z  INFO 1 --- [           main] org.xnio.nio                             : XNIO NIO Implementation Version 3.8.8.Final
2024-09-06T18:07:50.021Z  INFO 1 --- [           main] org.jboss.threads                        : JBoss Threads version 3.5.0.Final
2024-09-06T18:07:50.108Z  INFO 1 --- [           main] o.s.b.w.e.undertow.UndertowWebServer     : Undertow started on port 8080 (http)
2024-09-06T18:07:50.301Z  INFO 1 --- [           main] io.undertow.servlet                      : Initializing Spring embedded WebApplicationContext
2024-09-06T18:07:50.301Z  INFO 1 --- [           main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 188 ms
2024-09-06T18:07:50.309Z  INFO 1 --- [           main] o.s.b.a.e.web.EndpointLinksResolver      : Exposing 3 endpoint(s) beneath base path '/actuator'
2024-09-06T18:07:50.400Z  INFO 1 --- [           main] io.undertow                              : starting server: Undertow - 2.3.13.Final
2024-09-06T18:07:50.402Z  INFO 1 --- [           main] o.s.b.w.e.undertow.UndertowWebServer     : Undertow started on port 9090 (http)
2024-09-06T18:07:50.606Z  INFO 1 --- [           main] e.o.c.service.AppRecoveryService         : Recovery of running apps disabled
2024-09-06T18:07:50.606Z  INFO 1 --- [           main] e.o.c.util.StartupEventListener          : Started ShinyProxy 3.1.1 (ContainerProxy 1.1.1)
2024-09-06T18:07:50.790Z  INFO 1 --- [ taskExecutor-1] .o.c.s.l.r.RedisCheckLatestConfigService : This server is running the latest configuration (instanceId: 55ba1652b483cd33b2433ebb35136b980289f0ee, version: 1725646039375), taking part in leader election.
2024-09-06T18:07:50.903Z  INFO 1 --- [ck-leadership-1] e.o.c.s.leader.redis.RedisLeaderService  : This server (runtimeId: lbgh) is now the leader.
2024-09-06T18:08:29.805Z  INFO 1 --- [  XNIO-2 task-2] io.undertow.servlet                      : Initializing Spring DispatcherServlet 'dispatcherServletRegistration'
2024-09-06T18:08:29.805Z  INFO 1 --- [  XNIO-2 task-2] o.s.web.servlet.DispatcherServlet        : Initializing Servlet 'dispatcherServletRegistration'
2024-09-06T18:08:29.806Z  INFO 1 --- [  XNIO-2 task-2] o.s.web.servlet.DispatcherServlet        : Completed initialization in 1 ms
2024-09-06T18:09:10.632Z  INFO 1 --- [GlobalEventLoop] .o.c.s.l.r.RedisCheckLatestConfigService : This server is no longer running the latest configuration (instanceId: 55ba1652b483cd33b2433ebb35136b980289f0ee, version: 1725646039375), no longer taking part in leader election.
2024-09-06T18:09:35.634Z  INFO 1 --- [ck-leadership-1] e.o.c.s.leader.redis.RedisLeaderService  : This server (runtimeId: lbgh) is no longer the leader.

rs-0 has a massive log. I can provide this if needed. Likely similar to others just errors from actuator:

2024-09-06T19:21:28.532Z ERROR 1 --- [  XNIO-2 task-6] io.undertow.request                      : UT005023: Exception handling request to /actuator/health/readiness

java.lang.NullPointerException: null

Spent decent bit of time trying to resolve this but cannot figure out a solution. Besides the odd rs-null and actuator errors apps seem to be running fine. Though I previously had issues with SAML I am using simple authentication now. Will debug SAML at a later time.

Lukeesec avatar Sep 06 '24 19:09 Lukeesec

Hi @Lukeesec I never saw this behavior before, but here are a few idea that may help:

  • did you update the ShinyProxy CRD: https://github.com/openanalytics/shinyproxy-operator?tab=readme-ov-file#update-to-210 ?
  • during the update, the operator may temporarly create a pod with -rs-null suffix, this should not be the cause of your issues
  • you mentioned you tried updating by deploying in a new namespace, this is fine as long as the operator is running in namespaced mode (and as long as the operator is in the same namespace as ShinyProxy). If you are running the operator in clustered mode, you should not run multiple instances of the operator. This will cause very strange behavior.
  • if you are using clustered mode, you can upgrade the operator, even if you have older versions of ShinyProxy running in another namespace. The current version of the operator is compatible with version 3.0.0 of ShinyProxy.

LEDfan avatar Sep 12 '24 10:09 LEDfan

I am sorry for the delay! I saw this and forgot. @LEDfan.

  • Yes the CRD was updated
  • The rs-null never went away
  • We have a DEV and PROD deployment in namespaced mode

I did a fresh deployment in a new namespace, I also upgraded a namespace too. Tried quite a few different ways.

Lukeesec avatar Sep 25 '24 13:09 Lukeesec

Hi. Just bumping this. Thanks!

Lukeesec avatar Oct 10 '24 18:10 Lukeesec

Hi @Lukeesec

Can you provide the logs of ShinyProxy operator? It seems you posted logs of shinyproxy instead of the operator.

LEDfan avatar Oct 28 '24 13:10 LEDfan

Resolved. Due to a weird issue with CRDS that caused issues with multiple namespaced deployments.

Lukeesec avatar Nov 29 '24 15:11 Lukeesec