che
che copied to clipboard
eclipse-che
Deploying Eclipse Che on IPv6-only kubernetes cluster
I'm forcing a ipv6-only environment with kubernetes and calico. Here are some notes about the problems I'm having:
this is the setup commands and configs used
postgres pod doesn't work unless it has this line in "cat /data/userdata/pg_hba.conf" :
host all all ::1/128 trust
events in cluster without the v6 line:
eclipse-che 60m Warning Unhealthy pod/postgres-845fff6947-jhbq5 Liveness probe failed: dial tcp [1100:200::d9:796e]:5432: connect: connection refused
eclipse-che 60m Warning Unhealthy pod/postgres-845fff6947-jhbq5 Readiness probe failed: sh: cannot set terminal process group (-1): Inappropriate ioctl for device
sh: no job control in this shell
psql: could not connect to server: Connection refused
Is the server running on host "127.0.0.1" and accepting
TCP/IP connections on port 5432?
eclipse-che 60m Normal Killing pod/postgres-845fff6947-jhbq5 Container postgres failed liveness probe, will be restarted
then keycloack is all-around hardcoded with the "0.0.0.0" cidr, like here and here
relevant logs from the keycloak pod
Certificate was added to keystore
Certificate was added to keystore
Importing keystore /etc/pki/ca-trust/extracted/java/cacerts to /scripts/openshift.jks...
Entry for alias digicertassuredidrootca successfully imported.
Entry for alias trktrustelektroniksertifikahizmetsalaycsh5 successfully imported.
Entry for alias affirmtrustcommercial successfully imported.
Entry for alias t-telesecglobalrootclass3 successfully imported.
Entry for alias certinomis-rootca successfully imported.
Entry for alias t-telesecglobalrootclass2 successfully imported.
Entry for alias comodoecccertificationauthority successfully imported.
Entry for alias swisssignsilverca-g2 successfully imported.
Entry for alias cadisigrootr2 successfully imported.
Entry for alias securetrustca successfully imported.
Entry for alias accvraiz1 successfully imported.
Entry for alias staatdernederlandenrootca-g3 successfully imported.
Entry for alias staatdernederlandenrootca-g2 successfully imported.
Entry for alias entrustrootcertificationauthority successfully imported.
Entry for alias identrustpublicsectorrootca1 successfully imported.
Entry for alias entrust.netpremium2048secureserverca successfully imported.
Entry for alias secureglobalca successfully imported.
Entry for alias opentrustrootcag3 successfully imported.
Entry for alias netlockarany(classgold)ftanstvny successfully imported.
Entry for alias eecertificationcentrerootca successfully imported.
Entry for alias teliasonerarootcav1 successfully imported.
Entry for alias opentrustrootcag2 successfully imported.
Entry for alias autoridaddecertificacionfirmaprofesionalcifa62634068 successfully imported.
Entry for alias opentrustrootcag1 successfully imported.
Entry for alias acraizfnmt-rcm successfully imported.
Entry for alias gdcatrustauthr5root successfully imported.
Entry for alias izenpe.com successfully imported.
Entry for alias e-tugracertificationauthority successfully imported.
Entry for alias quovadisrootca3 successfully imported.
Entry for alias quovadisrootca2 successfully imported.
Entry for alias entrustrootcertificationauthority-ec1 successfully imported.
Entry for alias oistewisekeyglobalrootgbca successfully imported.
Entry for alias addtrustexternalroot successfully imported.
Entry for alias digicertglobalrootg3 successfully imported.
Entry for alias swisssigngoldca-g2 successfully imported.
Entry for alias comodoaaaservicesroot successfully imported.
Entry for alias digicertglobalrootg2 successfully imported.
Entry for alias oistewisekeyglobalrootgaca successfully imported.
Entry for alias dstrootcax3 successfully imported.
Entry for alias certigna successfully imported.
Entry for alias digicerthighassuranceevrootca successfully imported.
Entry for alias chambersofcommerceroot-2008 successfully imported.
Entry for alias soneraclass2rootca successfully imported.
Entry for alias usertrustrsacertificationauthority successfully imported.
Entry for alias geotrustuniversalca successfully imported.
Entry for alias certsignrootca successfully imported.
Entry for alias amazonrootca4 successfully imported.
Entry for alias amazonrootca3 successfully imported.
Entry for alias amazonrootca2 successfully imported.
Entry for alias verisignuniversalrootcertificationauthority successfully imported.
Entry for alias trustcorrootcertca-2 successfully imported.
Entry for alias amazonrootca1 successfully imported.
Entry for alias trustcorrootcertca-1 successfully imported.
Entry for alias ssl.comrootcertificationauthorityecc successfully imported.
Entry for alias ssl.comrootcertificationauthorityrsa successfully imported.
Entry for alias d-trustrootclass3ca2ev2009 successfully imported.
Entry for alias networksolutionscertificateauthority successfully imported.
Entry for alias affirmtrustnetworking successfully imported.
Entry for alias deutschetelekomrootca2 successfully imported.
Entry for alias globalsigneccrootca-r5 successfully imported.
Entry for alias globalsigneccrootca-r4 successfully imported.
Entry for alias szafirrootca2 successfully imported.
Entry for alias globalsignrootca-r3 successfully imported.
Entry for alias globalsignrootca-r2 successfully imported.
Entry for alias buypassclass3rootca successfully imported.
Entry for alias comodorsacertificationauthority successfully imported.
Entry for alias securitycommunicationrootca2 successfully imported.
Entry for alias starfieldclass2ca successfully imported.
Entry for alias actalisauthenticationrootca successfully imported.
Entry for alias cfcaevroot successfully imported.
Entry for alias digicerttrustedrootg4 successfully imported.
Entry for alias certumtrustednetworkca2 successfully imported.
Entry for alias entrustrootcertificationauthority-g2 successfully imported.
Entry for alias taiwangrca successfully imported.
Entry for alias hellenicacademicandresearchinstitutionseccrootca2015 successfully imported.
Entry for alias twcarootcertificationauthority successfully imported.
Entry for alias certplusrootcag2 successfully imported.
Entry for alias twcaglobalrootca successfully imported.
Entry for alias certplusrootcag1 successfully imported.
Entry for alias geotrustuniversalca2 successfully imported.
Entry for alias thawteprimaryrootca-g3 successfully imported.
Entry for alias thawteprimaryrootca-g2 successfully imported.
Entry for alias baltimorecybertrustroot successfully imported.
Entry for alias buypassclass2rootca successfully imported.
Entry for alias digicertassuredidrootg3 successfully imported.
Entry for alias certumtrustednetworkca successfully imported.
Entry for alias geotrustprimarycertificationauthority-g3 successfully imported.
Entry for alias digicertassuredidrootg2 successfully imported.
Entry for alias geotrustprimarycertificationauthority-g2 successfully imported.
Entry for alias isrgrootx1 successfully imported.
Entry for alias ec-acc successfully imported.
Entry for alias ssl.comevrootcertificationauthorityecc successfully imported.
Entry for alias certplusclass2primaryca successfully imported.
Entry for alias globalchambersignroot-2008 successfully imported.
Entry for alias digicertglobalrootca successfully imported.
Entry for alias d-trustrootclass3ca22009 successfully imported.
Entry for alias starfieldservicesrootcertificateauthority-g2 successfully imported.
Entry for alias thawteprimaryrootca successfully imported.
Entry for alias atostrustedroot2011 successfully imported.
Entry for alias luxtrustglobalroot2 successfully imported.
Entry for alias geotrustglobalca successfully imported.
Entry for alias visaecommerceroot successfully imported.
Entry for alias quovadisrootca successfully imported.
Entry for alias identrustcommercialrootca1 successfully imported.
Entry for alias staatdernederlandenevrootca successfully imported.
Entry for alias tubitakkamusmsslkoksertifikasi-surum1 successfully imported.
Entry for alias trustcoreca-1 successfully imported.
Entry for alias securitycommunicationrootca successfully imported.
Entry for alias comodocertificationauthority successfully imported.
Entry for alias verisignclass3publicprimarycertificationauthority-g5 successfully imported.
Entry for alias xrampglobalcaroot successfully imported.
Entry for alias verisignclass3publicprimarycertificationauthority-g4 successfully imported.
Entry for alias quovadisrootca3g3 successfully imported.
Entry for alias verisignclass3publicprimarycertificationauthority-g3 successfully imported.
Entry for alias securesignrootca11 successfully imported.
Entry for alias affirmtrustpremium successfully imported.
Entry for alias globalsignrootca successfully imported.
Entry for alias quovadisrootca2g3 successfully imported.
Entry for alias geotrustprimarycertificationauthority successfully imported.
Entry for alias affirmtrustpremiumecc successfully imported.
Entry for alias quovadisrootca1g3 successfully imported.
Entry for alias hongkongpostrootca1 successfully imported.
Entry for alias usertrustecccertificationauthority successfully imported.
Entry for alias cybertrustglobalroot successfully imported.
Entry for alias godaddyclass2ca successfully imported.
Entry for alias microsece-szignorootca2009 successfully imported.
Entry for alias hellenicacademicandresearchinstitutionsrootca2015 successfully imported.
Entry for alias hellenicacademicandresearchinstitutionsrootca2011 successfully imported.
Entry for alias godaddyrootcertificateauthority-g2 successfully imported.
Entry for alias trustisfpsrootca successfully imported.
Entry for alias epkirootcertificationauthority successfully imported.
Entry for alias starfieldrootcertificateauthority-g2 successfully imported.
Entry for alias ssl.comevrootcertificationauthorityrsar2 successfully imported.
Import command completed: 133 entries successfully imported, 0 entries failed or cancelled
Installing certificates into Keycloak
14:22:05,730 INFO [org.jboss.modules] (CLI command executor) JBoss Modules version 1.9.0.Final
14:22:05,774 INFO [org.jboss.msc] (CLI command executor) JBoss MSC version 1.4.5.Final
14:22:05,798 INFO [org.jboss.threads] (CLI command executor) JBoss Threads version 2.3.3.Final
14:22:05,894 INFO [org.jboss.as] (MSC service thread 1-3) WFLYSRV0049: Keycloak 6.0.1 (WildFly Core 8.0.0.Final) starting
14:22:06,534 INFO [org.wildfly.security] (ServerService Thread Pool -- 4) ELY00001: WildFly Elytron version 1.8.0.Final
14:22:06,968 INFO [org.jboss.as.controller.management-deprecated] (Controller Boot Thread) WFLYCTL0028: Attribute 'security-realm' in the resource at address '/core-service=management/management-interface=http-interface' is deprecated, and may be removed in a future version. See the attribute description in the output of the read-resource-description operation to learn more about the deprecation.
14:22:07,034 INFO [org.jboss.as.controller.management-deprecated] (Controller Boot Thread) WFLYCTL0028: Attribute 'security-realm' in the resource at address '/subsystem=undertow/server=default-server/https-listener=https' is deprecated, and may be removed in a future version. See the attribute description in the output of the read-resource-description operation to learn more about the deprecation.
14:22:07,190 INFO [org.jboss.as.patching] (MSC service thread 1-1) WFLYPAT0050: Keycloak cumulative patch ID is: base, one-off patches include: none
14:22:07,205 WARN [org.jboss.as.domain.management.security] (MSC service thread 1-3) WFLYDM0111: Keystore /opt/jboss/keycloak/standalone/configuration/application.keystore not found, it will be auto generated on first use with a self signed certificate for host localhost
14:22:07,247 INFO [org.jboss.as.server] (Controller Boot Thread) WFLYSRV0212: Resuming server
14:22:07,248 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0025: Keycloak 6.0.1 (WildFly Core 8.0.0.Final) started in 1514ms - Started 64 of 78 services (25 services are lazy, passive or on-demand)
{"outcome" => "success"}
{"outcome" => "success"}
14:22:07,393 INFO [org.jboss.as] (MSC service thread 1-3) WFLYSRV0050: Keycloak 6.0.1 (WildFly Core 8.0.0.Final) stopped in 45ms
Use fixed hostname provider to make working internal network requests
14:22:08,154 INFO [org.jboss.modules] (CLI command executor) JBoss Modules version 1.9.0.Final
14:22:08,196 INFO [org.jboss.msc] (CLI command executor) JBoss MSC version 1.4.5.Final
14:22:08,239 INFO [org.jboss.threads] (CLI command executor) JBoss Threads version 2.3.3.Final
14:22:08,326 INFO [org.jboss.as] (MSC service thread 1-3) WFLYSRV0049: Keycloak 6.0.1 (WildFly Core 8.0.0.Final) starting
14:22:08,939 INFO [org.wildfly.security] (ServerService Thread Pool -- 4) ELY00001: WildFly Elytron version 1.8.0.Final
14:22:09,389 INFO [org.jboss.as.controller.management-deprecated] (Controller Boot Thread) WFLYCTL0028: Attribute 'security-realm' in the resource at address '/core-service=management/management-interface=http-interface' is deprecated, and may be removed in a future version. See the attribute description in the output of the read-resource-description operation to learn more about the deprecation.
14:22:09,461 INFO [org.jboss.as.controller.management-deprecated] (Controller Boot Thread) WFLYCTL0028: Attribute 'security-realm' in the resource at address '/subsystem=undertow/server=default-server/https-listener=https' is deprecated, and may be removed in a future version. See the attribute description in the output of the read-resource-description operation to learn more about the deprecation.
14:22:09,617 INFO [org.jboss.as.patching] (MSC service thread 1-1) WFLYPAT0050: Keycloak cumulative patch ID is: base, one-off patches include: none
14:22:09,619 WARN [org.jboss.as.domain.management.security] (MSC service thread 1-4) WFLYDM0111: Keystore /opt/jboss/keycloak/standalone/configuration/application.keystore not found, it will be auto generated on first use with a self signed certificate for host localhost
14:22:09,666 INFO [org.jboss.as.server] (Controller Boot Thread) WFLYSRV0212: Resuming server
14:22:09,667 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0025: Keycloak 6.0.1 (WildFly Core 8.0.0.Final) started in 1509ms - Started 64 of 78 services (25 services are lazy, passive or on-demand)
{"outcome" => "success"}
{"outcome" => "success"}
{"outcome" => "success"}
{"outcome" => "success"}
14:22:09,857 INFO [org.jboss.as] (MSC service thread 1-4) WFLYSRV0050: Keycloak 6.0.1 (WildFly Core 8.0.0.Final) stopped in 41ms
Added 'admin' to '/opt/jboss/keycloak/standalone/configuration/keycloak-add-user.json', restart server to load user
WARNING: POSTGRES_ADDR variable name is DEPRECATED replace with DB_ADDR
WARNING: POSTGRES_DATABASE variable name is DEPRECATED replace with DB_DATABASE
WARNING: POSTGRES_USER variable name is DEPRECATED replace with DB_USER
WARNING: POSTGRES_PASSWORD variable name is DEPRECATED replace with DB_PASSWORD
WARNING: POSTGRES_PORT variable name is DEPRECATED replace with DB_PORT
=========================================================================
Using PostgreSQL database
=========================================================================
14:22:11,696 INFO [org.jboss.modules] (CLI command executor) JBoss Modules version 1.9.0.Final
14:22:11,748 INFO [org.jboss.msc] (CLI command executor) JBoss MSC version 1.4.5.Final
14:22:11,772 INFO [org.jboss.threads] (CLI command executor) JBoss Threads version 2.3.3.Final
14:22:11,856 INFO [org.jboss.as] (MSC service thread 1-2) WFLYSRV0049: Keycloak 6.0.1 (WildFly Core 8.0.0.Final) starting
14:22:12,490 INFO [org.wildfly.security] (ServerService Thread Pool -- 17) ELY00001: WildFly Elytron version 1.8.0.Final
14:22:12,954 INFO [org.jboss.as.controller.management-deprecated] (Controller Boot Thread) WFLYCTL0028: Attribute 'security-realm' in the resource at address '/core-service=management/management-interface=http-interface' is deprecated, and may be removed in a future version. See the attribute description in the output of the read-resource-description operation to learn more about the deprecation.
14:22:13,024 INFO [org.jboss.as.controller.management-deprecated] (Controller Boot Thread) WFLYCTL0028: Attribute 'security-realm' in the resource at address '/subsystem=undertow/server=default-server/https-listener=https' is deprecated, and may be removed in a future version. See the attribute description in the output of the read-resource-description operation to learn more about the deprecation.
14:22:13,138 INFO [org.jboss.as.patching] (MSC service thread 1-2) WFLYPAT0050: Keycloak cumulative patch ID is: base, one-off patches include: none
14:22:13,145 WARN [org.jboss.as.domain.management.security] (MSC service thread 1-3) WFLYDM0111: Keystore /opt/jboss/keycloak/standalone/configuration/application.keystore not found, it will be auto generated on first use with a self signed certificate for host localhost
14:22:13,224 INFO [org.jboss.as.server] (Controller Boot Thread) WFLYSRV0212: Resuming server
14:22:13,224 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0025: Keycloak 6.0.1 (WildFly Core 8.0.0.Final) started in 1525ms - Started 64 of 78 services (25 services are lazy, passive or on-demand)
The batch executed successfully
14:22:13,340 INFO [org.jboss.as] (MSC service thread 1-3) WFLYSRV0050: Keycloak 6.0.1 (WildFly Core 8.0.0.Final) stopped in 8ms
14:22:14,096 INFO [org.jboss.modules] (CLI command executor) JBoss Modules version 1.9.0.Final
14:22:14,148 INFO [org.jboss.msc] (CLI command executor) JBoss MSC version 1.4.5.Final
14:22:14,153 INFO [org.jboss.threads] (CLI command executor) JBoss Threads version 2.3.3.Final
14:22:14,279 INFO [org.jboss.as] (MSC service thread 1-3) WFLYSRV0049: Keycloak 6.0.1 (WildFly Core 8.0.0.Final) starting
14:22:14,958 INFO [org.wildfly.security] (ServerService Thread Pool -- 13) ELY00001: WildFly Elytron version 1.8.0.Final
14:22:15,576 INFO [org.jboss.as.controller.management-deprecated] (Controller Boot Thread) WFLYCTL0028: Attribute 'security-realm' in the resource at address '/core-service=management/management-interface=http-interface' is deprecated, and may be removed in a future version. See the attribute description in the output of the read-resource-description operation to learn more about the deprecation.
14:22:15,703 INFO [org.jboss.as.controller.management-deprecated] (Controller Boot Thread) WFLYCTL0028: Attribute 'security-realm' in the resource at address '/subsystem=undertow/server=default-server/https-listener=https' is deprecated, and may be removed in a future version. See the attribute description in the output of the read-resource-description operation to learn more about the deprecation.
14:22:15,836 INFO [org.jboss.as.patching] (MSC service thread 1-2) WFLYPAT0050: Keycloak cumulative patch ID is: base, one-off patches include: none
14:22:15,846 WARN [org.jboss.as.domain.management.security] (MSC service thread 1-3) WFLYDM0111: Keystore /opt/jboss/keycloak/standalone/configuration/application.keystore not found, it will be auto generated on first use with a self signed certificate for host localhost
14:22:15,919 INFO [org.jboss.as.server] (Controller Boot Thread) WFLYSRV0212: Resuming server
14:22:15,920 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0025: Keycloak 6.0.1 (WildFly Core 8.0.0.Final) started in 1821ms - Started 64 of 83 services (30 services are lazy, passive or on-demand)
The batch executed successfully
14:22:16,036 INFO [org.jboss.as] (MSC service thread 1-2) WFLYSRV0050: Keycloak 6.0.1 (WildFly Core 8.0.0.Final) stopped in 8ms
=========================================================================
JBoss Bootstrap Environment
JBOSS_HOME: /opt/jboss/keycloak
JAVA: /usr/lib/jvm/java/bin/java
JAVA_OPTS: -server -Xms64m -Xmx512m -XX:MetaspaceSize=96M -XX:MaxMetaspaceSize=256m -Djava.net.preferIPv4Stack=true -Djboss.modules.system.pkgs=org.jboss.byteman -Djava.awt.headless=true -agentlib:jdwp=transport=dt_socket,address=8787,server=y,suspend=n
=========================================================================
Listening for transport dt_socket at address: 8787
14:22:16,394 INFO [org.jboss.modules] (main) JBoss Modules version 1.9.0.Final
14:22:17,577 INFO [org.jboss.msc] (main) JBoss MSC version 1.4.5.Final
14:22:17,584 INFO [org.jboss.threads] (main) JBoss Threads version 2.3.3.Final
14:22:17,689 INFO [org.jboss.as] (MSC service thread 1-1) WFLYSRV0049: Keycloak 6.0.1 (WildFly Core 8.0.0.Final) starting
14:22:18,353 INFO [org.wildfly.security] (ServerService Thread Pool -- 15) ELY00001: WildFly Elytron version 1.8.0.Final
14:22:18,839 INFO [org.jboss.as.controller.management-deprecated] (Controller Boot Thread) WFLYCTL0028: Attribute 'security-realm' in the resource at address '/core-service=management/management-interface=http-interface' is deprecated, and may be removed in a future version. See the attribute description in the output of the read-resource-description operation to learn more about the deprecation.
14:22:18,878 INFO [org.jboss.as.repository] (ServerService Thread Pool -- 14) WFLYDR0001: Content added at location /opt/jboss/keycloak/standalone/data/content/43/baa2f4a0b4a5ddd4bce681fd554f4676ad7892/content
14:22:18,928 INFO [org.jboss.as.controller.management-deprecated] (ServerService Thread Pool -- 27) WFLYCTL0028: Attribute 'security-realm' in the resource at address '/subsystem=undertow/server=default-server/https-listener=https' is deprecated, and may be removed in a future version. See the attribute description in the output of the read-resource-description operation to learn more about the deprecation.
14:22:18,968 INFO [org.jboss.as.server] (Controller Boot Thread) WFLYSRV0039: Creating http management service using socket-binding (management-http)
14:22:18,978 INFO [org.xnio] (MSC service thread 1-3) XNIO version 3.6.5.Final
14:22:18,983 INFO [org.xnio.nio] (MSC service thread 1-3) XNIO NIO Implementation Version 3.6.5.Final
14:22:19,040 INFO [org.jboss.remoting] (MSC service thread 1-2) JBoss Remoting version 5.0.8.Final
14:22:19,062 INFO [org.jboss.as.connector.subsystems.datasources] (ServerService Thread Pool -- 31) WFLYJCA0004: Deploying JDBC-compliant driver class org.h2.Driver (version 1.4)
14:22:19,080 INFO [org.jboss.as.connector.subsystems.datasources] (ServerService Thread Pool -- 31) WFLYJCA0005: Deploying non-JDBC-compliant driver class org.postgresql.Driver (version 42.2)
14:22:19,089 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 36) WFLYCLINF0001: Activating Infinispan subsystem.
14:22:19,125 INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-4) WFLYJCA0010: Unbound data source [java:jboss/datasources/ExampleDS]
14:22:19,138 INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-2) WFLYJCA0010: Unbound data source [java:jboss/datasources/KeycloakDS]
14:22:19,139 INFO [org.wildfly.extension.io] (ServerService Thread Pool -- 37) WFLYIO001: Worker 'default' has auto-configured to 4 core threads with 32 task threads based on your 2 available processors
14:22:19,149 INFO [org.jboss.as.jaxrs] (ServerService Thread Pool -- 38) WFLYRS0016: RESTEasy version 3.6.3.Final
14:22:19,152 INFO [org.wildfly.extension.microprofile.config.smallrye._private] (ServerService Thread Pool -- 44) WFLYCONF0001: Activating WildFly MicroProfile Config Subsystem
14:22:19,172 INFO [org.wildfly.extension.microprofile.metrics.smallrye] (ServerService Thread Pool -- 46) WFLYMETRICS0001: Activating Eclipse MicroProfile Metrics Subsystem
14:22:19,173 INFO [org.wildfly.extension.microprofile.health.smallrye] (ServerService Thread Pool -- 45) WFLYHEALTH0001: Activating Eclipse MicroProfile Health Subsystem
14:22:19,216 INFO [org.jboss.as.connector] (MSC service thread 1-4) WFLYJCA0009: Starting JCA Subsystem (WildFly/IronJacamar 1.4.12.Final)
14:22:19,217 INFO [org.jboss.as.mail.extension] (MSC service thread 1-4) WFLYMAIL0002: Unbound mail session [java:jboss/mail/Default]
14:22:19,229 INFO [org.jboss.as.naming] (ServerService Thread Pool -- 47) WFLYNAM0001: Activating Naming Subsystem
14:22:19,230 INFO [org.jboss.as.security] (ServerService Thread Pool -- 50) WFLYSEC0002: Activating Security Subsystem
14:22:19,232 INFO [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-4) WFLYJCA0018: Started Driver service with driver-name = h2
14:22:19,233 INFO [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-4) WFLYJCA0018: Started Driver service with driver-name = postgresql
14:22:19,268 INFO [io.smallrye.metrics] (MSC service thread 1-3) Converted [2] config entries and added [4] replacements
14:22:19,270 INFO [io.smallrye.metrics] (MSC service thread 1-3) Converted [3] config entries and added [14] replacements
14:22:19,273 INFO [org.jboss.as.security] (MSC service thread 1-1) WFLYSEC0001: Current PicketBox version=5.0.3.Final
14:22:19,277 WARN [org.jboss.as.txn] (ServerService Thread Pool -- 52) WFLYTX0013: The node-identifier attribute on the /subsystem=transactions is set to the default value. This is a danger for environments running multiple servers. Please make sure the attribute value is unique.
14:22:19,282 INFO [org.wildfly.extension.undertow] (MSC service thread 1-3) WFLYUT0003: Undertow 2.0.19.Final starting
14:22:19,347 INFO [org.jboss.as.naming] (MSC service thread 1-1) WFLYNAM0003: Starting Naming Service
14:22:19,348 INFO [org.jboss.as.mail.extension] (MSC service thread 1-1) WFLYMAIL0001: Bound mail session [java:jboss/mail/Default]
14:22:19,524 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 53) WFLYUT0014: Creating file handler for path '/opt/jboss/keycloak/welcome-content' with options [directory-listing: 'false', follow-symlink: 'false', case-sensitive: 'true', safe-symlink-paths: '[]']
14:22:19,540 INFO [org.jboss.as.ejb3] (MSC service thread 1-2) WFLYEJB0482: Strict pool mdb-strict-max-pool is using a max instance size of 8 (per class), which is derived from the number of CPUs on this host.
14:22:19,540 INFO [org.jboss.as.ejb3] (MSC service thread 1-3) WFLYEJB0481: Strict pool slsb-strict-max-pool is using a max instance size of 32 (per class), which is derived from thread worker pool sizing.
14:22:19,658 INFO [org.wildfly.extension.undertow] (MSC service thread 1-2) WFLYUT0012: Started server default-server.
14:22:19,663 INFO [org.wildfly.extension.undertow] (MSC service thread 1-2) WFLYUT0018: Host default-host starting
14:22:19,743 INFO [org.wildfly.extension.undertow] (MSC service thread 1-1) WFLYUT0006: Undertow HTTP listener default listening on 0.0.0.0:8080
14:22:19,846 INFO [org.jboss.as.patching] (MSC service thread 1-1) WFLYPAT0050: Keycloak cumulative patch ID is: base, one-off patches include: none
14:22:19,857 WARN [org.jboss.as.domain.management.security] (MSC service thread 1-2) WFLYDM0111: Keystore /opt/jboss/keycloak/standalone/configuration/application.keystore not found, it will be auto generated on first use with a self signed certificate for host localhost
14:22:19,862 INFO [org.jboss.as.server.deployment] (MSC service thread 1-3) WFLYSRV0027: Starting deployment of "openshift4-extension-6.0.1.jar" (runtime-name: "openshift4-extension-6.0.1.jar")
14:22:19,869 INFO [org.jboss.as.server.deployment.scanner] (MSC service thread 1-1) WFLYDS0013: Started FileSystemDeploymentService for directory /opt/jboss/keycloak/standalone/deployments
14:22:19,869 INFO [org.jboss.as.server.deployment] (MSC service thread 1-3) WFLYSRV0027: Starting deployment of "keycloak-server.war" (runtime-name: "keycloak-server.war")
14:22:19,952 INFO [org.wildfly.extension.undertow] (MSC service thread 1-3) WFLYUT0006: Undertow HTTPS listener https listening on 0.0.0.0:8443
14:22:20,739 INFO [org.jboss.as.ejb3] (MSC service thread 1-3) WFLYEJB0493: EJB subsystem suspension complete
14:22:20,791 WARN [org.jboss.as.dependency.private] (MSC service thread 1-4) WFLYSRV0018: Deployment "deployment.openshift4-extension-6.0.1.jar" is using a private module ("org.keycloak.keycloak-server-spi-private") which may be changed or removed in future versions without notice.
14:22:20,791 WARN [org.jboss.as.dependency.private] (MSC service thread 1-4) WFLYSRV0018: Deployment "deployment.openshift4-extension-6.0.1.jar" is using a private module ("org.keycloak.keycloak-services") which may be changed or removed in future versions without notice.
14:22:20,817 INFO [org.keycloak.subsystem.server.extension.KeycloakProviderDeploymentProcessor] (MSC service thread 1-4) Deploying Keycloak provider: openshift4-extension-6.0.1.jar
14:22:20,822 INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-2) WFLYJCA0001: Bound data source [java:jboss/datasources/ExampleDS]
14:22:20,823 INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-3) WFLYJCA0001: Bound data source [java:jboss/datasources/KeycloakDS]
14:22:20,949 WARN [org.jboss.as.dependency.private] (MSC service thread 1-1) WFLYSRV0018: Deployment "deployment.keycloak-server.war" is using a private module ("org.kie") which may be changed or removed in future versions without notice.
14:22:21,250 INFO [org.infinispan.factories.GlobalComponentRegistry] (MSC service thread 1-1) ISPN000128: Infinispan version: Infinispan 'Infinity Minus ONE +2' 9.4.8.Final
14:22:21,656 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 54) WFLYCLINF0002: Started realms cache from keycloak container
14:22:21,658 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 64) WFLYCLINF0002: Started keys cache from keycloak container
14:22:21,659 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 62) WFLYCLINF0002: Started authorization cache from keycloak container
14:22:21,659 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 60) WFLYCLINF0002: Started users cache from keycloak container
14:22:21,659 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 61) WFLYCLINF0002: Started work cache from keycloak container
14:22:21,659 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 55) WFLYCLINF0002: Started authenticationSessions cache from keycloak container
14:22:21,660 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 57) WFLYCLINF0002: Started clientSessions cache from keycloak container
14:22:21,660 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 63) WFLYCLINF0002: Started actionTokens cache from keycloak container
14:22:21,660 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 59) WFLYCLINF0002: Started offlineSessions cache from keycloak container
14:22:21,659 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 56) WFLYCLINF0002: Started sessions cache from keycloak container
14:22:21,659 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 58) WFLYCLINF0002: Started loginFailures cache from keycloak container
14:22:21,660 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 65) WFLYCLINF0002: Started offlineClientSessions cache from keycloak container
14:22:21,731 WARN [org.jboss.as.server.deployment] (MSC service thread 1-1) WFLYSRV0273: Excluded subsystem weld via jboss-deployment-structure.xml does not exist.
14:22:21,732 WARN [org.jboss.as.server.deployment] (MSC service thread 1-1) WFLYSRV0273: Excluded subsystem webservices via jboss-deployment-structure.xml does not exist.
14:22:21,748 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 66) WFLYCLINF0002: Started client-mappings cache from ejb container
14:22:22,194 INFO [org.keycloak.services] (ServerService Thread Pool -- 66) KC-SERVICES0001: Loading config from standalone.xml or domain.xml
14:22:22,275 INFO [org.keycloak.common.Profile] (ServerService Thread Pool -- 66) Preview feature enabled: admin_fine_grained_authz
14:22:22,275 INFO [org.keycloak.common.Profile] (ServerService Thread Pool -- 66) Preview feature enabled: token_exchange
14:22:22,547 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 66) WFLYCLINF0002: Started realmRevisions cache from keycloak container
14:22:22,550 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 66) WFLYCLINF0002: Started userRevisions cache from keycloak container
14:22:22,555 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 66) WFLYCLINF0002: Started authorizationRevisions cache from keycloak container
14:22:22,556 INFO [org.keycloak.connections.infinispan.DefaultInfinispanConnectionProviderFactory] (ServerService Thread Pool -- 66) Node name: keycloak-7689594b78-c46ln, Site name: null
14:22:23,471 WARN [org.jboss.jca.core.connectionmanager.pool.strategy.OnePool] (ServerService Thread Pool -- 66) IJ000604: Throwable while attempting to get a new connection: null: javax.resource.ResourceException: IJ031084: Unable to create connection
at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createLocalManagedConnection(LocalManagedConnectionFactory.java:345)
at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:352)
at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:287)
at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreConcurrentLinkedDequeManagedConnectionPool.createConnectionEventListener(SemaphoreConcurrentLinkedDequeManagedConnectionPool.java:1325)
at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreConcurrentLinkedDequeManagedConnectionPool.getConnection(SemaphoreConcurrentLinkedDequeManagedConnectionPool.java:499)
at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:632)
at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:604)
at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:624)
at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:440)
at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:789)
at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:138)
at org.jboss.as.connector.subsystems.datasources.WildFlyDataSource.getConnection(WildFlyDataSource.java:64)
at org.keycloak.connections.jpa.DefaultJpaConnectionProviderFactory.getConnection(DefaultJpaConnectionProviderFactory.java:376)
at org.keycloak.connections.jpa.updater.liquibase.lock.LiquibaseDBLockProvider.lazyInit(LiquibaseDBLockProvider.java:65)
at org.keycloak.connections.jpa.updater.liquibase.lock.LiquibaseDBLockProvider.lambda$waitForLock$0(LiquibaseDBLockProvider.java:97)
at org.keycloak.models.utils.KeycloakModelUtils.suspendJtaTransaction(KeycloakModelUtils.java:678)
at org.keycloak.connections.jpa.updater.liquibase.lock.LiquibaseDBLockProvider.waitForLock(LiquibaseDBLockProvider.java:95)
at org.keycloak.services.resources.KeycloakApplication$1.run(KeycloakApplication.java:144)
at org.keycloak.models.utils.KeycloakModelUtils.runJobInTransaction(KeycloakModelUtils.java:227)
at org.keycloak.services.resources.KeycloakApplication.<init>(KeycloakApplication.java:137)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.jboss.resteasy.core.ConstructorInjectorImpl.construct(ConstructorInjectorImpl.java:152)
at org.jboss.resteasy.spi.ResteasyProviderFactory.createProviderInstance(ResteasyProviderFactory.java:2750)
at org.jboss.resteasy.spi.ResteasyDeployment.createApplication(ResteasyDeployment.java:364)
at org.jboss.resteasy.spi.ResteasyDeployment.startInternal(ResteasyDeployment.java:277)
at org.jboss.resteasy.spi.ResteasyDeployment.start(ResteasyDeployment.java:89)
at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.init(ServletContainerDispatcher.java:119)
at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.init(HttpServletDispatcher.java:36)
at io.undertow.servlet.core.LifecyleInterceptorInvocation.proceed(LifecyleInterceptorInvocation.java:117)
at org.wildfly.extension.undertow.security.RunAsLifecycleInterceptor.init(RunAsLifecycleInterceptor.java:78)
at io.undertow.servlet.core.LifecyleInterceptorInvocation.proceed(LifecyleInterceptorInvocation.java:103)
at io.undertow.servlet.core.ManagedServlet$DefaultInstanceStrategy.start(ManagedServlet.java:303)
at io.undertow.servlet.core.ManagedServlet.createServlet(ManagedServlet.java:143)
at io.undertow.servlet.core.DeploymentManagerImpl$2.call(DeploymentManagerImpl.java:583)
at io.undertow.servlet.core.DeploymentManagerImpl$2.call(DeploymentManagerImpl.java:554)
at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:42)
at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43)
at org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105)
at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502)
at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502)
at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502)
at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502)
at io.undertow.servlet.core.DeploymentManagerImpl.start(DeploymentManagerImpl.java:596)
at org.wildfly.extension.undertow.deployment.UndertowDeploymentService.startContext(UndertowDeploymentService.java:97)
at org.wildfly.extension.undertow.deployment.UndertowDeploymentService$1.run(UndertowDeploymentService.java:78)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982)
at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486)
at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377)
at java.lang.Thread.run(Thread.java:748)
at org.jboss.threads.JBossThread.run(JBossThread.java:485)
Caused by: org.postgresql.util.PSQLException: The connection attempt failed.
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:292)
at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:195)
at org.postgresql.Driver.makeConnection(Driver.java:454)
at org.postgresql.Driver.connect(Driver.java:256)
at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createLocalManagedConnection(LocalManagedConnectionFactory.java:321)
... 55 more
Caused by: java.net.UnknownHostException: postgres
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:184)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at org.postgresql.core.PGStream.<init>(PGStream.java:70)
at org.postgresql.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:91)
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:192)
... 60 more
14:22:23,479 INFO [org.jboss.as.server] (Thread-2) WFLYSRV0220: Server shutdown has been requested via an OS signal
14:22:23,481 ERROR [org.jboss.msc.service.fail] (ServerService Thread Pool -- 66) MSC000001: Failed to start service jboss.deployment.unit."keycloak-server.war".undertow-deployment: org.jboss.msc.service.StartException in service jboss.deployment.unit."keycloak-server.war".undertow-deployment: java.lang.RuntimeException: RESTEASY003325: Failed to construct public org.keycloak.services.resources.KeycloakApplication(javax.servlet.ServletContext,org.jboss.resteasy.core.Dispatcher)
at org.wildfly.extension.undertow.deployment.UndertowDeploymentService$1.run(UndertowDeploymentService.java:81)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982)
at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486)
at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377)
at java.lang.Thread.run(Thread.java:748)
at org.jboss.threads.JBossThread.run(JBossThread.java:485)
Caused by: java.lang.RuntimeException: RESTEASY003325: Failed to construct public org.keycloak.services.resources.KeycloakApplication(javax.servlet.ServletContext,org.jboss.resteasy.core.Dispatcher)
at org.jboss.resteasy.core.ConstructorInjectorImpl.construct(ConstructorInjectorImpl.java:164)
at org.jboss.resteasy.spi.ResteasyProviderFactory.createProviderInstance(ResteasyProviderFactory.java:2750)
at org.jboss.resteasy.spi.ResteasyDeployment.createApplication(ResteasyDeployment.java:364)
at org.jboss.resteasy.spi.ResteasyDeployment.startInternal(ResteasyDeployment.java:277)
at org.jboss.resteasy.spi.ResteasyDeployment.start(ResteasyDeployment.java:89)
at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.init(ServletContainerDispatcher.java:119)
at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.init(HttpServletDispatcher.java:36)
at io.undertow.servlet.core.LifecyleInterceptorInvocation.proceed(LifecyleInterceptorInvocation.java:117)
at org.wildfly.extension.undertow.security.RunAsLifecycleInterceptor.init(RunAsLifecycleInterceptor.java:78)
at io.undertow.servlet.core.LifecyleInterceptorInvocation.proceed(LifecyleInterceptorInvocation.java:103)
at io.undertow.servlet.core.ManagedServlet$DefaultInstanceStrategy.start(ManagedServlet.java:303)
at io.undertow.servlet.core.ManagedServlet.createServlet(ManagedServlet.java:143)
at io.undertow.servlet.core.DeploymentManagerImpl$2.call(DeploymentManagerImpl.java:583)
at io.undertow.servlet.core.DeploymentManagerImpl$2.call(DeploymentManagerImpl.java:554)
at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:42)
at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43)
at org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105)
at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502)
at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502)
at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502)
at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502)
at io.undertow.servlet.core.DeploymentManagerImpl.start(DeploymentManagerImpl.java:596)
at org.wildfly.extension.undertow.deployment.UndertowDeploymentService.startContext(UndertowDeploymentService.java:97)
at org.wildfly.extension.undertow.deployment.UndertowDeploymentService$1.run(UndertowDeploymentService.java:78)
... 8 more
Caused by: java.lang.RuntimeException: Failed to connect to database
at org.keycloak.connections.jpa.DefaultJpaConnectionProviderFactory.getConnection(DefaultJpaConnectionProviderFactory.java:382)
at org.keycloak.connections.jpa.updater.liquibase.lock.LiquibaseDBLockProvider.lazyInit(LiquibaseDBLockProvider.java:65)
at org.keycloak.connections.jpa.updater.liquibase.lock.LiquibaseDBLockProvider.lambda$waitForLock$0(LiquibaseDBLockProvider.java:97)
at org.keycloak.models.utils.KeycloakModelUtils.suspendJtaTransaction(KeycloakModelUtils.java:678)
at org.keycloak.connections.jpa.updater.liquibase.lock.LiquibaseDBLockProvider.waitForLock(LiquibaseDBLockProvider.java:95)
at org.keycloak.services.resources.KeycloakApplication$1.run(KeycloakApplication.java:144)
at org.keycloak.models.utils.KeycloakModelUtils.runJobInTransaction(KeycloakModelUtils.java:227)
at org.keycloak.services.resources.KeycloakApplication.<init>(KeycloakApplication.java:137)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.jboss.resteasy.core.ConstructorInjectorImpl.construct(ConstructorInjectorImpl.java:152)
... 31 more
Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:jboss/datasources/KeycloakDS
at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:146)
at org.jboss.as.connector.subsystems.datasources.WildFlyDataSource.getConnection(WildFlyDataSource.java:64)
at org.keycloak.connections.jpa.DefaultJpaConnectionProviderFactory.getConnection(DefaultJpaConnectionProviderFactory.java:376)
... 43 more
Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:jboss/datasources/KeycloakDS
at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:690)
at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:440)
at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:789)
at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:138)
... 45 more
Caused by: javax.resource.ResourceException: IJ031084: Unable to create connection
at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createLocalManagedConnection(LocalManagedConnectionFactory.java:345)
at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:352)
at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:287)
at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreConcurrentLinkedDequeManagedConnectionPool.createConnectionEventListener(SemaphoreConcurrentLinkedDequeManagedConnectionPool.java:1325)
at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreConcurrentLinkedDequeManagedConnectionPool.getConnection(SemaphoreConcurrentLinkedDequeManagedConnectionPool.java:499)
at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:632)
at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:604)
at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:624)
... 48 more
Caused by: org.postgresql.util.PSQLException: The connection attempt failed.
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:292)
at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:195)
at org.postgresql.Driver.makeConnection(Driver.java:454)
at org.postgresql.Driver.connect(Driver.java:256)
at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createLocalManagedConnection(LocalManagedConnectionFactory.java:321)
... 55 more
Caused by: java.net.UnknownHostException: postgres
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:184)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at org.postgresql.core.PGStream.<init>(PGStream.java:70)
at org.postgresql.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:91)
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:192)
... 60 more
14:22:23,504 INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-2) WFLYJCA0010: Unbound data source [java:jboss/datasources/KeycloakDS]
14:22:23,504 INFO [org.jboss.as.mail.extension] (MSC service thread 1-3) WFLYMAIL0002: Unbound mail session [java:jboss/mail/Default]
14:22:23,506 INFO [org.wildfly.extension.undertow] (MSC service thread 1-2) WFLYUT0008: Undertow HTTPS listener https suspending
14:22:23,510 INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-4) WFLYJCA0010: Unbound data source [java:jboss/datasources/ExampleDS]
14:22:23,511 INFO [org.wildfly.extension.undertow] (MSC service thread 1-2) WFLYUT0007: Undertow HTTPS listener https stopped, was bound to 0.0.0.0:8443
14:22:23,511 INFO [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-2) WFLYJCA0019: Stopped Driver service with driver-name = postgresql
14:22:23,534 INFO [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-1) WFLYJCA0019: Stopped Driver service with driver-name = h2
14:22:23,535 INFO [org.wildfly.extension.undertow] (MSC service thread 1-1) WFLYUT0019: Host default-host stopping
14:22:23,537 INFO [org.keycloak.subsystem.server.extension.KeycloakProviderDeploymentProcessor] (MSC service thread 1-4) Undeploying Keycloak provider: openshift4-extension-6.0.1.jar
14:22:23,539 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 66) WFLYCLINF0003: Stopped authenticationSessions cache from keycloak container
14:22:23,539 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 54) WFLYCLINF0003: Stopped loginFailures cache from keycloak container
14:22:23,539 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 64) WFLYCLINF0003: Stopped sessions cache from keycloak container
14:22:23,539 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 62) WFLYCLINF0003: Stopped work cache from keycloak container
14:22:23,540 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 61) WFLYCLINF0003: Stopped realms cache from keycloak container
14:22:23,541 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 55) WFLYCLINF0003: Stopped offlineClientSessions cache from keycloak container
14:22:23,541 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 58) WFLYCLINF0003: Stopped authorization cache from keycloak container
14:22:23,541 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 57) WFLYCLINF0003: Stopped clientSessions cache from keycloak container
14:22:23,541 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 59) WFLYCLINF0003: Stopped keys cache from keycloak container
14:22:23,542 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 60) WFLYCLINF0003: Stopped users cache from keycloak container
14:22:23,542 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 63) WFLYCLINF0003: Stopped offlineSessions cache from keycloak container
14:22:23,542 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 65) WFLYCLINF0003: Stopped actionTokens cache from keycloak container
14:22:23,545 INFO [org.jboss.as.server.deployment] (MSC service thread 1-4) WFLYSRV0028: Stopped deployment openshift4-extension-6.0.1.jar (runtime-name: openshift4-extension-6.0.1.jar) in 60ms
14:22:23,548 INFO [org.jboss.as.server.deployment] (MSC service thread 1-2) WFLYSRV0028: Stopped deployment keycloak-server.war (runtime-name: keycloak-server.war) in 64ms
14:22:23,551 INFO [org.wildfly.extension.undertow] (MSC service thread 1-1) WFLYUT0008: Undertow HTTP listener default suspending
14:22:23,552 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 56) WFLYCLINF0003: Stopped client-mappings cache from ejb container
14:22:23,552 INFO [org.wildfly.extension.undertow] (MSC service thread 1-1) WFLYUT0007: Undertow HTTP listener default stopped, was bound to 0.0.0.0:8080
14:22:23,553 INFO [org.wildfly.extension.undertow] (MSC service thread 1-4) WFLYUT0004: Undertow 2.0.19.Final stopping
14:22:23,557 ERROR [org.jboss.as.controller.management-operation] (Controller Boot Thread) WFLYCTL0013: Operation ("add") failed - address: ([("subsystem" => "microprofile-metrics-smallrye")]): java.lang.NullPointerException
at org.wildfly.extension.microprofile.metrics.MicroProfileMetricsSubsystemAdd$2.execute(MicroProfileMetricsSubsystemAdd.java:86)
at org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:999)
at org.jboss.as.controller.AbstractOperationContext.processStages(AbstractOperationContext.java:743)
at org.jboss.as.controller.AbstractOperationContext.executeOperation(AbstractOperationContext.java:467)
at org.jboss.as.controller.OperationContextImpl.executeOperation(OperationContextImpl.java:1412)
at org.jboss.as.controller.ModelControllerImpl.boot(ModelControllerImpl.java:521)
at org.jboss.as.controller.AbstractControllerService.boot(AbstractControllerService.java:472)
at org.jboss.as.controller.AbstractControllerService.boot(AbstractControllerService.java:434)
at org.jboss.as.server.ServerService.boot(ServerService.java:435)
at org.jboss.as.server.ServerService.boot(ServerService.java:394)
at org.jboss.as.controller.AbstractControllerService$1.run(AbstractControllerService.java:374)
at java.lang.Thread.run(Thread.java:748)
I think it is possible to build and use a custom PostgreSQL image with a patch to /data/userdata/pg_hba.conf and use in spec.database.postgresImage
For keycoak it requires custom che-operator image since 0.0.0.0 is hardcoded there as well:
- https://github.com/eclipse/che-operator/blob/master/pkg/deploy/identity-provider/deployment_keycloak.go#L551
- https://github.com/eclipse/che-operator/blob/master/pkg/deploy/identity-provider/deployment_keycloak.go#L569
@gattytto Would you like to contribute into Eclipse Che ?
this does not seem to be implemented in Jboss/keycloak (Djava.net.preferIPv4Stack is the only relevant option to this intent), as postgress does when it lets one to specify two "listen" stanzas in a config and have it continue startup if "any of them" turns succesfull.
I also don't think -b can be used twice when in https://github.com/eclipse/che-operator/blob/master/pkg/deploy/identity-provider/deployment_keycloak.go#L551
maybe this can be used to set the right CIDR to use before starting keycloak:
#!/bin/bash
BIND_CIDR=$(echo "$(ip -6 route list | awk '$1 == "default" {print $5}')" |head -1);
if [ -n "$BIND_CIDR" ]
then BIND_CIDR="::/0";
else BIND_CIDR="0.0.0.0";
fi;
I have a POC that succeeds for keycloak and postgres, devfile and plugin registries, but che pod says:
root@cgroovy:~# kubectl -n eclipse-che logs pod/che-7dddbd6675-j4h5l
2021-04-13 00:41:44,945[675-j4h5l-49576] [WARN ] [o.j.p.kubernetes.KUBE_PING 115] - failed getting JSON response from Kubernetes Client[masterUrl=https://1101:300:1:2::1:443/api/v1, headers={Authorization=#MASKED:901#}, connectTimeout=5000, readTimeout=30000, operationAttempts=3, operationSleep=1000, streamProvider=org.jgroups.protocols.kubernetes.stream.TokenStreamProvider@298a1b5a] for cluster [WorkspaceLocks], namespace [eclipse-che], labels [app.kubernetes.io/component=che,app.kubernetes.io/instance=che,app.kubernetes.io/managed-by=che-operator,app.kubernetes.io/name=che]; encountered [java.lang.Exception: 3 attempt(s) with a 1000ms sleep to execute [OpenStream] failed. Last failure was [java.net.MalformedURLException: Error at index 3 in: "300:1:2::1:443"]]
meaning there's some lines in che-server code that requires brackets for v6 urls like https://[300:1:2::1]:443
I can see che container ENV has a couple occurences of the kube api v6 without the [] but I don't know which one is used in the case of this error.
CHE_HOST_PORT_8080_TCP=tcp://[1101:300:1:2::9f47]:8080
POSTGRES_PORT_5432_TCP=tcp://[1101:300:1:2::8dc7]:5432
LANGUAGE=en_US:en
DEVFILE_REGISTRY_PORT_8080_TCP_ADDR=1101:300:1:2::b5d8
CHE_WORKSPACE_MAVEN__OPTIONS=-XX:MaxRAM=150m -XX:MaxRAMFraction=2 -XX:+UseParallelGC -XX:MinHeapFreeRatio=10 -XX:MaxHeapFreeRatio=20 -XX:GCTimeRatio=4 -XX:AdaptiveSizePolicyWeight=90 -Dsun.zip.disableMemoryMapping=true -Xms20m -Djava.security.egd=file:/dev/./urandom
KEYCLOAK_PORT=tcp://[1101:300:1:2::902f]:8080
KEYCLOAK_PORT_8080_TCP=tcp://[1101:300:1:2::902f]:8080
CHE_WEBSOCKET_ENDPOINT=ws://che-host.eclipse-che.svc:8080/api/websocket
POSTGRES_PORT_5432_TCP_ADDR=1101:300:1:2::8dc7
CHE_API_INTERNAL=http://che-host.eclipse-che.svc:8080/api
CHE_KEYCLOAK_AUTH__INTERNAL__SERVER__URL=http://keycloak.eclipse-che.svc:8080/auth
JAVA_OPTS=-XX:MaxRAMPercentage=85.0
CHE_WORKSPACE_JAVA__OPTIONS=-XX:MaxRAM=150m -XX:MaxRAMFraction=2 -XX:+UseParallelGC -XX:MinHeapFreeRatio=10 -XX:MaxHeapFreeRatio=20 -XX:GCTimeRatio=4 -XX:AdaptiveSizePolicyWeight=90 -Dsun.zip.disableMemoryMapping=true -Xms20m -Djava.security.egd=file:/dev/./urandom
CHE_HOST_PORT_8087_TCP=tcp://[1101:300:1:2::9f47]:8087
DEVFILE_REGISTRY_PORT=tcp://[1101:300:1:2::b5d8]:8080
CHE_SERVER_SECURE__EXPOSER_JWTPROXY_IMAGE=quay.io/eclipse/che-jwtproxy:0.10.0
CHE_WORKSPACE_PLUGIN__REGISTRY__URL=https://plugin-registry-eclipse-che.che.lasmalvinassonargentinas.com/v3
CHE_HOST_PORT_8080_TCP_ADDR=1101:300:1:2::9f47
KUBERNETES_PORT_443_TCP=tcp://[1101:300:1:2::1]:443
KEYCLOAK_SERVICE_HOST=1101:300:1:2::902f
DEVFILE_REGISTRY_SERVICE_HOST=1101:300:1:2::b5d8
CHE_WORKSPACE_DEVFILE__REGISTRY__URL=https://devfile-registry-eclipse-che.che.lasmalvinassonargentinas.com
CHE_INFRA_KUBERNETES_INGRESS_ANNOTATIONS__JSON={"kubernetes.io/ingress.class": nginx, "nginx.ingress.kubernetes.io/rewrite-target": "/$1","nginx.ingress.kubernetes.io/ssl-redirect": true,"nginx.ingress.kubernetes.io/proxy-connect-timeout": "3600","nginx.ingress.kubernetes.io/proxy-read-timeout": "3600"}
CHE_HOST_PORT_8087_TCP_ADDR=1101:300:1:2::9f47
PLUGIN_REGISTRY_SERVICE_HOST=1101:300:1:2::a141
DEVFILE_REGISTRY_PORT_8080_TCP=tcp://[1101:300:1:2::b5d8]:8080
CHE_WORKSPACE_PLUGIN__BROKER_ARTIFACTS_IMAGE=quay.io/eclipse/che-plugin-artifacts-broker:v3.4.0
PLUGIN_REGISTRY_PORT_8080_TCP=tcp://[1101:300:1:2::a141]:8080
CHE_JDBC_URL=jdbc:postgresql://postgres:5432/dbche
CHE_API=https://che-eclipse-che.che.lasmalvinassonargentinas.com/api
CHE_HOST_SERVICE_HOST=1101:300:1:2::9f47
KEYCLOAK_PORT_8080_TCP_ADDR=1101:300:1:2::902f
PLUGIN_REGISTRY_PORT_8080_TCP_ADDR=1101:300:1:2::a141
POSTGRES_PORT=tcp://[1101:300:1:2::8dc7]:5432
CHE_WEBSOCKET_ENDPOINT__MINOR=ws://che-host.eclipse-che.svc:8080/api/websocket-minor
CHE_KEYCLOAK_AUTH__SERVER__URL=https://keycloak-eclipse-che.che.lasmalvinassonargentinas.com/auth
KUBERNETES_PORT_443_TCP_ADDR=1101:300:1:2::1
CHE_WORKSPACE_PLUGIN__BROKER_METADATA_IMAGE=quay.io/eclipse/che-plugin-metadata-broker:v3.4.0
KUBERNETES_SERVICE_HOST=1101:300:1:2::1
KUBERNETES_PORT=tcp://[1101:300:1:2::1]:443
PATH=/opt/java/openjdk/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
PLUGIN_REGISTRY_PORT=tcp://[1101:300:1:2::a141]:8080
CHE_WORKSPACE_DEVFILE__REGISTRY__INTERNAL__URL=http://devfile-registry.eclipse-che.svc:8080
CHE_WORKSPACE_PLUGIN__REGISTRY__INTERNAL__URL=http://plugin-registry.eclipse-che.svc:8080/v3
CHE_INFRA_KUBERNETES_PVC_JOBS_IMAGE=registry.access.redhat.com/ubi8-minimal:8.3-298
CHE_HOST_PORT=tcp://[1101:300:1:2::9f47]:8080
POSTGRES_SERVICE_HOST=1101:300:1:2::8dc7
in che-operator there's this 2 lines (1 & 2) that use the env var that has:
KUBERNETES_PORT_443_TCP_ADDR=1101:300:1:2::1
this will result in:
https://1101:300:1:2::1/.well-known/oauth-authorization-server
and
https://1101:300:1:2::1/apis/config.openshift.io/v1/infrastructures/cluster
respectively
I think this is the KUBE_PING call that's assembling something like:
"%s://%s:%s/api/%s", masterProtocol, masterHost, masterPort, apiVersion
masterHost being:
@Property(description="The URL of the Kubernetes server", systemProperty="KUBERNETES_SERVICE_HOST")
protected String masterHost;
in my case from the che pod env vars:
KUBERNETES_SERVICE_HOST=1101:300:1:2::1
@tolusha upgrading keycloack is a nice idea, although v6 can be sorted out for the old version too from what I tested, but I don't know what can be done about this KUBE_PING since it comes from external jboss library in the che-server java project.
any idea?
adding brackets to the kubernetes service host makes catalina and other apis fail in the kubernetes httpclient call with Invalid host: [[1101:300:1:2::1].
I think this is the field that gets filled with $KUBERNETES_SERVICE_HOST without protocol distinction or checking.
eclipse che pod logs
root@cgroovy:~# kubectl -n eclipse-che logs pod/che-748cfdb669-wm7zw
using v6
Using embedded assembly in /home/user/eclipse-che/.
Found a custom cert. Adding it to java trust store /home/user/cacerts based on /opt/java/openjdk/lib/security/cacerts
Trust this certificate? [no]: Certificate was added to keystore
NOTE: Picked up JDK_JAVA_OPTIONS: --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED
16-Apr-2021 16:12:58.492 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server version name: Apache Tomcat/8.5.64
16-Apr-2021 16:12:58.498 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server built: Mar 4 2021 23:14:16 UTC
16-Apr-2021 16:12:58.498 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server version number: 8.5.64.0
16-Apr-2021 16:12:58.499 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log OS Name: Linux
16-Apr-2021 16:12:58.499 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log OS Version: 5.4.101-1-pve
16-Apr-2021 16:12:58.499 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Architecture: amd64
16-Apr-2021 16:12:58.499 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Java Home: /opt/java/openjdk
16-Apr-2021 16:12:58.499 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log JVM Version: 11.0.8+10
16-Apr-2021 16:12:58.499 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log JVM Vendor: AdoptOpenJDK
16-Apr-2021 16:12:58.499 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log CATALINA_BASE: /home/user/eclipse-che/tomcat
16-Apr-2021 16:12:58.500 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log CATALINA_HOME: /home/user/eclipse-che/tomcat
16-Apr-2021 16:12:58.500 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: --add-opens=java.base/java.lang=ALL-UNNAMED
16-Apr-2021 16:12:58.500 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: --add-opens=java.base/java.io=ALL-UNNAMED
16-Apr-2021 16:12:58.500 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: --add-opens=java.base/java.util=ALL-UNNAMED
16-Apr-2021 16:12:58.500 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: --add-opens=java.base/java.util.concurrent=ALL-UNNAMED
16-Apr-2021 16:12:58.500 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: --add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED
16-Apr-2021 16:12:58.501 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.util.logging.config.file=/home/user/eclipse-che//tomcat/conf/logging.properties
16-Apr-2021 16:12:58.501 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
16-Apr-2021 16:12:58.501 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -XX:MaxRAMPercentage=85.0
16-Apr-2021 16:12:58.501 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dche.docker.network=bridge
16-Apr-2021 16:12:58.501 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djavax.net.ssl.trustStore=/home/user/cacerts
16-Apr-2021 16:12:58.501 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djavax.net.ssl.trustStorePassword=changeit
16-Apr-2021 16:12:58.501 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dport.http=8080
16-Apr-2021 16:12:58.501 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dche.home=/home/user/eclipse-che/
16-Apr-2021 16:12:58.501 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dche.logs.dir=/logs/
16-Apr-2021 16:12:58.501 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dche.logs.level=INFO
16-Apr-2021 16:12:58.501 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djuli-logback.configurationFile=file:/home/user/eclipse-che//tomcat/conf/tomcat-logger.xml
16-Apr-2021 16:12:58.501 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djdk.tls.ephemeralDHKeySize=2048
16-Apr-2021 16:12:58.501 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.protocol.handler.pkgs=org.apache.catalina.webresources
16-Apr-2021 16:12:58.565 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dorg.apache.catalina.security.SecurityListener.UMASK=0022
16-Apr-2021 16:12:58.565 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dche.local.conf.dir=/home/user/eclipse-che//tomcat/conf/
16-Apr-2021 16:12:58.565 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dignore.endorsed.dirs=
16-Apr-2021 16:12:58.565 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcatalina.base=/home/user/eclipse-che//tomcat
16-Apr-2021 16:12:58.566 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcatalina.home=/home/user/eclipse-che//tomcat
16-Apr-2021 16:12:58.566 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.io.tmpdir=/home/user/eclipse-che//tomcat/temp
16-Apr-2021 16:12:58.587 INFO [main] org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler ["http-nio-8080"]
16-Apr-2021 16:12:58.597 INFO [main] org.apache.tomcat.util.net.NioSelectorPool.getSharedSelector Using a shared selector for servlet write/read
16-Apr-2021 16:12:58.669 INFO [main] org.apache.catalina.startup.Catalina.load Initialization processed in 989 ms
16-Apr-2021 16:12:58.694 INFO [main] org.apache.catalina.core.StandardService.startInternal Starting service [Catalina]
16-Apr-2021 16:12:58.694 INFO [main] org.apache.catalina.core.StandardEngine.startInternal Starting Servlet Engine: Apache Tomcat/8.5.64
16-Apr-2021 16:12:58.777 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployWAR Deploying web application archive [/home/user/eclipse-che/tomcat/webapps/ROOT.war]
16-Apr-2021 16:13:01.479 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployWAR Deployment of web application archive [/home/user/eclipse-che/tomcat/webapps/ROOT.war] has finished in [2,702] ms
16-Apr-2021 16:13:01.480 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployWAR Deploying web application archive [/home/user/eclipse-che/tomcat/webapps/api.war]
2021-04-16 16:13:05,803[ost-startStop-1] [INFO ] [.e.c.c.d.JNDIDataSourceFactory 63] - This=org.eclipse.che.core.db.postgresql.PostgreSQLJndiDataSourceFactory@363b8c3c obj=ResourceRef[className=javax.sql.DataSource,factoryClassLocation=null,factoryClassName=org.apache.naming.factory.ResourceFactory,{type=scope,content=Shareable},{type=auth,content=Container},{type=singleton,content=true},{type=factory,content=org.eclipse.che.api.CommonJndiDataSourceFactory}] name=che Context=org.apache.naming.NamingContext@687afbc1 environment={}
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.google.inject.assistedinject.FactoryProvider2$MethodHandleWrapper (file:/home/user/eclipse-che/tomcat/webapps/api/WEB-INF/lib/guice-assistedinject-4.2.2.jar) to constructor java.lang.invoke.MethodHandles$Lookup(java.lang.Class,int)
WARNING: Please consider reporting this to the maintainers of com.google.inject.assistedinject.FactoryProvider2$MethodHandleWrapper
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
2021-04-16 16:13:08,683[ost-startStop-1] [INFO ] [o.e.c.m.k.s.OIDCInfoProvider 72] - Retrieving OpenId configuration from endpoint: http://keycloak.eclipse-che.svc:8080/auth/realms/che/.well-known/openid-configuration
2021-04-16 16:13:08,908[ost-startStop-1] [INFO ] [o.e.c.m.k.s.OIDCInfoProvider 81] - openid configuration = {issuer=https://keycloak-eclipse-che.che.lasmalvinassonargentinas.com/auth/realms/che, authorization_endpoint=https://keycloak-eclipse-che.che.lasmalvinassonargentinas.com/auth/realms/che/protocol/openid-connect/auth, token_endpoint=https://keycloak-eclipse-che.che.lasmalvinassonargentinas.com/auth/realms/che/protocol/openid-connect/token, token_introspection_endpoint=https://keycloak-eclipse-che.che.lasmalvinassonargentinas.com/auth/realms/che/protocol/openid-connect/token/introspect, userinfo_endpoint=https://keycloak-eclipse-che.che.lasmalvinassonargentinas.com/auth/realms/che/protocol/openid-connect/userinfo, end_session_endpoint=https://keycloak-eclipse-che.che.lasmalvinassonargentinas.com/auth/realms/che/protocol/openid-connect/logout, jwks_uri=https://keycloak-eclipse-che.che.lasmalvinassonargentinas.com/auth/realms/che/protocol/openid-connect/certs, check_session_iframe=https://keycloak-eclipse-che.che.lasmalvinassonargentinas.com/auth/realms/che/protocol/openid-connect/login-status-iframe.html, grant_types_supported=[authorization_code, implicit, refresh_token, password, client_credentials], response_types_supported=[code, none, id_token, token, id_token token, code id_token, code token, code id_token token], subject_types_supported=[public, pairwise], id_token_signing_alg_values_supported=[PS384, ES384, RS384, HS256, HS512, ES256, RS256, HS384, ES512, PS256, PS512, RS512], userinfo_signing_alg_values_supported=[PS384, ES384, RS384, HS256, HS512, ES256, RS256, HS384, ES512, PS256, PS512, RS512, none], request_object_signing_alg_values_supported=[PS384, ES384, RS384, ES256, RS256, ES512, PS256, PS512, RS512, none], response_modes_supported=[query, fragment, form_post], registration_endpoint=https://keycloak-eclipse-che.che.lasmalvinassonargentinas.com/auth/realms/che/clients-registrations/openid-connect, token_endpoint_auth_methods_supported=[private_key_jwt, client_secret_basic, client_secret_post, client_secret_jwt], token_endpoint_auth_signing_alg_values_supported=[RS256], claims_supported=[aud, sub, iss, auth_time, name, given_name, family_name, preferred_username, email], claim_types_supported=[normal], claims_parameter_supported=false, scopes_supported=[openid, microprofile-jwt, web-origins, roles, phone, address, email, profile, offline_access], request_parameter_supported=true, request_uri_parameter_supported=true, code_challenge_methods_supported=[plain, S256], tls_client_certificate_bound_access_tokens=true, introspection_endpoint=https://keycloak-eclipse-che.che.lasmalvinassonargentinas.com/auth/realms/che/protocol/openid-connect/token/introspect}
2021-04-16 16:13:10,274[ost-startStop-1] [INFO ] [o.j.p.kubernetes.KUBE_PING 131] - namespace eclipse-che set; clustering enabled
-------------------------------------------------------------------
GMS: address=che-748cfdb669-wm7zw-34061, cluster=RemoteSubscriptionChannel, physical address=fe80:0:0:0:4cea:76ff:fedd:f11f%eth0:7800
-------------------------------------------------------------------
2021-04-16 16:13:14,817[ost-startStop-1] [INFO ] [o.jgroups.protocols.pbcast.GMS 125] - che-748cfdb669-wm7zw-34061: no members discovered after 4523 ms: creating cluster as coordinator
2021-04-16 16:13:14,838[ost-startStop-1] [INFO ] [o.j.p.kubernetes.KUBE_PING 131] - namespace eclipse-che set; clustering enabled
-------------------------------------------------------------------
GMS: address=che-748cfdb669-wm7zw-57449, cluster=WorkspaceLocks, physical address=fe80:0:0:0:4cea:76ff:fedd:f11f%eth0:7801
-------------------------------------------------------------------
2021-04-16 16:13:17,955[ost-startStop-1] [INFO ] [o.jgroups.protocols.pbcast.GMS 125] - che-748cfdb669-wm7zw-57449: no members discovered after 3113 ms: creating cluster as coordinator
2021-04-16 16:13:17,962[ost-startStop-1] [INFO ] [o.j.p.kubernetes.KUBE_PING 131] - namespace eclipse-che set; clustering enabled
-------------------------------------------------------------------
GMS: address=che-748cfdb669-wm7zw-60194, cluster=WorkspaceStateCache, physical address=fe80:0:0:0:4cea:76ff:fedd:f11f%eth0:7802
-------------------------------------------------------------------
2021-04-16 16:13:22,383[ost-startStop-1] [INFO ] [o.jgroups.protocols.pbcast.GMS 125] - che-748cfdb669-wm7zw-60194: no members discovered after 4415 ms: creating cluster as coordinator
2021-04-16 16:13:22,512[ost-startStop-1] [INFO ] [o.f.c.i.d.DbSupportFactory 44] - Database: jdbc:postgresql://postgres:5432/dbche (PostgreSQL 9.6)
2021-04-16 16:13:22,533[ost-startStop-1] [INFO ] [o.f.c.i.util.VersionPrinter 44] - Flyway 4.2.0 by Boxfuse
2021-04-16 16:13:22,566[ost-startStop-1] [INFO ] [o.f.c.i.d.DbSupportFactory 44] - Database: jdbc:postgresql://postgres:5432/dbche (PostgreSQL 9.6)
2021-04-16 16:13:22,590[ost-startStop-1] [INFO ] [i.f.CustomSqlMigrationResolver 158] - Searching for SQL scripts in locations [classpath:che-schema]
2021-04-16 16:13:22,683[ost-startStop-1] [INFO ] [o.f.c.i.command.DbValidate 44] - Successfully validated 63 migrations (execution time 00:00.094s)
2021-04-16 16:13:22,693[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Current version of schema "public": 7.26.0.1
2021-04-16 16:13:22,694[ost-startStop-1] [INFO ] [o.f.c.i.command.DbMigrate 44] - Schema "public" is up to date. No migration necessary.
2021-04-16 16:13:23,967[ost-startStop-1] [INFO ] [o.j.p.kubernetes.KUBE_PING 131] - namespace eclipse-che set; clustering enabled
-------------------------------------------------------------------
GMS: address=che-748cfdb669-wm7zw-33943, cluster=EclipseLinkCommandChannel, physical address=fe80:0:0:0:4cea:76ff:fedd:f11f%eth0:7803
-------------------------------------------------------------------
2021-04-16 16:13:27,010[ost-startStop-1] [INFO ] [o.jgroups.protocols.pbcast.GMS 125] - che-748cfdb669-wm7zw-33943: no members discovered after 3018 ms: creating cluster as coordinator
2021-04-16 16:13:27,071[ost-startStop-1] [INFO ] [o.e.c.a.c.u.ApiInfoLogInformer 36] - Eclipse Che Api Core: Build info '7.29.0-SNAPSHOT' scmRevision 'afa0232cda6046c6c589d5100e9806dc212cb085' implementationVersion '7.29.0-SNAPSHOT'
2021-04-16 16:13:27,103[ost-startStop-1] [WARN ] [p.s.AdminPermissionInitializer 69] - Admin admin not found yet.
2021-04-16 16:13:27,184[ost-startStop-1] [INFO ] [o.e.c.c.metrics.MetricsServer 46] - Metrics server started at port 8087 successfully
2021-04-16 16:13:27,395[ost-startStop-1] [ERROR] [o.a.c.c.C.[.[localhost].[/api] 175] - Exception sending context initialized event to listener instance of class [org.eclipse.che.inject.CheBootstrap]
com.google.inject.CreationException: Unable to create injector, see the following errors:
1) Error injecting constructor, io.fabric8.kubernetes.client.KubernetesClientException: Invalid proxy server configuration
at org.eclipse.che.workspace.infrastructure.kubernetes.CheServerKubernetesClientFactory.<init>(CheServerKubernetesClientFactory.java:43)
at org.eclipse.che.workspace.infrastructure.kubernetes.CheServerKubernetesClientFactory.class(CheServerKubernetesClientFactory.java:43)
while locating org.eclipse.che.workspace.infrastructure.kubernetes.CheServerKubernetesClientFactory
for the 2nd parameter of org.eclipse.che.workspace.infrastructure.kubernetes.namespace.CheNamespace.<init>(CheNamespace.java:54)
at org.eclipse.che.workspace.infrastructure.kubernetes.namespace.CheNamespace.class(CheNamespace.java:54)
while locating org.eclipse.che.workspace.infrastructure.kubernetes.namespace.CheNamespace
Caused by: io.fabric8.kubernetes.client.KubernetesClientException: Invalid proxy server configuration
at io.fabric8.kubernetes.client.utils.HttpClientUtils.createHttpClient(HttpClientUtils.java:161)
at io.fabric8.kubernetes.client.utils.HttpClientUtils.createHttpClient(HttpClientUtils.java:65)
at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesClientFactory.<init>(KubernetesClientFactory.java:73)
at org.eclipse.che.workspace.infrastructure.kubernetes.CheServerKubernetesClientFactory.<init>(CheServerKubernetesClientFactory.java:43)
at org.eclipse.che.workspace.infrastructure.kubernetes.CheServerKubernetesClientFactory$$FastClassByGuice$$51b86eab.newInstance(<generated>)
at com.google.inject.internal.DefaultConstructionProxyFactory$FastClassProxy.newInstance(DefaultConstructionProxyFactory.java:89)
at com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:114)
at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:91)
at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:306)
at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)
at com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:168)
at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:39)
at com.google.inject.internal.SingleParameterInjector.inject(SingleParameterInjector.java:42)
at com.google.inject.internal.SingleParameterInjector.getAll(SingleParameterInjector.java:65)
at com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:113)
at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:91)
at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:306)
at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)
at com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:168)
at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:39)
at com.google.inject.internal.InternalInjectorCreator.loadEagerSingletons(InternalInjectorCreator.java:211)
at com.google.inject.internal.InternalInjectorCreator.injectDynamically(InternalInjectorCreator.java:182)
at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:109)
at com.google.inject.Guice.createInjector(Guice.java:87)
at org.everrest.guice.servlet.EverrestGuiceContextListener.getInjector(EverrestGuiceContextListener.java:141)
at com.google.inject.servlet.GuiceServletContextListener.contextInitialized(GuiceServletContextListener.java:45)
at org.everrest.guice.servlet.EverrestGuiceContextListener.contextInitialized(EverrestGuiceContextListener.java:86)
at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4705)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5171)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:743)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:719)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:705)
at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:1015)
at org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1895)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.base/java.lang.Thread.run(Unknown Source)
Caused by: java.net.MalformedURLException: Invalid host: [[1101:300:1:2::1]
at java.base/java.net.URL.<init>(Unknown Source)
at java.base/java.net.URL.<init>(Unknown Source)
at java.base/java.net.URL.<init>(Unknown Source)
at io.fabric8.kubernetes.client.utils.HttpClientUtils.getProxyUrl(HttpClientUtils.java:204)
at io.fabric8.kubernetes.client.utils.HttpClientUtils.createHttpClient(HttpClientUtils.java:147)
... 39 more
Caused by: java.lang.IllegalArgumentException: Invalid host: [[1101:300:1:2::1]
at java.base/java.net.URLStreamHandler.parseURL(Unknown Source)
... 44 more
@tolusha can you pls reopen? I have a real pure ipv6 cluster to test this more now
Issues go stale after 180 days of inactivity. lifecycle/stale issues rot after an additional 7 days of inactivity and eventually close.
Mark the issue as fresh with /remove-lifecycle stale in a new comment.
If this issue is safe to close now please do so.
Moderators: Add lifecycle/frozen label to avoid stale mode.
Issues go stale after 180 days of inactivity. lifecycle/stale issues rot after an additional 7 days of inactivity and eventually close.
Mark the issue as fresh with /remove-lifecycle stale in a new comment.
If this issue is safe to close now please do so.
Moderators: Add lifecycle/frozen label to avoid stale mode.