Upgrade our JFROG suite of products

[Jose-Matsuda]

STATUS AS OF 05/10/2022

Ticket submitted to have cns review it

Moved to Backlog so that it is not counted in the sprint

---

This is to help solve https://github.com/StatCan/daaas/issues/978

Should hopefully fix as noted https://www.jfrog.com/confluence/display/JFROG/Indexing+Xray+Resources

Tasks TODO

• [x] Investigate if there are any breaking changes, leave a comment showing discoveries
• [x] Backup settings (done nightly)
• [x] Check if any NetPols or svc will be affected by the apparent change from artifactory-ha to just artifactory For example this line may need to be updated. Yes this will need to be updated
• [x] Create the new jfrog.tf file to be used as a DRAFT PR Remember to update the bit in the tf where artifactory-ha is referenced in the ingress and change to just artifactory, the user in the database block might just be ok staying artifactoryha
• [x] Check the plan and see what happens, if ok let CNS take a review at it
• [x] VERY BIG WARNING, IF THE artifactory-ha TO artifactory goes through we ~~WILL NEED TO alert the other people who use our artifactory instance to change what they point to I believe~~ (they use the public ip not internal with the service-name), will also need to update our urls in kubeflow containers

Helpful Information

Relevant PRs

• [ ] Gitlab pr
• [ ] Pr to update kubeflow-containers
• [ ] Update dev netpol
• [ ] Update prod netpol

Versions

We are using the jfrog-platform helm chart version 0.4.1 artifactory is on version 7.19.13 xray is on version 3.26.1

Some helpful links

https://www.jfrog.com/confluence/display/JFROG/Upgrading+Artifactory https://www.jfrog.com/confluence/display/JFROG/Artifactory+Release+Notes#ArtifactoryReleaseNotes-PreviousVersions https://www.jfrog.com/confluence/display/JFROG/Xray+Release+Notes

[Jose-Matsuda]

Artifactory upgrade Investigation

Breaking Changes in versions

7.19.13 (fwiw I dont see our version on the release page) https://www.jfrog.com/confluence/display/JFROG/Artifactory+Release+Notes

Note that in our current version of Artifactory we use OpenJDK Version of 11.0.10 (embedded in binary package) Also use Tomcat 8.5.63 (again embedded), so I think when we upgrade this will upgrade automatically.

Looking through release notes 7.21.x: Breaking change in first upgrade, but only if we use Reason-Phrase in Tomcat 7.23.x - 7.24.x: nothing breaking of note 7.25.5: breaking change, now has OpenJDK 11.0.11 or higher, Need to check if database supports TLS 1.2 or later, if not need to enable TLS 1.0 and 1.1 in the JDK 7.26.x - 7.42.x: nothing breaking of note that relates to us, one of them is mysql but we use postgresql, a maven change again don't use any maven repos.

TLDR Check the following

• [x] Tomcat (I dont think we use reason-phrase / I cannot see it anywhere), cant see in any yaml in deploying
• [x] Postgresql and support of TLS1.2 (haven't found a specific source but it seems like it will be ok)

---

Actual Upgrade Notes

https://www.jfrog.com/confluence/display/JFROG/Upgrading+Artifactory We use the artifactory-ha chart via the jfrog-platform chart of 0.4.1

*** Before you upgrade Artifactory, the service must be shut down, Artifactory will not be online during upgrade. Note that in the past it seems Zach and Will just bumped the version tags. https://www.jfrog.com/confluence/display/JFROG/Upgrading+Artifactory#UpgradingArtifactory-HelmUpgrade.1

• [x] I guess want to see the apply when they did the update and see what went on (it just applied, nothing of note happened)

---

So barring anything crazy should be able to go to 7.42.x

[Jose-Matsuda]

Xray upgrade investigation

---

Breaking changes in versions

https://www.jfrog.com/confluence/display/JFROG/Xray+Release+Notes We are on 3.26.1 and I can see it in the list at least 3.27.x - 3.55.x: No breaking changes, just features that are available if you have upgraded artifactory as well (which we will).

---

Actual Upgrade notes

Actual Upgrade notes

Again the service has to be stopped before upgrading. Need to check the plan I guess for this as it asks us to delete the statefulset, then run the helm upgrade.

---

So barring anything crazy should be able to go to 3.55.2

[Jose-Matsuda]

Helm Charts

https://charts.jfrog.io/, we are currently using jfrog-platform 0.4.1

Install the JFrog Platform using Helm chart

In the readme's at least, there is no big difference in what is offered between 0.4.1 and 0.9.3 (but backwards compatibility is not guaranteed, the UPGRADE_NOTES (this appears to be for artifactory/artifactory-ha charts to jfrog-platform) are the same as well. There is nothing in the current chart readmes for the GA charts that indicate how to upgrade from a beta to the GA.

---

It also seems unavoidable that we will need to move to the GA charts, I might be missing something but looking at the changelog for the charts as well when they update say I would think that whatever these charts are, these would be the maximum versions (in this case for 0.9.3 the artifactory-ha chart in here is appVersion 7.24.3 and xray at 3.31.1.

The most recent GA chart as of today (10.8.3) has xray at 3.55.2 and artifactory at 7.41.7

[Jose-Matsuda]

Comparisons of settings in the values.yaml, block by block / anything of note

rabbitmq also exists in our current 0.4.1 chart version, and in our jfrog.tf we did not disable it, so we again keep it.

redis, we don't explicitly disable this, but can we? I don't think we use pipelines, of note yeah it does exist in the jfrog-system ns

postgresql we keep enabled: false

Artifactory-ha --> just artifactory Block Notes

artifactory this is where we changed, they don't have a "distinguishing" between artifactory-ha and artifactory at the base level anymore, it is just artifactory followed by another artifactory

database so under artifactory.artifactory.database

primary, javaOpts block, in the GA chart since there is no distinction I think we lose the primary as it is just javaOpts like it did here so this is under artifactory.artifactory.javaopts

startupProbe: in 0.4.1 this existed in two spots, but now it exists in 9 possible spots, the one linked here is the startupProbe directly underneath artifactory but it exists in other services.

node block in our current tf also goes away, or at least might have moved to the systemYaml block, **NOTE THAT CURRENTLY, ** we do not have any jfrog replicasets, so we may want this to stay at 0, may need to override the default setting

Persistence similar to javaOpts is not underneath anything it seems (other than artifactory)

nginx same as above

ingress seems to have been promoted to be above the second artifactory so just under artifactory.ingress and not artifactory.artifactory.ingress

Xray block

Nothing needs to change can probably make use of a correct systemYamlOverride setting for the retentionperiod now.

Other services that we have enabled:false for currently

mission-control, distribution, pipelines. We should not need to change these at all

Additional services that we will need to consider adding enabled:false that are in the most recent chart

insight, pdnServer (this is already enabled:false by default but be explicit)

[Jose-Matsuda]

Testing out values

The following works, I can at least see the javaopts in the result after doing something like helm install test jfrog/jfrog-platform -f custom.yaml --dry-run and the bottom does provide me saying that it worked successfully.

## Validate this against the current chart
global:
  joinKey: test
  masterKey: test
  versions:
    artifactory: 7.41.7
    xray: 3.55.2
  database:
    initDBCreation: false
    host: blah
    sslMode: require
    secrets:
      adminUsername:
        name: asdsa
        key: "admin-user"
      adminPassword:
        name: asdsa
        key: "admin-password"

# We are using an external database
postgresql:
  enabled: false

# this is now just artifactory, no artifactory-ha
artifactory:

  database: #good
    user: artifactoryhaasdsa
    password: asdsa

  artifactory:
    javaOpts:
      other: "-Dartifactory.list.of.repos.hosts.need.to.avoid.url.normalisation=k8scc01covidacr.azurecr.io"

    #unsure about this startupProbe, in 0.4.1 it could exist in two spots, but now it can
    # exist in 9 total
    startupProbe:
      enabled: false

    # in Specific want NODE replica count to be 0.
    # after some testing this should probably stay at 1, with this setting there are 3 resources with a replicacount of 1
    # xray, rabbitmq, and artifactory, the original 0.4.1 had 4 resources with there being a "ha-member" whose count was 0
    replicaCount: 1

    # persistence is good to stay unchanged.
    persistence:
      maxCacheSize: '50000000000'
      type: azure-blob
      azureBlob:
        accountName: asdsa
        accountKey: asdsa
        endpoint: asdsa
        containerName: artifactory
        multiPartLimit: '100000000'
        multipartElementSize: '50000000'

  # good to stay off, removes one of the 9 startupProbes, curr position good
  nginx:
    enabled: false

  # good to keep off, curr position good
  ingress:
    enabled: false

# This xray block is all fine, nothing has to change
xray:
  enabled: true
  database:
    user: xrayblah
    actualUsername: xray
    password: blah
  systemYamlOverride:
    server.repo.defaultRetentionDaysForIndexedRepo: 30

# doesnt seem to exist in new chart
#mission-control:
#  enabled: false

# good to keep off
distribution:
  enabled: false

# new addition
insight:
  enabled: false

# good to keep off
pipelines:
  enabled: false

# off by default but be explicit
pdnServer:
  enabled: false

With the output of

NOTES:
Congratulations. You have just deployed JFrog Platform Chart with following products:
- artifactory
- xray

NOTES

However, I could not view any evidence of the "systemYamlOverride" option in XRAY that we were passing in the resulting yaml. That can be a separate thing to investigate as that's not vital to the upgrade.

Regarding replicas it's funny because while we did use the ha chart, we only ever had of each pod, and is already 1 by default

Regarding the license we probably don't need to do anything here? Since it would be persisted.

[Jose-Matsuda]

Comparing Outputs of the helm install from the above to the original destined with 0.4.1

Regarding the startupProbe On 0.4.1 we have 5 resources that use it, whereas with the up to date using the yaml above we get 6 resources. 5 of them are the same with the sixth looking at port 8036 at an observability image, which seems to be part of the ecosystem. If I comment out the startupProbe for 0.4.1, I get 7 hits, the same occurs for if I comment it out for the recent chart, so it appears to be ok, the startupProbe being 'false' in both cases removes a startupProbe from the artifactory container

Gist containing findings, everything / new additions seem fine, can maybe turn off jfconnect and change to enabled: false but shouldn't be a dealbreaker, actually they do suggest here to have it as enabled as we do have enterprise, no we have enterprise x

[Jose-Matsuda]

Update: is actively being reviewed, though want to use systemyamloverride

SystemYamlOverride: look to this for extra information. I'll try the helm dry-runs and see if it configures fine.

Oh the reason I don't think I saw anything was because the systemyamloverride has to be configured as a secret...

Not sure why this is here twice but ok (will remove it)

[Jose-Matsuda]

Closing as this was completed by Souheil and the team while I was gone