forcedotcom
force:apex:test:run in JWT context seems to rely on the jwtkeyfile always being present
Summary
force:apex:test:run command fails when using JWT connection and the jwtkeyfile file is missing from project folder
Steps To Reproduce:
- Have jwtKeyFile at project root
- Authenticate to Dev Hub via JWT auth and make it defaultDevHub at least within the context of current SFDX project.
- Execute a scratch org creation
-
force:org:create --definitionfile config/project-scratch-def.json --json --setdefaultusername --durationdays 1
-
- Push of source code
-
force:source:push --json - executes correctly
-
- Conduct an Apex Test run
-
force:apex:test:run --testlevel RunLocalTests --outputdir target --resultformat tap --json - executes correctly
-
- remove jwtKeyFile from project directory
- Push of source code
-
force:source:push --json - executes correctly
-
- Conduct an Apex Test run
-
force:apex:test:run --testlevel RunLocalTests --outputdir target --resultformat tap --json - execution fails
-
Expected result
Execution of force:apex:test:run should succeed regardless of the presence of the jwtKeyFile
Actual result
Execution of force:apex:test:run fails if jwtKeyFile is not present in project directory.
Additional information
This becomes a blocker in Jenkins CI processes on SFDX projects. The recommended way to manage the jwtKeyFile on Jenkins is store it in the "Jenkins Admin Credentials interface." During build job execution, Jenkins will checkout all code to the build job's "workspace" folder. It will download the the jetKeyFile and other secret files to the adjacent folder "workspace@tmp" and inject that file during commands that explicitly utilize it (like the force:auth:jwt:grant command).
As it stands now, I am unable to utilize force:apex:test:run as part of my CI process.
SFDX CLI Version(to find the version of the CLI engine run sfdx --version):
- sfdx-cli/7.4.0-99233fd3af darwin-x64 node-v10.15.3
SFDX plugin Version(to find the version of the CLI plugin run sfdx plugins --core)
- @oclif/plugin-commands 1.2.2 (core)
- @oclif/plugin-help 2.1.6 (core)
- @oclif/plugin-not-found 1.2.2 (core)
- @oclif/plugin-plugins 1.7.8 (core)
- @oclif/plugin-update 1.3.9 (core)
- @oclif/plugin-warn-if-update-available 1.7.0 (core)
- @oclif/plugin-which 1.0.3 (core)
- @salesforce/sfdx-trust 3.0.2 (core)
- analytics 1.1.2 (core)
- generator 1.1.0 (core)
- salesforcedx 45.11.0 (core)
- force-language-services 45.10.0 (core)
- salesforce-alm 45.14.0 (core)
- sfdx-cli 7.4.0 (core)
OS and version:
- MacOS v10.14.4
- Ubuntu Linux (amd64) latest version
The error message that I get is the following:
sfdx force:apex:test:run --testlevel RunLocalTests --outputdir target/27 --resultformat tap --json
{
"status": 1,
"name": "InvalidAsyncTestJob",
"message": "Unable to invoke async test job: ENOENT: no such file or directory, open 'server.key'",
"exitCode": 1,
"commandName": "ApexTestRunCommand",
"stack": "InvalidAsyncTestJob: Unable to invoke async test job: ENOENT: no such file or directory, open 'server.key'\n at ALMError (/Users/johndaniel/.local/share/sfdx/client/7.
4.0-99233fd3af/node_modules/salesforce-alm/dist/lib/core/almError.js:44:19)\n at waitTillConnected.then.then.then.then.then.catch.err (/Users/johndaniel/.local/share/sfdx/client/7.4
.0-99233fd3af/node_modules/salesforce-alm/dist/lib/apex/apexTestApi.js:532:29)\n at tryCatcher (/Users/johndaniel/.local/share/sfdx/client/7.4.0-99233fd3af/node_modules/salesforce-a
lm/node_modules/bluebird/js/release/util.js:16:23)\n at Promise._settlePromiseFromHandler (/Users/johndaniel/.local/share/sfdx/client/7.4.0-99233fd3af/node_modules/salesforce-alm/no
de_modules/bluebird/js/release/promise.js:510:31)\n at Promise._settlePromise (/Users/johndaniel/.local/share/sfdx/client/7.4.0-99233fd3af/node_modules/salesforce-alm/node_modules/b
luebird/js/release/promise.js:567:18)\n at Promise._settlePromise0 (/Users/johndaniel/.local/share/sfdx/client/7.4.0-99233fd3af/node_modules/salesforce-alm/node_modules/bluebird/js/
release/promise.js:612:10)\n at Promise._settlePromises (/Users/johndaniel/.local/share/sfdx/client/7.4.0-99233fd3af/node_modules/salesforce-alm/node_modules/bluebird/js/release/pro
mise.js:687:18)\n at Async._drainQueue (/Users/johndaniel/.local/share/sfdx/client/7.4.0-99233fd3af/node_modules/salesforce-alm/node_modules/bluebird/js/release/async.js:138:16)\n
at Async._drainQueues (/Users/johndaniel/.local/share/sfdx/client/7.4.0-99233fd3af/node_modules/salesforce-alm/node_modules/bluebird/js/release/async.js:148:10)\n at Immediate.Asy
nc.drainQueues [as _onImmediate] (/Users/johndaniel/.local/share/sfdx/client/7.4.0-99233fd3af/node_modules/salesforce-alm/node_modules/bluebird/js/release/async.js:17:14)\n at runCa
llback (timers.js:705:18)\n at tryOnImmediate (timers.js:676:5)\n at processImmediate (timers.js:658:5)\nOuter stack:\n at Function.wrap (/Users/johndaniel/.local/share/sfdx/c
lient/7.4.0-99233fd3af/node_modules/@salesforce/core/lib/sfdxError.js:151:27)\n at ApexTestRunCommand.catch (/Users/johndaniel/.local/share/sfdx/client/7.4.0-99233fd3af/node_modules
/salesforce-alm/dist/ToolbeltCommand.js:216:46)",
"warnings": []
}```
Other commands that are affected by this are:
- force:source:status
- force:source:pull
- force:source:push (yes, I listed above that it worked and it did, but now it doesn't work)
Is this an issue where the jwt connection works for a while and then the CLI needs to send a refreshtoken request and when that request occurs, it needs the jwtkeyfile to make that refresh request? Is there a way to observe this? Does this suggest that the connected app on Salesforce is not configured correctly?
This is working as designed. Calling Salesforce APIs requires an access token. Access tokens are valid for a certain amount of time before they expire. In the case of JWT auth there are no refresh tokens. When the access token expires, the CLI will use the persisted auth data to get a new, valid access token and update the local auth data so that API calls will succeed. For CI, you can use an environment variable for the path to your jwtkeyfile. @clairebianchi
@shetzel thanks for the update. That may be that it is working as designed, but it breaks the use case that I mentioned above about the "....Jenkins CI processes on SFDX projects" and that Salesforce recommends that the management of the jwtKeyFile on Jenkins is to store it in the "Jenkins Admin Credentials interface." (see documentation here, section 2b). The "Jenkins Admin Credentials interface" stores the file in another directory outside of the Jenkins workspace folder and thus the error. The force:auth:jwt:grant command gets the JWT file injected by the withCredentials( [ file( credentialsId: JWT_KEY_CRED_ID, variable: 'jwt_key_file') ] ) command.
Also, why would it be that I am able to execute a couple of commands (i.e. force:org:create, force:source:push, etc.) before the process fails on force:apex:test:run??
@clairebianchi
Sounds like there might still be an issue if you were able to run some commands and not others. Also maybe the documentation on Jenkins is wrong the way we are suggesting you set up isn't correct. @shetzel could you shed some light on why the other commands worked but test:run failed?
Commands will work as long as the token is valid or if the command doesn't make an API call, such as force:alias:list. When the token expires and you run a command that makes an API call, it will fail during auth.
As for Jenkins, we had a similar issue but it was resolved by wrapping all the steps inside a withCredentials block in order to guarantee that the location Jenkins creates to store the jwt key file remains the same. If multiple withCredential blocks are used then jenkins may actually place the key file in a different spot, and the persisted auth files that the CLI uses will point to an invalid jwtkeyfile location. Try using a single withCredentials block in your Jenkins script.
@shetzel - the force:apex:test:run command is in a different stage. I did wrap that command with the withCredentials block. That did not help. Still got the same error. I did not wrap the force:source:push with the withCredentials block and that part worked just fine again.
I am not certain how wrapping the force:apex:test:run command in a withCredentials block would do. The withCredentials command simply exposes the server.key file as an environment variable within that block. Since the force:apex:test:run command does not utilize the jwtKeyFile environment variable like the force:auth:jwt:grant does, I do see how this is helping.
Thoughts?
As long as everything is wrapped in the same withCredentials it should work. The auth files that are written during force:auth:jwt:grant and force:org:create reference the temporary location of the jwt file. That location should be there as long as the stages are executing within the same withCredentials scope, so the behind the scenes re-auth should work. Here is a sample Jenkinsfile from an old repo we used in the past:
` #!groovy import groovy.json.JsonSlurperClassic node {
def BUILD_NUMBER=env.BUILD_NUMBER
def RUN_ARTIFACT_DIR="tests/${BUILD_NUMBER}"
def SFDC_USERNAME
def HUB_ORG=env.HUB_ORG_DH
def SFDC_HOST = env.SFDC_HOST_DH
def JWT_KEY_CRED_ID = env.JWT_CRED_ID_DH
def CONNECTED_APP_CONSUMER_KEY=env.CONNECTED_APP_CONSUMER_KEY_DH
def toolbelt = tool 'toolbelt'
stage('checkout source') {
// when running in multi-branch job, one must issue this command
checkout scm
}
withCredentials([file(credentialsId: JWT_KEY_CRED_ID, variable: 'jwt_key_file')]) {
stage('Create Scratch Org') {
rc = sh returnStatus: true, script: "${toolbelt}/sfdx force:auth:jwt:grant --clientid ${CONNECTED_APP_CONSUMER_KEY} --username ${HUB_ORG} --jwtkeyfile ${jwt_key_file} --setdefaultdevhubusername --instanceurl ${SFDC_HOST}"
if (rc != 0) { error 'hub org authorization failed' }
// need to pull out assigned username
rmsg = sh returnStdout: true, script: "${toolbelt}/sfdx force:org:create --definitionfile config/project-scratch-def.json --json --setdefaultusername"
printf rmsg
def jsonSlurper = new JsonSlurperClassic()
def robj = jsonSlurper.parseText(rmsg)
if (robj.status != 0) { error 'org creation failed: ' + robj.message }
SFDC_USERNAME=robj.result.username
robj = null
}
stage('Push To Test Org') {
rc = sh returnStatus: true, script: "${toolbelt}/sfdx force:source:push --targetusername ${SFDC_USERNAME}"
if (rc != 0) {
error 'push failed'
}
// assign permset
rc = sh returnStatus: true, script: "${toolbelt}/sfdx force:user:permset:assign --targetusername ${SFDC_USERNAME} --permsetname DreamHouse"
if (rc != 0) {
error 'permset:assign failed'
}
}
stage('Run Apex Test') {
sh "mkdir -p ${RUN_ARTIFACT_DIR}"
timeout(time: 120, unit: 'SECONDS') {
rc = sh returnStatus: true, script: "${toolbelt}/sfdx force:apex:test:run --testlevel RunLocalTests --outputdir ${RUN_ARTIFACT_DIR} --resultformat tap --targetusername ${SFDC_USERNAME}"
if (rc != 0) {
error 'apex test run failed'
}
}
}
stage('collect results') {
junit keepLongStdio: true, testResults: 'tests/**/*-junit.xml'
}
}
} `
@shetzel -- a couple of questions:
- Have you had any success using that approach with the pipeline -> stages -> stage syntax?. The tests that I have run thus far prevent the
withCrendentialsblock from wrapping thepipelineblock aspipelinehas to be top tier. - Why does the CLI need to connect to the DevHub in the JWT Auth Flow for each command. I understand that it needs to authenticate to the DevHub in that manner, but once it is authenticated, doesn't the CLI get an access token to use from there forward? Why would the CLI not utilize the access token for the subsequent commands?
- The command fails when trying to access the scratch org. Why is the connection to the scratch org dependent on a JWT type flow? The connection to the DevHub needs to be JWT, I get that. But I don't understand why that is ? Can you shed some light on why this is the case or at least why this appears to be the case?
@shetzel -- additional question would be:
- Why is it that the
force:apex:test:runcommand fails when the jwtKeyFile is not present but theforce:source:pushcommand works. Both make API calls but only to the scratch org; not the Dev Hub
re: jenkins - Seems like you're using the newer, declarative method, and it appears that the withCredentials plugin has issues with that method. I don't have any experience with it so doubt I'll be much help there.
The CLI doesn't need to connect to the DevHub for each command. The auth for the scratch org uses the same flow as for the devhub, as opposed to the auth code from the scratch org signup request (in which case it would follow the refresh token flow). API requests to the scratch org should be using the access token obtained for the initial scratch org auth. It's possible that the jwtkeyfile is being checked when the access token is still valid, which sounds like what you're describing.
Assuming there is no access token expiration happening between source:push and apex:test:run, then you're correct that the code is checking the jwtkeyfile existence unnecessarily. I'll create an investigation so we can track this. @clairebianchi
@shetzel -- That is strange.
I see now that, if you have a devhubdefault connection that was setup via JWT, then any scratch org created by that connection will also have a JWT File reference -- accessToken and privateKey. If you have a devhubdefault connection that was setup via the Web Flow, the scratch org created has a accessToken/refreshToken setup in its definition file in the ~/.sfdx/test-XXXXXXexample.com.json file.
Do you know what the reason is to make that kind of distinction between DevHub-WebAuthFlow scratch orgs and DevHub-JWTFlow scratch orgs? Why would you not want to always use the accessToken/refreshToken setup for scratch orgs?
The decision predates me but I believe it was done that way for customer convenience. I reproduced the issue you're seeing and entered a bug for us to track internally. I would try working around the issue from the Jenkins side of things until the fix is released.
@shetzel -- I mentioned to @clairebianchi that I have a Salesforce support ticket active on this issue. It is case number 22563597.
@shetzel -- you mentioned that it would be "....done that way for customer convenience", but you are not certain. I hear ya. Personally, I cannot, at the moment, think of a use case where having the scratch org authenticate via JWT would be helpful at all, but as you point out, there may be a reason. If there is such a reason, I would love to know what that is.
My current hunch is that it is closer to a basic design flaw in the approach of how scratch org connections are managed when they are created from a devhub-jwt-type-connnection. I really do expect that we would find that the original CLI Command code was setup to match the devhub connection type for no specific reason.
Personally, I am thinking that scratch orgs created from a devhub-jwt-type-connnection should follow the same OAuth connection setup that devhub-webauth-type-connections would setup their scratch orgs with. That would definitely eliminate this issue that I am seeing here on Jenkins and allow SFDX CLI to be used in all ways that Jenkins would consider best practices for CI builds.
Any thoughts on this?
Could the way that the CLI manages scratch orgs created from a devhub-jwt-type-connnections be switched? Is there any reason that this could not be done?
As always, I appreciate the help on these matters!!! Cheers!
cc: @clairebianchi
@clairebianchi, @shetzel, and @sfdc-db-gmail -- Any update on this issue?
@ImJohnMDaniel I have this on our backlog and hope to get it picked up by the end of September
The behavior here seems to have changed again - it used to be possible to logout and back into the hub to update the cached path to the jwt file, but it now seems like the path to the file at the time of creating an org is also now pinned - and I'm not aware of a way to update that.
+ sfdx force:auth:jwt:grant -u [email protected] -f **** -i **** -d
Successfully authorized [email protected] with org ID 00D30000000jW0rEAE
+ sfdx force:org:list --all
=== Orgs
ALIAS USERNAME ORG ID CONNECTED STATUS
─── ───── ─────────────────────────── ────────────────── ────────────────
(D) [email protected] 00D30000000jW0rEAE Connected
ALIAS SCRATCH ORG NAME USERNAME ORG ID STATUS EXPIRATION DATE
───── ──────────────── ──────────────────────────────────────────────────────── ────────────────── ────── ───────────────
TPAY2 [email protected] 00D3F000000ETGsUAO Active 2019-10-01
TPAY2 [email protected] 00D1F000000FOhEUAW Active 2019-10-01
TPAY2 [email protected] 00D0R000000ERv5UAG Active 2019-10-01
TPAY2 [email protected] 00D3F000000ETDAUA4 Active 2019-10-01
TPAY2 [email protected] 00D1D00000022fOUAQ Active 2019-10-01
TPAY2 [email protected] 00D0R000000ERuqUAG Active 2019-10-01
TPAY2 [email protected] 00D1D00000022fsUAA Active 2019-10-01
TPAY2 [email protected] 00D1D00000022hZUAQ Active 2019-10-01
TPAY2 [email protected] 00D1F000000FOiRUAW Active 2019-10-01
TPAY2 [email protected] 00D3F000000ETE8UAO Active 2019-10-01
TPAY2 [email protected] 00D1D00000022h0UAA Active 2019-10-01
TPAY2 [email protected] 00D1D00000022h5UAA Active 2019-10-01
TPAY2 [email protected] 00DS0000003Mb60MAC Active 2019-10-01
TPAY2 [email protected] 00D1F000000FOgpUAG Active 2019-10-01
TPAY2 [email protected] 00D1F000000FOhYUAW Active 2019-10-01
TPAY2 [email protected] 00D1F000000FOiMUAW Active 2019-10-01
sfdx force:mdapi:deploy -u [email protected] -c -d build/package/result -l RunLocalTests -w 120
1131518 bytes written to /tmp/result.zip using 1698.190ms
Deploying /tmp/result.zip...
ERROR running force:mdapi:deploy: ENOENT: no such file or directory, open '/jenkins/jenkins/workspace/TPAY2-CreateCIOrg@tmp/secretFiles/982914f0-493a-46a4-b9c4-bfa3e40ddf3f/server.key'
This is not every command. sfdx force:org:open -r -u [email protected] works even though force:mdapi:deploy does not.
Copying the jenkins temp path to the key to a relative path at the start of the jobs requiring it solves the issue.
Hi All,
I am trying to use the mdapi deploy command and I am getting the same issue. Org Authorization is success however during mdapi deploy command it is failing saying: ERROR running force:source:deploy: ENOENT: no such file or directory, open '/app/jenkins/workspace/Salesforce/deploy@tmp/secretFiles/15353656372736372898727829-2626/server.key'
Even I am written the script wrapped in the same withCredentials block.
Can anyone help on this?
@sanpatnaik Here is the known issue that is logged for this https://success.salesforce.com/issues_view?id=a1p3A000001SGxLQAW
We were unable to get to this issue last month, it is on our board and I am hoping it will get picked up in the next few weeks.
@sanpatnaik Hello Friend! I was trying to do a similar thing, calling the mdapi deploy command. I gave up after a couple hours of trying to use a jenkinsfile to try this workaround. I prefer using Freestyle Jenkins projects because they are WAY easier to set up.
- I figured out what the problem was!
- When using the secret file, as others have pointed out, the location of that file only exists until the end of the build. BUT! if you have already authenticated an org (even just in a prior build), it will still be trying to reference the file location that no longer exists!
- The solution I found was to logout of the org you are trying to authorize as the first part of the build step. Then re-authorize, then do your logic. Here is the build code I am using that is working now:
export SFDX_USE_GENERIC_UNIX_KEYCHAIN=true
cd test-ci
echo y | /usr/local/bin/sfdx force:auth:logout --targetusername my-devhub
/usr/local/bin/sfdx force:auth:jwt:grant --clientid {myclientId} \
--jwtkeyfile $JWT_AUTH_KEY --username {myusername} \
--setdefaultdevhubusername --setalias my-devhub
/usr/local/bin/sfdx force:mdapi:deploy -d mdapi-out -u my-devhub -w -1 -c
Hi @ImJohnMDaniel are you still experiencing this issue? I've tried reproducing it on my local CLI and can't reproduce it. Have you tried running these commands through your own CLI and not Jenkins?
@williamruemmele-sf and @clairebianchi -- I just checked again and this issue is still present. I was able to reproduce the issue locally; outside of Jenkins.
Here is the CLI version info that I am currently on...
sfdx version
sfdx-cli/7.31.0-6a986f8be5 darwin-x64 node-v10.15.3
sfdx plugins --core
@oclif/plugin-commands 1.2.3 (core)
@oclif/plugin-help 2.2.1 (core)
@oclif/plugin-not-found 1.2.3 (core)
@oclif/plugin-plugins 1.7.8 (core)
@oclif/plugin-update 1.3.9 (core)
@oclif/plugin-warn-if-update-available 1.7.0 (core)
@oclif/plugin-which 1.0.3 (core)
@salesforce/sfdx-diff 0.0.3
@salesforce/sfdx-trust 3.0.5 (core)
analytics 1.2.1 (core)
generator 1.1.1 (core)
salesforcedx 47.4.0 (core)
├─ salesforcedx-templates 47.3.2 (core)
└─ salesforce-alm 47.7.0 (core)
FWIW, I am happy to catch up on a call to discuss this and show you what is happening. Just let me know.
@sanpatnaik Hello Friend! I was trying to do a similar thing, calling the mdapi deploy command. I gave up after a couple hours of trying to use a jenkinsfile to try this workaround. I prefer using Freestyle Jenkins projects because they are WAY easier to set up.
I figured out what the problem was!
- When using the secret file, as others have pointed out, the location of that file only exists until the end of the build. BUT! if you have already authenticated an org (even just in a prior build), it will still be trying to reference the file location that no longer exists!
The solution I found was to logout of the org you are trying to authorize as the first part of the build step. Then re-authorize, then do your logic. Here is the build code I am using that is working now:
export SFDX_USE_GENERIC_UNIX_KEYCHAIN=true cd test-ci echo y | /usr/local/bin/sfdx force:auth:logout --targetusername my-devhub /usr/local/bin/sfdx force:auth:jwt:grant --clientid {myclientId} \ --jwtkeyfile $JWT_AUTH_KEY --username {myusername} \ --setdefaultdevhubusername --setalias my-devhub /usr/local/bin/sfdx force:mdapi:deploy -d mdapi-out -u my-devhub -w -1 -c
Hello - Could you please provide more details on this as I am still facing the issue as John already mentioned - would love to have a workaround! Are you trying to say, authorize, logout and then re-authorize?
@siddharthmani -- The main issue stems from the fact that when you authenticate to the DevHub with a JWT token, the CLI using that connection will create all scratch orgs also with a JWT token. Best practices in Jenkins and declarative pipelines with Jenkinsfile usage don't really work well with this approach. The best practice assumes that you use the withCredentials tag in the Jenkinsfile to expose the JWT_AUTH_KEY but you are only exposing it for the force:auth:jwt:grant command to the DevHub. Once you exit the withCrendetials tag, you typically are only calling out to the scratch org but since that scratch's OAuth connection is based on JWT and the JWT_AUTH_KEY file is no longer available, it will eventually fail somewhere in the process (for me, it failed once I got to the test run stage).
I can confirm that there is movement on this issue. I talked with @clairebianchi and @williamruemmele-sf yesterday about this issue. I gave them my recommendation that scratch orgs should always be created with a standard WebFlow OAuth approach and not a JWT token based approach. Connections to DevHubs in a CI "headless" scenario should definitely still use the JWT token based approach. They were going to discuss with their teams let us know what they plan to do as a next step.
In the meantime, if you need a work around, what I did was find where the CI Server is putting the hidden JWT_AUTH_KEY file to expose it to the build. Then copy that file to a location inside the project workspace during the withCredentials tag. Then reference the key file from the new location to auth against the DevHub. Leave the file there in the workspace until the end of the build so that commands to the scratch org can find it. Then you just delete JWT_AUTH_KEY file from the project workspace as part of the last cleanup step. It is not the most secure approach but it is a reasonable workaround until @clairebianchi's team can fix this. If you need Jenkinsfile examples of how this works, just let me know.
Thank you so much for the detailed explanation John. Indeed my build also fails with the same error message although I am not creating any scratch orgs as of now. What I am essentially doing consists of 2 stages (both wrapped in "withcredentials") - the first one is the jwt authorisation which succeeds and the second one is the actual force:source:deploy command which fails mentioning that it cannot find the key. For now I tried jc-torrents approach and it seemed to work for me. What I am doing is after successfully authorising on the first stage, when I move to the second stage - I do a logout and then again authorise and deploy using "withcredentials" - seems redundant, but does work for me (for now!!).
@sanpatnaik Hello Friend! I was trying to do a similar thing, calling the mdapi deploy command. I gave up after a couple hours of trying to use a jenkinsfile to try this workaround. I prefer using Freestyle Jenkins projects because they are WAY easier to set up.
I figured out what the problem was!
- When using the secret file, as others have pointed out, the location of that file only exists until the end of the build. BUT! if you have already authenticated an org (even just in a prior build), it will still be trying to reference the file location that no longer exists!
The solution I found was to logout of the org you are trying to authorize as the first part of the build step. Then re-authorize, then do your logic. Here is the build code I am using that is working now:
export SFDX_USE_GENERIC_UNIX_KEYCHAIN=true cd test-ci echo y | /usr/local/bin/sfdx force:auth:logout --targetusername my-devhub /usr/local/bin/sfdx force:auth:jwt:grant --clientid {myclientId} \ --jwtkeyfile $JWT_AUTH_KEY --username {myusername} \ --setdefaultdevhubusername --setalias my-devhub /usr/local/bin/sfdx force:mdapi:deploy -d mdapi-out -u my-devhub -w -1 -cHello - Could you please provide more details on this as I am still facing the issue as John already mentioned - would love to have a workaround! Are you trying to say, authorize, logout and then re-authorize?
Hello! My issue in particular was that, once authorized, any additional builds will try to use the same authentication, because SFDX doesn't remove authorized orgs. I am hosting jenkins on an ec2, so not locally. I log out of the org in the beginning of the build so that it is forced to re-authorize the org using the secret file. Because the secret file is only in a particular location for the duration of the build, if you don't logout and then reauthorize, any commands that require authorization will look in the location from the first time you authorized the org to find the authentication file (the secret file).