terraform-provider-github icon indicating copy to clipboard operation
terraform-provider-github copied to clipboard

[BUG]: failed to create OAuth token from GitHub App

Open mattduguid opened this issue 1 month ago • 15 comments

Expected Behavior

that terraform plan would succeed

Actual Behavior

terraform plan failed

but if we pin back to an earlier provider version before https://github.com/integrations/terraform-provider-github/releases/tag/v6.9.0 it works fine, so that is our current workaround

Terraform Version

terraform versions tested, 1.13.5 on [CUSTOMER-A].ghe.com 1.14.0 on [CUSTOMER-B].ghe.com

Affected Resource(s)

  • provider "github"

Terraform Configuration Files

provider "github" {
  alias    = "enterprise"
  owner    = var.ghe_owner
  base_url = var.ghe_base_url
  app_auth {
    id              = var.ghe_app_id
    installation_id = var.ghe_installation_id_enterprise
    pem_file        = file("${path.module}/../../modules/ghe/private.pem") # dynamically generated at runtime from gh secret on ephemeral gh agent
  }
}

provider "github" {
  alias    = "organization"
  owner    = var.ghe_owner
  base_url = var.ghe_base_url
  app_auth {
    id              = var.ghe_app_id
    installation_id = var.ghe_installation_id_organization
    pem_file        = file("${path.module}/../../modules/ghe/private.pem") # dynamically generated at runtime from gh secret on ephemeral gh agent
  }
}

Steps to Reproduce

terraform apply

Debug Output

2025-12-07T23:57:06.406Z [DEBUG] provider: starting plugin: path=.terraform/providers/registry.terraform.io/integrations/github/6.9.0/linux_amd64/terraform-provider-github_v6.9.0 args=[".terraform/providers/registry.terraform.io/integrations/github/6.9.0/linux_amd64/terraform-provider-github_v6.9.0"]
2025-12-07T23:57:06.407Z [DEBUG] provider: plugin started: path=.terraform/providers/registry.terraform.io/integrations/github/6.9.0/linux_amd64/terraform-provider-github_v6.9.0 pid=2118
2025-12-07T23:57:06.407Z [DEBUG] provider: waiting for RPC address: plugin=.terraform/providers/registry.terraform.io/integrations/github/6.9.0/linux_amd64/terraform-provider-github_v6.9.0
2025-12-07T23:57:06.411Z [INFO]  provider.terraform-provider-github_v6.9.0: configuring server automatic mTLS: timestamp=2025-12-07T23:57:06.411Z
2025-12-07T23:57:06.427Z [DEBUG] provider.terraform-provider-github_v6.9.0: plugin address: address=/tmp/plugin56106065 network=unix timestamp=2025-12-07T23:57:06.427Z
2025-12-07T23:57:06.427Z [DEBUG] provider: using plugin: version=5
2025-12-07T23:57:06.441Z [WARN]  ValidateProviderConfig from "provider[\"registry.terraform.io/integrations/github\"].organization" changed the config value, but that value is unused
2025-12-07T23:57:06.442Z [INFO]  provider.terraform-provider-github_v6.9.0: 2025/12/07 23:57:06 [INFO] Selecting owner  from GITHUB_OWNER environment variable: timestamp=2025-12-07T23:57:06.442Z
Error: -07T23:57:06.519Z [ERROR] provider.terraform-provider-github_v6.9.0: Response contains error diagnostic: @caller=github.com/hashicorp/[email protected]/tfprotov5/internal/diag/diagnostics.go:58 @module=sdk.proto diagnostic_detail="failed to create OAuth token from GitHub App: " tf_provider_addr=provider tf_req_id=502a1d0f-501f-1fc5-49f8-805ccc51dab5 tf_rpc=Configure diagnostic_severity=ERROR diagnostic_summary=Error tf_proto_version=5.10 timestamp=2025-12-07T23:57:06.519Z
Error: -07T23:57:06.520Z [ERROR] vertex "provider[\"registry.terraform.io/integrations/github\"].organization" error: Error
2025-12-07T23:57:06.520Z [WARN]  Planning encountered errors, so plan is not applyable
2025-12-07T23:57:06.520Z [INFO]  backend/local: plan operation completed
2025-12-07T23:57:06.520Z [INFO]  backend/local: writing plan output to: plan.tfplan
Planning failed. Terraform encountered an error while generating this plan.
╷
│ Error: Error
│ 
│   with provider["registry.terraform.io/integrations/github"].organization,
│   on main.tf line 31, in provider "github":
│   31: provider "github" {
│ 
│ failed to create OAuth token from GitHub App: 
╵
2025-12-07T23:57:06.839Z [DEBUG] provider.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = error reading from server: EOF"
2025-12-07T23:57:06.840Z [INFO]  provider: plugin process exited: plugin=.terraform/providers/registry.terraform.io/integrations/github/6.9.0/linux_amd64/terraform-provider-github_v6.9.0 id=2118
2025-12-07T23:57:06.840Z [DEBUG] provider: plugin exited

Error: Process completed with exit code 1.

Panic Output

N/A

Code of Conduct

  • [x] I agree to follow this project's Code of Conduct

mattduguid avatar Dec 08 '25 02:12 mattduguid

👋 Hi! Thank you for this contribution! Just to let you know, our GitHub SDK team does a round of issue and PR reviews twice a week, every Monday and Friday! We have a process in place for prioritizing and responding to your input. Because you are a part of this community please feel free to comment, add to, or pick up any issues/PRs that are labeled with Status: Up for grabs. You & others like you are the reason all of this works! So thank you & happy coding! 🚀

github-actions[bot] avatar Dec 08 '25 02:12 github-actions[bot]

I would like to second this one, when running any version below 6.9.0, such as 6.8.3 the plans do not fail, when running the version as 6.9.0, the following error is observed:

"Error: BaseURL must have a trailing slash, but "https://api.github.com/****" does not"

ZacharyW1992 avatar Dec 08 '25 18:12 ZacharyW1992

Hey @mattduguid @ZacharyW1992 thanks for raising the issue!

This definitely sounds like a problem. Which GH product are you using?

@ZacharyW1992 I assume you are using either GHEC or DOTCOM (aka github.com)? @mattduguid Since you are configuring URLs are you using GHES?

I just tested locally with a GHEC instance and I don't get this error :/ Do you consistently get this error? Did it start erroring right after upgrade or after a few runs?

deiga avatar Dec 09 '25 07:12 deiga

@deiga both of our tests used 2 different instances of (GHE) GitHub Enterprise deployed via data residency in Australia region

terraform v1.13.5 on [CUSTOMER-A].ghe.com terraform v1.14.0 on [CUSTOMER-B].ghe.com

issue is consistent with v6.9.0 of provider, v6.8.1 & v6.8.3 both still work

thanks

mattduguid avatar Dec 09 '25 08:12 mattduguid

@mattduguid Thanks for the details! We need more info to figure out what's causing this. We don't have access to a GHEC Data Residency instance unfortunately :/

  • Is [CUSTOMER-A].ghe.com the exact format of the base_url argument in the provider? If not, could you post the exact format?
  • Could you run TF_LOG=trace terraform plan and paste the output with the latest provider (redact any sensitive information)

deiga avatar Dec 09 '25 10:12 deiga

@mattduguid do you have a trailing / in the base URL you're providing? If not does it make any difference if you add it?

stevehipwell avatar Dec 09 '25 12:12 stevehipwell

I have more or less the same issue, just that I'm using token instead. base_url=https://temp.ghe.com/ where temp is our organization. If i run it on version v6.8.3 it works fine, but on v6.0.0 it fails on the plan with the error Error: GET https://temp.ghe.com/login?return_to=https%3A%2F%2Ftemp.ghe.com%2Frepos%2Ftemp2%2Ftemp3: 406 []

thomasfjortoft avatar Dec 09 '25 13:12 thomasfjortoft

@stevehipwell To me, it looks like this change had some side effect that made it break in 6.9.0 https://github.com/integrations/terraform-provider-github/pull/2951

thomasfjortoft avatar Dec 09 '25 14:12 thomasfjortoft

@thomasfjortoft yes that's my understanding too. I'm just working on a PR with tests to resolve this and make sure that it can't regress.

stevehipwell avatar Dec 09 '25 14:12 stevehipwell

@thomasfjortoft yes that's my understanding too. I'm just working on a PR with tests to resolve this and make sure that it can't regress.

Great, thanks for looking/fixing this in no time! 👏

thomasfjortoft avatar Dec 09 '25 14:12 thomasfjortoft

Hey @mattduguid @ZacharyW1992 thanks for raising the issue!

This definitely sounds like a problem. Which GH product are you using?

@ZacharyW1992 I assume you are using either GHEC or DOTCOM (aka github.com)? @mattduguid Since you are configuring URLs are you using GHES?

I just tested locally with a GHEC instance and I don't get this error :/ Do you consistently get this error? Did it start erroring right after upgrade or after a few runs?

This is a GHEC implementation, thanks for the quick response and turnaround!

ZacharyW1992 avatar Dec 09 '25 14:12 ZacharyW1992

@mattduguid do you have a trailing / in the base URL you're providing? If not does it make any difference if you add it?

yes we do use trailing slash on both, let me know if you need any other tests, that original debug output was from using v6.9.0 of the provider where we first hit the issue, thanks

ghe_base_url = "https://[CUSTOMER_X].ghe.com/"

mattduguid avatar Dec 11 '25 20:12 mattduguid

@mattduguid don't you currently need to use https://api.<customer>.ghec.com)?

stevehipwell avatar Dec 12 '25 01:12 stevehipwell

@stevehipwell dont believe so, we have been using those URL's fine for automating everything to date across multiple customers, the only change where it broke was changing to the newer 6.9.0 provider, pinning backwards to earlier still works

mattduguid avatar Dec 12 '25 01:12 mattduguid

@mattduguid are we talking about GHEC with data residency? If so the docs say a api. prefix is required? What does the URL look like if you want to curl a rest endpoint?

stevehipwell avatar Dec 12 '25 10:12 stevehipwell

apologies for delay @stevehipwell just had our weekend here in NZ, tried both on the v.6.9.0 of the provider for https://registry.terraform.io/providers/integrations/github/latest/docs#base_url-1 results were,

test with "api." in url, https:///api.[CUSTOMER_X].ghe.com/

result,

Planning failed. Terraform encountered an error while generating this plan.
╷
│ Error: Error
│ 
│   with provider["registry.terraform.io/integrations/github"].organization,
│   on main.tf line 31, in provider "github":
│   31: provider "github" {
│ 
│ Post
│ "https:///api.[CUSTOMER_X].ghe.com/api/v3/app/installations/[CUSTOMER_X_INSTALLATION_ID]/access_tokens":
│ http: no Host in request URL

test without "api." in the url, https:///[CUSTOMER_X].ghe.com/

result,

Planning failed. Terraform encountered an error while generating this plan.
╷

│ Error: Error
│ 
│   with provider["registry.terraform.io/integrations/github"].organization,
│   on main.tf line 31, in provider "github":
│   31: provider "github" {
│ 
│ failed to create OAuth token from GitHub App: 
╵

Error: Process completed with exit code 1.

mattduguid avatar Dec 14 '25 21:12 mattduguid

@mattduguid the bottom URL appears to have an extra /? But what I was really after was a working curl URL.

stevehipwell avatar Dec 14 '25 21:12 stevehipwell

@stevehipwell this is the variable we have in the .tfvars file which has worked in all provider versions prior to v6.9.0, its passed into the enterprise and organisation as shown at start of issue in section "Terraform Configuration Files"

ghe_base_url = "https:///[CUSTOMER_X].ghe.com/"

i also just tested without the "/" and got the same error i got when adding the "api." to front of URL

Planning failed. Terraform encountered an error while generating this plan.

╷
│ Error: Error
│ 
│   with provider["registry.terraform.io/integrations/github"].organization,
│   on main.tf line 31, in provider "github":
│   31: provider "github" {
│ 
│ Post
│ "https:///[CUSTOMER_X].ghe.com/api/v3/app/installations/[CUSTOMER_X_INSTALLATION_ID]/access_tokens":
│ http: no Host in request URL
╵

Error: Process completed with exit code 1.

mattduguid avatar Dec 14 '25 21:12 mattduguid

@mattduguid the URL you're using has no host (///) so fails in the newer code with tighter validation. Could you please try https://api.[CUSTOMER_X].ghe.com/? But if you look at the REST API docs you should be able to run a curl command to see what the actual URL should be.

stevehipwell avatar Dec 14 '25 21:12 stevehipwell

@stevehipwell tested using v.6.9.0 with "api." and tried with and without trailing "/"

Planning failed. Terraform encountered an error while generating this plan.

╷
│ Error: Error
│ 
│   with provider["registry.terraform.io/integrations/github"].organization,
│   on main.tf line 31, in provider "github":
│   31: provider "github" {
│ 
│ Post
│ "https:///api.[CUSTOMER_X].ghe.com/api/v3/app/installations/[CUSTOMER_X_INSTALLATION_ID]/access_tokens":
│ http: no Host in request URL

also tried the API via curl which does work with the "api." but not without which is expected,

curl -L -H "Accept: application/vnd.github+json" -H "Authorization: Bearer ghp_[REMOVED]" -H "X-GitHub-Api-Version: 2022-11-28" https://api.[CUSTOMER_X].ghe.com/user
{
  "login": "matt-duguid",
  ...etc...
}

mattduguid avatar Dec 14 '25 22:12 mattduguid

@mattduguid you need to remove the third / after https: otherwise it will not work.

stevehipwell avatar Dec 14 '25 22:12 stevehipwell

@stevehipwell fixed the typo its now working 100% on v6.9.0 thanks for your help

ghe_base_url = "https://api.[CUSTOMER_X].ghe.com/"

mattduguid avatar Dec 14 '25 22:12 mattduguid

@mattduguid thanks for letting me know. The PR to close this issue would allow you to use either the API or non-API URL as it appears I managed to inadvertently remove support for not providing the api. prefix in a previous PR.

stevehipwell avatar Dec 14 '25 22:12 stevehipwell

In my case, github_app_token was mutating the provider client's BaseURL in-place. If you didn't allocate it, don't mutate it.

https://github.com/integrations/terraform-provider-github/blob/a3b39ede0d4a813b81ed93413c07d8ac58343743/github/apps.go#L41 https://github.com/integrations/terraform-provider-github/pull/2951

jstachowiak avatar Dec 15 '25 12:12 jstachowiak

@jstachowiak this has been fixed in #2990.

stevehipwell avatar Dec 15 '25 17:12 stevehipwell