integrations
[BUG] Provider config not being loaded when passed directly in config block
Terraform Version
Terraform v1.2.4
on linux_amd64
Affected Resource(s)
Provider config can affect resources indirectly, for example:
- github_repository -> The owner can be implicitly set via the PAT token passed to the provider.
If this issue appears to affect multiple resources, it may be an issue with Terraform's core, so please mention this.
Terraform Configuration Files
terraform {
required_providers {
github = {
source = "integrations/github"
version = "~> 4.0"
}
}
}
provider "github" {
token = "<PAT>"
}
resource "github_repository" "project_repository" {
name = "testing"
description = "test description"
visibility = "public"
template {
owner = "IgorAssuncao"
repository = "igor-nodejs-boilerplate"
}
}
Debug Output
Panic Output
N/A
Expected Behavior
According to the documentation we can set the PAT directly in the provider config and the owner of the provided PAT will be used implicitly.
Actual Behavior
The owner isn't being set up accordingly but only when I provide the PAT in the provider config.
If I use the PAT as an environment variable then everything works as expected.
Steps to Reproduce
Please list the steps required to reproduce the issue, for example:
- Configure the provider providing a PAT directly in the provider config block.
- Define a
github_repositoryresource. - Run
TF_LOG="DEBUG" terraform apply, provideyeswhen requested and check therequestsection of the output of the failed creation (should be the same as in the gist).
Important Factoids
N/A
References
N/A
Extras
Maybe if this is the desired behaviour then it should be documented that you can only provide a PAT as an environment variable.
Hmm...I'm unable to reproduce this in the latest release, v4.29.0. I've used the following Terraform snippet for testing purposes:
provider "github" {
#owner = "kfcampbell-terraform-provider"
#token = "REDACTED"
}
terraform {
required_providers {
github = {
source = "integrations/github"
}
}
}
resource "github_repository" "test_repository" {
name = "test-terraform-config-block-repo"
description = "testing for https://github.com/integrations/terraform-provider-github/issues/1231"
visibility = "private"
}
and tested the following cases:
- provider block config sets owner and PAT (no environment variable config set)
- provider block config sets PAT (no environment variable config set)
- environment variables set owner and PAT (no provider block config set)
- environment variable sets PAT (no provider block config set)
In cases 1 and 3, my testing organization was correctly set as the owner of the created repo. In cases 2 and 4, my personal account was correctly inferred as the owner of the token and became the owner of the created repo.
In none of the cases did the repo ever fail to create via terraform apply or be removed via terraform destroy.
Are you still seeing this issue in the latest version of the provider? If so, can you share a little bit more about your reproduction case?
@kfcampbell
Well, I was testing with version 4.27.1 and also tested right now with 4.29. I think I needed to describe the issue a little better.
I was working with a module I created for creating multiple github_repository and by doing so I discovered that when I have a PAT set in the providers config file it's not passed into the module.
So my code is like this:
./terraform-project/
- modules/
-
- github/
-
-
- main.tf
-
-
-
- variables.tf
-
-
-
- outputs.tf
-
- project/
-
- main.tf
-
- providers.tf
My project/main.tf is the following:
module "github_repository2" {
source = "../modules/github"
project_name = "example2"
project_description = "example2"
project_visibility = "public"
template_repository_info = {
owner = "IgorAssuncao"
repository = "igor-nodejs-boilerplate"
}
}
My project/providers.tf:
terraform {
required_providers {
github = {
source = "integrations/github"
version = "~> 4.29"
}
}
}
provider "github" {
token = "REDACTED"
}
My modules/github/main.tf:
resource "github_repository" "project_repository" {
name = var.project_name
description = var.project_description
visibility = var.project_visibility
template {
owner = try(var.template_repository_info["owner"], "")
repository = try(var.template_repository_info["repository"], "")
}
}
By running TF_LOG="DEBUG" terraform apply with the provider correctly setup the log says:
module.github_repository2.github_repository.project_repository: Creating...
2022-08-24T14:43:55.371-0300 [INFO] Starting apply for module.github_repository2.github_repository.project_repository
2022-08-24T14:43:55.371-0300 [DEBUG] module.github_repository2.github_repository.project_repository: applying the planned Create change
2022-08-24T14:43:55.371-0300 [DEBUG] provider.terraform-provider-github_v4.29.0: 2022/08/24 14:43:55 [DEBUG] setting computed for "branches" from ComputedKeys
2022-08-24T14:43:55.372-0300 [DEBUG] provider.terraform-provider-github_v4.29.0: 2022/08/24 14:43:55 [DEBUG] Github API Request Details:
2022-08-24T14:43:55.372-0300 [DEBUG] provider.terraform-provider-github_v4.29.0: ---[ REQUEST ]---------------------------------------
2022-08-24T14:43:55.372-0300 [DEBUG] provider.terraform-provider-github_v4.29.0: POST /repos/IgorAssuncao/igor-nodejs-boilerplate/generate HTTP/1.1
2022-08-24T14:43:55.372-0300 [DEBUG] provider.terraform-provider-github_v4.29.0: Host: api.github.com
2022-08-24T14:43:55.372-0300 [DEBUG] provider.terraform-provider-github_v4.29.0: User-Agent: go-github
2022-08-24T14:43:55.372-0300 [DEBUG] provider.terraform-provider-github_v4.29.0: Content-Length: 72
2022-08-24T14:43:55.372-0300 [DEBUG] provider.terraform-provider-github_v4.29.0: Accept: application/vnd.github.baptiste-preview+json
2022-08-24T14:43:55.372-0300 [DEBUG] provider.terraform-provider-github_v4.29.0: Content-Type: application/json
2022-08-24T14:43:55.372-0300 [DEBUG] provider.terraform-provider-github_v4.29.0: Accept-Encoding: gzip
2022-08-24T14:43:55.372-0300 [DEBUG] provider.terraform-provider-github_v4.29.0:
2022-08-24T14:43:55.372-0300 [DEBUG] provider.terraform-provider-github_v4.29.0: {
2022-08-24T14:43:55.372-0300 [DEBUG] provider.terraform-provider-github_v4.29.0: "name": "example2",
2022-08-24T14:43:55.372-0300 [DEBUG] provider.terraform-provider-github_v4.29.0: "owner": "",
2022-08-24T14:43:55.372-0300 [DEBUG] provider.terraform-provider-github_v4.29.0: "description": "example2",
2022-08-24T14:43:55.372-0300 [DEBUG] provider.terraform-provider-github_v4.29.0: "private": false
2022-08-24T14:43:55.372-0300 [DEBUG] provider.terraform-provider-github_v4.29.0: }
2022-08-24T14:43:55.372-0300 [DEBUG] provider.terraform-provider-github_v4.29.0:
2022-08-24T14:43:55.372-0300 [DEBUG] provider.terraform-provider-github_v4.29.0: -----------------------------------------------------
2022-08-24T14:43:55.590-0300 [DEBUG] provider.terraform-provider-github_v4.29.0: 2022/08/24 14:43:55 [DEBUG] Github API Response Details:
2022-08-24T14:43:55.590-0300 [DEBUG] provider.terraform-provider-github_v4.29.0: ---[ RESPONSE ]--------------------------------------
2022-08-24T14:43:55.590-0300 [DEBUG] provider.terraform-provider-github_v4.29.0: HTTP/2.0 404 Not Found
2022-08-24T14:43:55.590-0300 [DEBUG] provider.terraform-provider-github_v4.29.0: Access-Control-Allow-Origin: *
2022-08-24T14:43:55.590-0300 [DEBUG] provider.terraform-provider-github_v4.29.0: Access-Control-Expose-Headers: ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset
2022-08-24T14:43:55.590-0300 [DEBUG] provider.terraform-provider-github_v4.29.0: Content-Security-Policy: default-src 'none'
2022-08-24T14:43:55.590-0300 [DEBUG] provider.terraform-provider-github_v4.29.0: Content-Type: application/json; charset=utf-8
2022-08-24T14:43:55.590-0300 [DEBUG] provider.terraform-provider-github_v4.29.0: Date: Wed, 24 Aug 2022 20:45:29 GMT
2022-08-24T14:43:55.590-0300 [DEBUG] provider.terraform-provider-github_v4.29.0: Referrer-Policy: origin-when-cross-origin, strict-origin-when-cross-origin
2022-08-24T14:43:55.590-0300 [DEBUG] provider.terraform-provider-github_v4.29.0: Server: GitHub.com
2022-08-24T14:43:55.590-0300 [DEBUG] provider.terraform-provider-github_v4.29.0: Strict-Transport-Security: max-age=31536000; includeSubdomains; preload
2022-08-24T14:43:55.590-0300 [DEBUG] provider.terraform-provider-github_v4.29.0: Vary: Accept-Encoding, Accept, X-Requested-With
2022-08-24T14:43:55.590-0300 [DEBUG] provider.terraform-provider-github_v4.29.0: X-Content-Type-Options: nosniff
2022-08-24T14:43:55.590-0300 [DEBUG] provider.terraform-provider-github_v4.29.0: X-Frame-Options: deny
2022-08-24T14:43:55.590-0300 [DEBUG] provider.terraform-provider-github_v4.29.0: X-Github-Media-Type: github.v3; param=baptiste-preview; format=json
2022-08-24T14:43:55.590-0300 [DEBUG] provider.terraform-provider-github_v4.29.0: X-Github-Request-Id: 273F:44C5:6EBCAF:75B606:63068DE8
2022-08-24T14:43:55.590-0300 [DEBUG] provider.terraform-provider-github_v4.29.0: X-Ratelimit-Limit: 60
2022-08-24T14:43:55.590-0300 [DEBUG] provider.terraform-provider-github_v4.29.0: X-Ratelimit-Remaining: 56
2022-08-24T14:43:55.590-0300 [DEBUG] provider.terraform-provider-github_v4.29.0: X-Ratelimit-Reset: 1661377108
2022-08-24T14:43:55.590-0300 [DEBUG] provider.terraform-provider-github_v4.29.0: X-Ratelimit-Resource: core
2022-08-24T14:43:55.590-0300 [DEBUG] provider.terraform-provider-github_v4.29.0: X-Ratelimit-Used: 4
2022-08-24T14:43:55.590-0300 [DEBUG] provider.terraform-provider-github_v4.29.0: X-Xss-Protection: 0
2022-08-24T14:43:55.590-0300 [DEBUG] provider.terraform-provider-github_v4.29.0:
2022-08-24T14:43:55.590-0300 [DEBUG] provider.terraform-provider-github_v4.29.0: {
2022-08-24T14:43:55.590-0300 [DEBUG] provider.terraform-provider-github_v4.29.0: "message": "Not Found",
2022-08-24T14:43:55.590-0300 [DEBUG] provider.terraform-provider-github_v4.29.0: "documentation_url": "https://docs.github.com/rest/reference/repos#create-a-repository-using-a-template"
2022-08-24T14:43:55.590-0300 [DEBUG] provider.terraform-provider-github_v4.29.0: }
2022-08-24T14:43:55.590-0300 [DEBUG] provider.terraform-provider-github_v4.29.0: -----------------------------------------------------
2022-08-24T14:43:55.591-0300 [ERROR] vertex "module.github_repository2.github_repository.project_repository" error: POST https://api.github.com/repos/IgorAssuncao/igor-nodejs-boilerplate/generate: 404 Not Found []
╷
│ Error: POST https://api.github.com/repos/IgorAssuncao/igor-nodejs-boilerplate/generate: 404 Not Found []
│
│ with module.github_repository2.github_repository.project_repository,
│ on ../modules/github/main.tf line 1, in resource "github_repository" "project_repository":
│ 1: resource "github_repository" "project_repository" {
As you can see, the owner is not loaded into the module but this behavior does not occur when I don't reference any module.
I'm sorry, I've attempted to narrow the reproduction case down a bit and I've been unable to come to a satisfactory conclusion. Currently, I have a directory structure that looks like the following:
sh$ ls -R
.:
modules/ project/
./modules:
main.tf
./project:
main.tf
modules/main.tf contains the following:
resource "github_repository" "project_repository" {
name = "test-issue-1231"
description = "testing for https://github.com/integrations/terraform-provider-github/issues/1231"
visibility = "private"
template_repository_info = {
owner = "kfcampbell-terraform-provider"
repository = "test-terraform-config-block-repo"
}
}
where kfcampbell-terraform-provider is an organization and test-terraform-config-block-repo is a repo within the org that's marked as a template repository.
project/main.tf consists of the following:
provider "github" {
token = "REDACTED"
owner = "kfcampbell-terraform-provider"
}
terraform {
required_providers {
github = {
source = "integrations/github"
}
}
}
module "github_repository2" {
source = "../modules/"
name = "example2"
description = "example2"
visibility = "private"
template {
owner = try(var.template_repository_info["owner"], "")
repository = try(var.template_repository_info["repository"], "")
}
}
When I change directories into the project folder and run terraform apply, I get the following output:
│ Error: Unsupported argument
│
│ on main.tf line 17, in module "github_repository2":
│ 17: name = "example2"
│
│ An argument named "name" is not expected here.
╵
╷
│ Error: Unsupported argument
│
│ on main.tf line 18, in module "github_repository2":
│ 18: description = "example2"
│
│ An argument named "description" is not expected here.
╵
╷
│ Error: Unsupported argument
│
│ on main.tf line 19, in module "github_repository2":
│ 19: visibility = "private"
│
│ An argument named "visibility" is not expected here.
╵
╷
│ Error: Unsupported block type
│
│ on main.tf line 21, in module "github_repository2":
│ 21: template {
│
│ Blocks of type "template" are not expected here.
I'm not sure what I'm doing wrong here. Can you help make the reproduction case more simple for me?
You can remove the name, description, visibility and template variables from the module github_repository2 because you're already declaring them directly in the resource. That's what the error is about.
And also you can remove the owner variable from the provider config to narrow down the test scenario.
Okay! Thanks for the additional help. I've managed to reproduce this error with the following configuration:
sh$ ls -R
.:
modules/ project/
./modules:
main.tf
./project:
main.tf
./modules/main.tf:
resource "github_repository" "project_repository" {
name = "test-issue-1231"
description = "testing for https://github.com/integrations/terraform-provider-github/issues/1231"
visibility = "private"
template {
owner = "kfcampbell-terraform-provider"
repository = "test-terraform-config-block-repo"
}
}
./project/main.tf:
provider "github" {
token = "REDACTED"
owner = "kfcampbell-terraform-provider"
}
terraform {
required_providers {
github = {
source = "integrations/github"
}
}
}
# MY EXAMPLE
module "github_repository2" {
source = "../modules/"
}
Running terraform plan succeeds, but terraform apply fails with the following error:
2022-09-19T14:23:01.271-0700 [ERROR] vertex "module.github_repository2.github_repository.project_repository" error: POST https://api.github.com/repos/kfcampbell-terraform-provider/test-terraform-config-block-repo/generate: 404 Not Found []
╷
│ Error: POST https://api.github.com/repos/kfcampbell-terraform-provider/test-terraform-config-block-repo/generate: 404 Not Found []
│
│ with module.github_repository2.github_repository.project_repository,
│ on ../modules/main.tf line 3, in resource "github_repository" "project_repository":
│ 3: resource "github_repository" "project_repository" {
│
╵
Above in the HTTP request that gets 404'd, I see:
2022-09-19T14:23:01.019-0700 [DEBUG] provider.terraform-provider-github_v5.1.0: ---[ REQUEST ]---------------------------------------
2022-09-19T14:23:01.019-0700 [DEBUG] provider.terraform-provider-github_v5.1.0: POST /repos/kfcampbell-terraform-provider/test-terraform-config-block-repo/generate HTTP/1.1
2022-09-19T14:23:01.019-0700 [DEBUG] provider.terraform-provider-github_v5.1.0: Host: api.github.com
2022-09-19T14:23:01.019-0700 [DEBUG] provider.terraform-provider-github_v5.1.0: User-Agent: go-github/v47.0.0
2022-09-19T14:23:01.019-0700 [DEBUG] provider.terraform-provider-github_v5.1.0: Content-Length: 151
2022-09-19T14:23:01.019-0700 [DEBUG] provider.terraform-provider-github_v5.1.0: Accept: application/vnd.github.baptiste-preview+json
2022-09-19T14:23:01.019-0700 [DEBUG] provider.terraform-provider-github_v5.1.0: Content-Type: application/json
2022-09-19T14:23:01.019-0700 [DEBUG] provider.terraform-provider-github_v5.1.0: Accept-Encoding: gzip
2022-09-19T14:23:01.019-0700 [DEBUG] provider.terraform-provider-github_v5.1.0:
2022-09-19T14:23:01.019-0700 [DEBUG] provider.terraform-provider-github_v5.1.0: {
2022-09-19T14:23:01.019-0700 [DEBUG] provider.terraform-provider-github_v5.1.0: "name": "test-issue-1231",
2022-09-19T14:23:01.019-0700 [DEBUG] provider.terraform-provider-github_v5.1.0: "owner": "",
2022-09-19T14:23:01.019-0700 [DEBUG] provider.terraform-provider-github_v5.1.0: "description": "testing for https://github.com/integrations/terraform-provider-github/issues/1231",
2022-09-19T14:23:01.019-0700 [DEBUG] provider.terraform-provider-github_v5.1.0: "private": true
2022-09-19T14:23:01.019-0700 [DEBUG] provider.terraform-provider-github_v5.1.0: }
2022-09-19T14:23:01.019-0700 [DEBUG] provider.terraform-provider-github_v5.1.0:
So the owner isn't coming through as you've mentioned.
Interestingly, I see in the logs:
2022-09-19T14:45:11.682-0700 [DEBUG] provider.terraform-provider-github_v5.1.0: 2022/09/19 14:45:11 [INFO] Selecting owner kfcampbell-terraform-provider from GITHUB_OWNER environment variable
2022-09-19T14:45:11.682-0700 [DEBUG] provider.terraform-provider-github_v5.1.0: 2022/09/19 14:45:11 [INFO] Setting write_delay_ms to 1000
2022-09-19T14:45:11.682-0700 [DEBUG] provider.terraform-provider-github_v5.1.0: 2022/09/19 14:45:11 [DEBUG] Setting read_delay_ms to 0
2022-09-19T14:45:11.682-0700 [DEBUG] provider.terraform-provider-github_v5.1.0: 2022/09/19 14:45:11 [INFO] No token present; configuring anonymous owner.
"No token present; configuring anonymous owner" seems to be important. When debugging, I'm not getting into the logic that generates a repo here.
Interestingly, exporting GITHUB_TOKEN makes the same request succeed. It seems as though the issue is related to the token rather than the owner. Is exporting the token in the environment possible in your setup?
Is exporting the token in the environment possible in your setup?
Yes, exporting it works and I'm using it to get around this issue.
When debugging, I'm not getting into the logic that generates a repo here.
I'm not really sure that the problem is specifically in the line you mentioned (L346), I think that it has something to do with this line since it is trying to get the name of the owner from the meta interface.
Right, though in debugging I'm not getting there at all. I can add a time.Sleep(5 * time.Second) before that owner line and application is never delayed. terraform apply isn't even hitting breakpoints, logs, or sleeps even when I put them here.
I'm not executing apply with a planned state file. I know I'm using my debug provider (and not a custom version) because putting Sleeps/breakpoints/logs/etc. in the provider.go does work. I'm very perplexed.
Maybe I can help you more... I was reading the provider source code and found this
This is the provider definition, take a look and you'll see that token, owner, organization and base_url have the DefaultFunc as the schema.EnvDefaultFunc.
The schema.EnvDefaultFunc only tries to get the value from the environment and from nowhere else.
I thought that it would be worth taking a look at the terraform-provider-aws repository as well to see how they behave because I know that this issue does not occur with their provider configuration.
So maybe this EnvDefaultFunc has something to do with the bug, I can't really say for sure but I would take a look at it as well.
I apologize for the slow progress here; I've been busy and unable to prioritize this work. I'm still interested in it though!
I'm confused about this one, it seems that I'm facing the same issue
What confuses me is that it was working when I was testing my workflow with a small amount of data/resources... but when I moved to production it started failing
I'm setting it up with Flux/tf-controller, is there any way to make it work apart from setting as an env variable?
Do we know if this was introduced in some version? So I can whether downgrading the version would help
@kfcampbell it seems that you are right, I have moved the config block to the module and it worked
For anyone that comes across this, the issue seems to be if you don't specify the provider in a sub-module it tries to default a hashicorp provider that no longer exists. To resolve the issue make sure your sub-module has a terraform block like this:
terraform {
required_providers {
github = {
source = "integrations/github"
version = "~> 5.36"
}
}
}
I think that the issue has been solved therefore we can close it. Feel free to reopen it if necessary @jaredfholgate @dcfranca @kfcampbell
For anyone that comes across this, the issue seems to be if you don't specify the provider in a sub-module it tries to default a hashicorp provider that no longer exists. To resolve the issue make sure your sub-module has a
terraformblock like this:terraform { required_providers { github = { source = "integrations/github" version = "~> 5.36" } } }
Unfortunately this does not work anymore, current Terraform versions will not allow you to use the submodule if your top level module uses a for-each loop, count, or any other iterator.
For anyone that comes across this, the issue seems to be if you don't specify the provider in a sub-module it tries to default a hashicorp provider that no longer exists. To resolve the issue make sure your sub-module has a
terraformblock like this:terraform { required_providers { github = { source = "integrations/github" version = "~> 5.36" } } }Unfortunately this does not work anymore, current Terraform versions will not allow you to use the submodule if your top level module uses a for-each loop, count, or any other iterator.
Can you provide an example of the issue you see? Happy to take a look as not seen this problem myself.
required_provider blocks are allowed -- and often required -- in any module.
It's the provider block deleting the provider configuration itself -- that is, the block that instantiates the provider -- that should be only in the root module. That then means that the provider instance can be shared by all modules in the configuration, as long as they all agree about which provider source address they require.
