Bug while adding new ACLs

[HaruChebrolu]

Description

When we are trying to add new ACLs, its deleting old ones and replacing with new ones.

Versions

We are using mongey kafka provider.

https://registry.terraform.io/providers/Mongey/kafka/latest github.com/IBM/sarama v1.43.0

Configuration

provider "kafka" { bootstrap_servers = ["localhost:9092"] ca_cert = file("../secrets/ca.crt") client_cert = file("../secrets/terraform-cert.pem") client_key = file("../secrets/terraform.pem") }

resource "kafka_acl" "test" { resource_name = "syslog" resource_type = "Topic" acl_principal = "User:Alice" acl_host = "*" acl_operation = "Write" acl_permission_type = "Deny" }

im using 0.6.0 version, using Amazon Managed Kafka. running terraform in an EC2 Terraform v1.5.7 on linux_amd64 output: build 11-Apr-2024 09:45:06 �[1m # kafka_acl.acl[61]�[0m will be created build 11-Apr-2024 09:45:06 �[0m �[32m+�[0m�[0m resource "kafka_acl" "acl" { build 11-Apr-2024 09:45:06 �[32m+�[0m�[0m acl_host = "" build 11-Apr-2024 09:45:06 �[32m+�[0m�[0m acl_operation = "Read" build 11-Apr-2024 09:45:06 �[32m+�[0m�[0m acl_permission_type = "Allow" build 11-Apr-2024 09:45:06 �[32m+�[0m�[0m acl_principal = "User:CN=test.user" build 11-Apr-2024 09:45:06 �[32m+�[0m�[0m id = (known after apply) build 11-Apr-2024 09:45:06 �[32m+�[0m�[0m resource_name = "test_dev" build 11-Apr-2024 09:45:06 �[32m+�[0m�[0m resource_pattern_type_filter = "Literal" build 11-Apr-2024 09:45:06 �[32m+�[0m�[0m resource_type = "Group" build 11-Apr-2024 09:45:06 } build 11-Apr-2024 09:45:06 build 11-Apr-2024 09:45:06 �[1m # kafka_acl.acl[62]�[0m will be created build 11-Apr-2024 09:45:06 �[0m �[32m+�[0m�[0m resource "kafka_acl" "acl" { build 11-Apr-2024 09:45:06 �[32m+�[0m�[0m acl_host = "" build 11-Apr-2024 09:45:06 �[32m+�[0m�[0m acl_operation = "Read" build 11-Apr-2024 09:45:06 �[32m+�[0m�[0m acl_permission_type = "Allow" build 11-Apr-2024 09:45:06 �[32m+�[0m�[0m acl_principal = "User:CN=test.user" build 11-Apr-2024 09:45:06 �[32m+�[0m�[0m id = (known after apply) build 11-Apr-2024 09:45:06 �[32m+�[0m�[0m resource_name = "kafka_topic" build 11-Apr-2024 09:45:06 �[32m+�[0m�[0m resource_pattern_type_filter = "Prefixed" build 11-Apr-2024 09:45:06 �[32m+�[0m�[0m resource_type = "Topic" build 11-Apr-2024 09:45:06 } build 11-Apr-2024 09:45:06 build 11-Apr-2024 09:45:06 �[1mPlan:�[0m 58 to add, 0 to change, 55 to destroy.

[puellanivis]

This doesn’t seem related at all to sarama here, which is merely the Golang interface with Kafka. Rather you’re looking for help with terraform code.

I have seen this sort of issue before, though, and I think it’s a fairly common sort of issue with terraform sometimes: https://github.com/hashicorp/terraform/issues/19974 Common situations are when you change the ordering of a list, terraform cannot just mutate into that state, so it deletes and replaces rather than in-place updates.

[HaruChebrolu]

@puellanivis When we are trying to create ACL from here, its not where comparing between comparing new ACLs with existing ACLs. Its trying to create all ACLs whatever we send as input. https://github.com/IBM/sarama/blob/main/broker.go#L718

Don't we need to have a check here

[puellanivis]

I’m still confused. Are you creating those ACLs by calling sarama/Broker.CreateAcls or are you creating these ACLs via terraform? If you’re not creating them via terraform, why are you including that in the issue report?

[HaruChebrolu]

We are calling it via terraform provider which internally calls sarama package to create ACL. Please check here https://github.com/Mongey/terraform-provider-kafka/blob/main/kafka/kafka_acls.go#L216

[puellanivis]

Yes, but sarama’s sarama/Broker.CreateAcls isn’t deleting your ACLs. Mongey is calling into sarama’s DeleteAcls the same as it is calling into sarama’s CreateAcls. The logic on deciding what to delete and recreate, or what to modify is not in sarama’s code.

I strongly think you need to be reporting this to either Mongey or terraform itself. There’s nothing anyone here can do about what those codebases do with sarama’s code.

[github-actions[bot]]

Thank you for taking the time to raise this issue. However, it has not had any activity on it in the past 90 days and will be closed in 30 days if no updates occur. Please check if the main branch has already resolved the issue since it was raised. If you believe the issue is still valid and you would like input from the maintainers then please comment to ask for it to be reviewed.