Should it be possible to register several reconcilers for the same native k8s resources?

[metacosm]

Since we allow registering reconcilers for native resources, would it make sense to allow a user to register several different controllers for the same resource (e.g. ConfigMap) provided their configuration is different?

[csviri]

Good question, I cannot think of a use case, since it's anyways our controller the logic of multiple might be merged to one reconciler. maybe for different versions?

[scrocquesel]

Would it be possible to add a label filter by configuration ? For those using config map as CRD it can help.

[csviri]

Would it be possible to add a label filter by configuration ? For those using config map as CRD it can help.

That should be already possible, see: https://github.com/java-operator-sdk/java-operator-sdk/blob/5ad74190bd891cb4b184de35af748aff7d195175/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/api/reconciler/ControllerConfiguration.java#L51-L51

[10000-ki]

@csviri hello~ now I'm using josdk to develop an operator

sometimes we need to multiple controller about same CR

https://github.com/Opster/opensearch-k8s-operator/blob/c12351d36fd8feb9fc8129b45eb60249adf947f6/opensearch-operator/controllers/opensearchController.go#L295

like an example of the above

For example,

• creating an opensearch cluster
• upgrading an opensearch cluster
• etc...

They have different interests But putting all the logic in a single reconciler and managing it becomes very complicated

Please consider it

[metacosm]

@10000-ki how do you determine which controller needs to be called, though? If all reconcilers all process the resources to decide whether or not they need to handle the resource fully, then that means that your cluster has to work a lot more just to do the same thing. You could have just one controller that then dispatches to separate pieces of the code to deal with the different cases.

[csviri]

@10000-ki there controllers would have different Informers that would trigger them? Or what would be the benefit?

I know it's not nice but now just making just spanning an additional thread on the beginning of reconciliation would have the same effect result in a very similar code ( I know would not be that nice ,)

So basically I'm more interested in your use case, are there some configuration changes between the two controllers?

I'm not against this, if it would not bring too much complexity to the code, and there are reasonable use cases that we see.

[csviri]

Discussed on Discord: https://discord.com/channels/723455000604573736/723455000604573739/1172281835879936030

Going to close this for now if no objections

[metacosm]

I would rather see the discussion here so that when/if people have the same question, they can see the answer here instead of having to go to Discord… 😢

[csviri]

I would rather see the discussion here so that when/if people have the same question, they can see the answer here instead of having to go to Discord… 😢

That is fair, will do a summary:

So the debate was if there are more dependent resources, (>~10) should be this split up because of some code quality considerations, (not performance reasons), and basically the conclusion was that that is not a good idea. There might be multiple status updates concurrently, will result in more informers registered. Also IMO just conceptually separate controllers are not the place to do that.

[10000-ki]

@csviri

I know it's not nice but now just making just spanning an additional thread on the beginning of reconciliation would have the same effect result in a very similar code ( I know would not be that nice ,) So basically I'm more interested in your use case, are there some configuration changes between the two controllers? I'm not against this, if it would not bring too much complexity to the code, and there are reasonable use cases that we see.

I'm sorry for checking it late

I'm develping OpenSearch Cluster Operator in my company

like this..

@ControllerConfiguration(
    labelSelector = "app.kubernetes.io/managed-by=$OPERATOR_LABEL_V1",
    dependents = [
        Dependent(
            type = OpenSearchConfigMap::class,
            name = "OpenSearchConfigMap",
            useEventSourceWithName = CONFIGMAP_EVENT_SOURCE,
        ),
        Dependent(
            type = OpenSearchService::class,
            name = "OpenSearchService",
            useEventSourceWithName = SERVICE_EVENT_SOURCE,
        ),
        Dependent(
            type = OpenSearchDiscoveryService::class,
            name = "OpenSearchDiscoveryService",
            useEventSourceWithName = SERVICE_EVENT_SOURCE,
        ),
        Dependent(
            type = OpenSearchNodeStatefulSet::class,
            name = "OpenSearchNodeStatefulSet",
            useEventSourceWithName = STS_EVENT_SOURCE,
            dependsOn = arrayOf("OpenSearchConfigMap"),
        ),
        Dependent(
            type = OpenSearchNodeDeployment::class,
            name = "OpenSearchNodeDeployment",
            useEventSourceWithName = DEPLOYMENT_EVENT_SOURCE,
            dependsOn = arrayOf("OpenSearchConfigMap"),
        ),
        Dependent(
            type = OpenSearchDashboardService::class,
            name = "OpenSearchDashboardService",
            useEventSourceWithName = SERVICE_EVENT_SOURCE,
            dependsOn = arrayOf("OpenSearchNodeDeployment"),
        ),
        Dependent(
            type = OpenSearchDashboardDeployment::class,
            name = "OpenSearchDashboardDeployment",
            useEventSourceWithName = DEPLOYMENT_EVENT_SOURCE,
            dependsOn = arrayOf("OpenSearchNodeDeployment"),
        ),
    ],
)
class OpenSearchReconciler(
    private val client: KubernetesApiService,
    private val openSearchClientService: OpenSearchClientService,
) :
    Reconciler<OpenSearch>,
    ErrorStatusHandler<OpenSearch>,
    EventSourceInitializer<OpenSearch> {

    private val logger = logger("[Operator][OpenSearchReconciler]")
    private val openSearchBootstrapEventHandler = OpenSearchBootstrapHandler(client, openSearchClientService)
    private val openSearchUpgradeHandler = OpenSearchUpgradeHandler(client, openSearchClientService)

    override fun reconcile(resource: OpenSearch, context: Context<OpenSearch>): UpdateControl<OpenSearch> {
        logger.info { "Reconcile opensearch spec : ${resource.spec}" }

        // TODO: remove null check after set default status value
        if (resource.shouldBootstrap() && !resource.shouldUpgrade()) {
            return openSearchBootstrapEventHandler.handle(resource)
        }

        if (resource.shouldUpgrade()) {
            return openSearchUpgradeHandler.handle(resource)
        }

        return UpdateControl.noUpdate()
    }

Unlike the OpenSearchCluster Create situation, the Version Upgrade situation is actually similar to a simple one-time job.

However, because the version needs to be modified to OpenSearchCR, the event enters 'OpenSearchReconciler' and

The desire or match method in DependencyResource is repeatedly called but sometimes this is unnecessarily

And For OpenSearchDashboard, it can also be created and managed separately from OpenSearch. However, it is cumbersome to create and manage 'OpenSearchDashboardCR' to do so.

I've been thinking about this.

• OpenSearchCR
  • OpenSearchReconciler
  • OpenSearchUpgradeReconciler
  • OpenSearchDashboardReconciler

[csviri]

@10000-ki It's a good question. I think until you don't have some real life performance issues this is fine. Dependent Resources already do a bunch of optimizations, that are not present for example in standard go operators (regarding eventing and triggering see: https://javaoperatorsdk.io/docs/dependent-resources#caching-and-event-handling-in-kubernetesdependentresource ). It is usually happens that there is unnecessary checking of resources, to just calculating desired and matching is not really that much of an overhead. So would not split it up until there is a real issue.

[10000-ki]

okay

I'll give you an additional opinion later if you want

thank you

[csviri]

sure, pls let us know @10000-ki , thx!!