crossplane-runtime icon indicating copy to clipboard operation
crossplane-runtime copied to clipboard

Support async reconciliation

Open turkenf opened this issue 1 year ago • 2 comments

What problem are you facing?

For some time, upjet-based providers have been reconciling resources asynchronously. In order to do so, it "lies" to managed reconciler. Doing so causes lots of issues. So far, we have discussed the topic under those issues. It would be better if we could track them here, in a single place.

  • https://github.com/crossplane/crossplane/issues/5918 see this comment
  • https://github.com/crossplane/upjet/issues/439 see this comment
  • https://github.com/crossplane-contrib/provider-upjet-azure/issues/802
  • https://github.com/crossplane-contrib/provider-upjet-aws/issues/1715

How could Crossplane help solve your problem?

Support async reconciliation in managed reconciler.

turkenf avatar Nov 26 '24 13:11 turkenf

Provider helm has a similar issue: https://github.com/crossplane-contrib/provider-helm/issues/63

We could live without it to date, but, if there were support for such a mode in crossplane runtime, we could have provided a better UX for certain charts (i.e. the ones with pre-install hooks) and with wait: true.

turkenh avatar Nov 27 '24 08:11 turkenh

Is there a design document explaining why the C/U/D operations are running in a new goroutine? I understand they need a longer timeout, so the context.Context would either need a longer timeout or run in a new one based on .Background().

But I don't know the reason for a new goroutine. I would suspect you could achieve the same effect with a high MaxConcurrentReconciles.

This would be more in line with what controller-runtime and crossplane-runtime expects, but I'm sure there's a good reason I don't know of :)

chlunde avatar Mar 04 '25 21:03 chlunde