temporal icon indicating copy to clipboard operation
temporal copied to clipboard

Support sql driver connector

Open ysdanielkim opened this issue 2 years ago • 7 comments

What changed?

SQL config now supports passing driver.Connector instance.

Why?

It's currently not possible to embed a Temporal server to a host application where the database connection is managed by the host application and is provided as a database connector. This allows the host application to pass a driver connector instance rather than the the individual sql config strings.

How did you test it?

I have embedded this code into a custom application and verified that the driver connector instance is applied properly.

Potential risks

This is a non-breaking change with very minimal risks.

Is hotfix candidate?

No.

ysdanielkim avatar Jan 05 '24 00:01 ysdanielkim

CLA assistant check
All committers have signed the CLA.

CLAassistant avatar Jan 05 '24 00:01 CLAassistant

This PR was marked as stale. Please update or close it.

github-actions[bot] avatar May 16 '24 00:05 github-actions[bot]

I left a comment on the correctness of your postgres changes. I don't like adding this to config.SQL as that's all coming from YAML and this feels a bit messy, but I don't have a better answer right now (though I'll think about it).

I agree. Thank you for thinking about it. I'd be happy to make any necessary changes.

ysdanielkim avatar May 17 '24 03:05 ysdanielkim

So with https://github.com/temporalio/temporal/pull/5926 we're about to automatically reconnect on connection errors. Rather than provide the driver connection itself, we need you to provide a function that will connect on demand.

We're implementing this to deal with, for example, Aurora RDS failovers

tdeebswihart avatar May 17 '24 21:05 tdeebswihart

I left a comment on the correctness of your postgres changes. I don't like adding this to config.SQL as that's all coming from YAML and this feels a bit messy, but I don't have a better answer right now (though I'll think about it).

I agree. Thank you for thinking about it. I'd be happy to make any necessary changes.

If I figure something out I'll let you know, otherwise I can refactor it later once I've figured it out :)

tdeebswihart avatar May 17 '24 21:05 tdeebswihart

So with #5926 we're about to automatically reconnect on connection errors. Rather than provide the driver connection itself, we need you to provide a function that will connect on demand.

We're implementing this to deal with, for example, Aurora RDS failovers

Okay. So something like this?

type SQL struct {
  // DatabaseConnection returns a new database connection. Using this will bypass the string based data source.
  DatabaseConnection func() (*sqlx.DB, error)
  ...
}

ysdanielkim avatar May 17 '24 22:05 ysdanielkim

So with #5926 we're about to automatically reconnect on connection errors. Rather than provide the driver connection itself, we need you to provide a function that will connect on demand. We're implementing this to deal with, for example, Aurora RDS failovers

Okay. So something like this?

type SQL struct {
  // DatabaseConnection returns a new database connection. Using this will bypass the string based data source.
  DatabaseConnection func() (*sqlx.DB, error)
  ...
}

That would be perfect, though I'd name it something like Connect or something. I've merged the PR I referenced; you may wish to rebase/merge it in. I'm happy to help if you need it

tdeebswihart avatar May 18 '24 00:05 tdeebswihart

@ysdanielkim can you add ,json:"-" to the struct tag for this new field? We need to make sure it's ignored by the JSON serializer. We do something messy when using these in our tests, it looks like

tdeebswihart avatar May 24 '24 16:05 tdeebswihart

@tdeebswihart thank you for guiding me through! Looks like I don't have write access. Please merge this for me. Screenshot 2024-05-24 at 4 27 33 PM

ysdanielkim avatar May 24 '24 21:05 ysdanielkim

Done! Thanks for your contribution :)

tdeebswihart avatar May 24 '24 21:05 tdeebswihart