TobikoData
WIP: Add Materialize Engine Adapter
As a follow up to the https://github.com/tobymao/sqlglot/pull/3577 PR, starting the initial work to add the SQLMesh Materialize engine adapter.
This is still WIP, just starting with the initial work.
Excited about the progress! When you are ready, make sure to add materialize here: https://github.com/TobikoData/sqlmesh/blob/0607f65800d231d70fdc2c50c0550c77b39eb873/tests/core/engine_adapter/test_integration.py#L538
These tests ensure that the engine implements the engine adapter interface correctly which tends to pick up most of the issues. It also includes a sushi test now which involves actually running a SQLMesh project. Typically passing these tests is where most of your time will be spent when adding a new engine.
hey @bobbyiliev how's progress on this? do you need any assistance?
Hey @tobymao, sorry for the delay here. I will be picking this up again starting tomorrow.
The two main things that are currently left are the fact that Materialize does not support ALTER TABLE and CREATE TABLE .. AS SELECT which I will be looking into tomorrow.
Hey @tobymao, quick update here, I had a chat with the team regarding the limitation where Materialize does not support ALTER TABLE .. ADD COLUMN .. and we will actually be working on this soon and hope to have this implemented by the beginning of the next quarter. I was thinking that in this case, it might be better to wait for that to get implemented, as SQMesh migrations seem to depend on it quite a bit, and as far as I can tell all other drivers that you currently have, all support it. Let me know what you think!
@bobbyiliev that's fine. thanks for digging into it. looking forward to adding materialize when it's ready :)
i'm going to close this now but feel free to reopen when you have @bobbyiliev