Mayur Singal

Results 31 comments of Mayur Singal

@codingwithabhi can we add the docs for dasgter as well in this pr?

Also we might need to modify the [test_connection](https://github.com/open-metadata/OpenMetadata/blob/5c0016d827ed71c5fa387fdcb226a6b03af3c0ab/ingestion/src/metadata/utils/connections.py#L863) for dagster as well as we will not be using db connection anymore

@harshach I believe we are directly calling the backend api instead of querying to ES first [here](https://github.com/open-metadata/OpenMetadata/blob/0b0d1c51d92f004ca67e81db0d0cb01a81a4d728/ingestion/src/metadata/ingestion/source/pipeline/fivetran.py#L168)

``` Cannot extract source table information from query [select ind_head.indexrelid index_id, k col_idx, k

- @OnkarVO7 https://sandbox.open-metadata.org/table/MySQL_Test.sql12611281.sql12611281.vRisk_Denorm/lineage query (mysql) dialect: ``` CREATE ALGORITHM=UNDEFINED DEFINER=`sql12611281`@`%` SQL SECURITY DEFINER VIEW `sql12611281`.`vRisk_Denorm` AS select `a`.`AsOfDate` AS `AsOfDate`,`a`.`AssetID` AS `AssetID`,`a`.`AssetRate` AS `AssetRate`,`a`.`AssetRateIndex` AS `AssetRateIndex`,`a`.`AssetRateIndexBehavior` AS `AssetRateIndexBehavior`,`a`.`CostPriceSettled` AS `CostPriceSettled`,`a`.`CostPriceTraded`...

related to @SabineGl's comment the query looks like this `create view new_view (col1,col2) select col1,col2 from table` to replicate this we can ingest the oracle source containing views context: https://openmetadata.slack.com/archives/C02B6955S4S/p1681204033955849

dialect: snowflake support thread: https://openmetadata.slack.com/archives/C02B6955S4S/p1681277509847309 ``` CREATE OR REPLACE TRANSIENT TABLE final_table copy grants as Select a.*, coalesce(table_1.column_d, table_2.column_d, table_3.column_d, table_4.column_d) AS column_a, coalesce(table_1.type, table_2.type, table_3.type, table_4.type) AS column_b, table_5.column_b,...

dialect: sqlserver ``` CREATE VIEW [dbo].[vDMPrep] AS SELECT pc.EnglishProductCategoryName ,Coalesce(p.ModelName, p.EnglishProductName) AS Model ,c.CustomerKey ,s.SalesTerritoryGroup AS Region ,CASE WHEN Month(GetDate()) < Month(c.BirthDate) THEN DateDiff(yy,c.BirthDate,GetDate()) - 1 WHEN Month(GetDate()) = Month(c.BirthDate)...

dialect: snowflake ``` UPDATE RDBAPP_CATALOG_VISIBILITIES_HISTORY_STG HST SET _END_DATE = NVL(LEAST(HST._END_DATE, SRC._UPDATE_DATE), SRC._UPDATE_DATE), _IS_CURRENT = IFF(_ACTION = 'DEL', TRUE, FALSE), _IS_HARD_DELETED = IFF(_ACTION = 'UPD', FALSE, TRUE), _ROW_UPDATE_DATE = SYSDATE() FROM...

dialect: postgres ``` CREATE VIEW views_api AS SELECT max(t.id) AS id, t.day, t.ab_id, t.ab_group_id, t.ab_group_name, t.ab_is_test, t.hw_type, false AS feature_hit, t.metric, t.type, sum(t.count) AS value, a.name, a.description, a.t_start, a.t_end, a.layer_name,...