redshift_plugin
redshift_plugin copied to clipboard
airflow-plugins
This change fixes Issue #18 by ensuring column names are lowercased before column differential check.
operators/s3_to_redshift_operator.py (Lines 189-191) ------------------------------------ ``` pg_query = \ """ SELECT column_name, udt_name FROM information_schema.columns WHERE table_schema = '{0}' AND table_name = '{1}'; """.format(self.redshift_schema, self.table) pg_schema = dict(pg_hook.get_records(pg_query)) incoming_keys = [column['name']...
This is used for checking when we resize a cluster or newly create it from snapshots, and hence wait until the cluster is available for us to run queries on...
Hi. I am fairly new to airflow. I have cloned the repository in plugins folder of airflow home. How to I access the plugin? I have tried importing the following...
The function is not robust. To cover all cases you need to comply with: https://docs.aws.amazon.com/datapipeline/latest/DeveloperGuide/dp-template-redshift.html For example, what if you get int(8) from MySQL?
Accessing keys and values of the json schema is incorrect. Change the code to: ``` def create_if_not_exists(self, schema, pg_hook, temp=False): output = '' for item in schema: k = "{quote}{key}{quote}".format(quote='"',...