Create dag to load shortage detail information from ASHP

[eddie-cosma]

Explanation

Built dag that polls ASHP for detailed shortage information and loads into SageRx database.

Rationale

New feature implemented after discussion with @jrlegrand

Tests

1. What testing did you do? Manual testing; see below for evidence
2. Attach testing logs inside a summary block:

testing logs

Note: I have truncated result tables for the sake of readability.

select * from sagerx_lake.ashp_shortage_list;

name	detail_url	shortage_reasons	resupply_dates	alternatives_and_management	care_implications	safety_notices	created_date	updated_date
0.45% Sodium Chloride Injection Bags	Drug-Shortage-Detail.aspx?id=1100	["Baxter has sustained damage to their North Cove plant that supplies large volume fluids."]	["All marketed presentations are on allocation."]	["See ASHP Suggestions for Management and Conservation during Fluid Shortages at https://www.ashp.org/drug-shortages/shortage-resources/publications/fluid-shortages-suggestions-for-management-and-conservation"]	null	null	2024-10-07 00:00:00.000000	null
0.9% Sodium Chloride Irrigation	Drug-Shortage-Detail.aspx?id=811	["Baxter has sustained damage to their North Cove plant that supplies large volume fluids."]	["All marketed presentations are on allocation."]	["See ASHP Suggestions for Management and Conservation during Fluid Shortages at https://www.ashp.org/drug-shortages/shortage-resources/publications/fluid-shortages-suggestions-for-management-and-conservation"]	null	null	2022-02-25 00:00:00.000000	2024-10-04 00:00:00.000000
0.9% Sodium Chloride Large Volume Bags	Drug-Shortage-Detail.aspx?id=1094	["Baxter has sustained damage to their North Cove plant that supplies large volume fluids."]	["All marketed presentations are on allocation."]	["See ASHP Suggestions for Management and Conservation during Fluid Shortages at https://www.ashp.org/drug-shortages/shortage-resources/publications/fluid-shortages-suggestions-for-management-and-conservation"]	null	null	2024-10-04 00:00:00.000000	null

select * from sagerx_lake.ashp_shortage_list_ndcs;

detail_url	ndc	ndc_type
Drug-Shortage-Detail.aspx?id=1100	00264-7802-00	affected
Drug-Shortage-Detail.aspx?id=1100	00264-7802-10	affected
Drug-Shortage-Detail.aspx?id=1100	00338-0043-04	affected

select * from sagerx_dev.stg_ashp__current_drug_shortages;

id	name	url	shortage_reasons	resupply_dates	alternatives_and_management	care_implications	safety_notices	created_date	updated_date
1100	0.45% Sodium Chloride Injection Bags	https://www.ashp.org/drug-shortages/current-shortages/drug-shortage-detail.aspx?id=1100	["Baxter has sustained damage to their North Cove plant that supplies large volume fluids."]	["All marketed presentations are on allocation."]	["See ASHP Suggestions for Management and Conservation during Fluid Shortages at https://www.ashp.org/drug-shortages/shortage-resources/publications/fluid-shortages-suggestions-for-management-and-conservation"]	null	null	2024-10-07	null
811	0.9% Sodium Chloride Irrigation	https://www.ashp.org/drug-shortages/current-shortages/drug-shortage-detail.aspx?id=811	["Baxter has sustained damage to their North Cove plant that supplies large volume fluids."]	["All marketed presentations are on allocation."]	["See ASHP Suggestions for Management and Conservation during Fluid Shortages at https://www.ashp.org/drug-shortages/shortage-resources/publications/fluid-shortages-suggestions-for-management-and-conservation"]	null	null	2022-02-25	2024-10-04
1094	0.9% Sodium Chloride Large Volume Bags	https://www.ashp.org/drug-shortages/current-shortages/drug-shortage-detail.aspx?id=1094	["Baxter has sustained damage to their North Cove plant that supplies large volume fluids."]	["All marketed presentations are on allocation."]	["See ASHP Suggestions for Management and Conservation during Fluid Shortages at https://www.ashp.org/drug-shortages/shortage-resources/publications/fluid-shortages-suggestions-for-management-and-conservation"]	null	null	2024-10-04	null

select * from sagerx_dev.stg_ashp__current_drug_shortages_ndcs;

id	ndc_11	ndc_type
1100	00264780200	affected
1100	00264780210	affected
1100	00338004304	affected

PostgreSQL JSON syntax is supported for fields containing bracketed JSON lists in the tables above:

select shortage_reasons->0 from sagerx_dev.stg_ashp__current_drug_shortages;

?column?
"Baxter has sustained damage to their North Cove plant that supplies large volume fluids."
"Baxter has sustained damage to their North Cove plant that supplies large volume fluids."
"Baxter has sustained damage to their North Cove plant that supplies large volume fluids."