Breaking changes coming from BC Data Catalogue update

[smnorris]

https://github.com/bcgov/bcdata/issues/342

Swapping in the test url (https://toyger.data.gov.bc.ca/) results in several failures:

$ py.test tests/test_bcdc.py
========================================= test session starts =========================================
platform darwin -- Python 3.11.9, pytest-8.0.1, pluggy-1.4.0
rootdir: /Users/snorris/Projects/repo/bcdata
plugins: requests-mock-1.11.0
collected 8 items

tests/test_bcdc.py ...FFFFF
....

================================================== short test summary info ==================================================
FAILED tests/test_bcdc.py::test_get_table_definition - TypeError: the JSON object must be str, bytes or bytearray, not list
FAILED tests/test_bcdc.py::test_get_table_definition_format_multi - TypeError: list indices must be integers or slices, not str
FAILED tests/test_bcdc.py::test_get_table_definition_format_multi_nopreview - TypeError: the JSON object must be str, bytes or bytearray, not list
FAILED tests/test_bcdc.py::test_get_table_definition_format_multi_nolayer - TypeError: the JSON object must be str, bytes or bytearray, not list
FAILED tests/test_bcdc.py::test_get_table_definition_format_oracle_sde - TypeError: list indices must be integers or slices, not str

[smnorris]

Testing current fixes/PR fails because no results are found when searching for these packages. Maybe not all packages/resources are included in the testing instance?

https://toyger.data.gov.bc.ca/api/3/action/package_search?q=WHSE_LAND_USE_PLANNING.RMP_LANDSCAPE_RSRV_DESIGN_SP

https://toyger.data.gov.bc.ca/api/3/action/package_search?q=WHSE_FOREST_VEGETATION.OGSR_PRIORITY_DEF_AREA_CUR_SP

[smnorris]

Tests now mostly pass, the only error is when scanning for preview_info key where resource["format"] == "multiple" (not present for WHSE_HUMAN_CULTURAL_ECONOMIC.HIST_HISTORIC_ENVIRONMENTS_SP.

The current scanning of bcdc api results/resources for the table schema and description is scattered and could be improved. Would searching by object name and bcdc_type=geographic work instead of looking at each resource format?

https://bcgov.github.io/data-publication/pages/dps_bcdc_api_w_common_calls.html#search-by-object_name https://bcgov.github.io/data-publication/pages/dps_bcdc_api_w_common_calls.html#search-by-bcdc_type

edit - seems to work just fine, PR is updated

[smnorris]

Once bcdc api upgrade is deployed (I don't see any guidance on date):

• [ ] update api url in PR
• [ ] merge and cut a new release
• [ ] upgrade bcdata in downstream projects

[smnorris]

https://www2.gov.bc.ca/gov/content/data/finding-and-sharing/bc-data-catalogue/bc-catalogue-service-disruptions

[smnorris]

branch v0.13 is ready to go to support this update (although tests currently fail, testing endpoint is not available) When the API has been migrated:

• update endpoint url, test locally
• if tests pass, commit and push
• if CI tests pass, merge the PR
• tag/release

[smnorris]

API has been migrated, bcdata < v0.13 are non-functional