Antony Milne

Should kedro-datasets be a dependency of kedro?

1

My latest proposal in the ongoing discussion of kedro-datasets. See https://github.com/kedro-org/kedro/issues/1758 for context. ## Proposal 1. We continue with `kedro-datasets` becoming a separate namespace package as we are currently in...

Re-design io.core and io.data_catalog

9

Spun out of https://github.com/kedro-org/kedro/issues/1691#issuecomment-1176679924... Let's collect ideas here on what current problems are with `io`. To me it feels like we've neglected it and it's ripe for a re-design. >...

kedro-datasets: dependencies and package structure. Are we doing the right thing?

7

## Context Let’s pause and take stock of where we are in https://github.com/kedro-org/kedro/issues/1457. This is where I think things stand: * @idanov planned for us to move kedro datasets into...

Copying relevant parts from https://github.com/kedro-org/kedro/issues/1758... > [@AntonyMilneQB] Let’s investigate how other libraries are handling similar situations. e.g. I believe the idea for `kedro-datasets` might have been inspired by how `django`...

# How do credentials currently work in kedro? The basic pattern is as follows: ```yaml # conf/base/catalog.yml dataset_name: ... credentials: credentials_key # conf/local/credentials.yml credentials_key: kwarg1: value1 kwarg2: value2 ... ```...

Now we have beautiful richly formatted logs, we should update our docs to show it off! The key places to update are https://kedro.readthedocs.io/en/stable/get_started/example_project.html and https://kedro.readthedocs.io/en/stable/tutorial/create_pipelines.html. The best way to do...

## Description A fix for the issue raised by @pstanisl in https://github.com/kedro-org/kedro/issues/1728. Hopefully this will be fixed on the rich side in https://github.com/Textualize/rich/issues/2455, but if that doesn't happen soon then...

Alias IPython extension

2

I'm fed up of (mis)typing `%load_ext kedro.extras.extensions.ipython`. Let's make everyone's lives a bit easier and alias it. Previously we had said that `%load_ext kedro.ipython` would be an improvement, but I'm...

## Description Resolves #1590. > **Note**. I still need to double check my tests cover everything they should, but this is ready for review. There were two problems with how...

Use fsspec's infer_storage_options

3

## Description Closes: #1632. Thanks to https://github.com/fsspec/filesystem_spec/pull/988 we no longer need to write our own version of fsspec's `infer_storage_options`. However, we do need to wait until they do a release...