Joel comments

Results 372 comments of


                                            Joel

[KED-2744] Memory Leakage - Unexpected Caching behaviour with CacheDataset

[See full discussion on Discord](https://discord.com/channels/778216384475693066/778998585454755870/860336257355218974)

[KED-2744] Memory Leakage - Unexpected Caching behaviour with CacheDataset

Created ticket on backlog

[KED-2744] Memory Leakage - Unexpected Caching behaviour with CacheDataset

@acnazarejr - to confirm is this also using the `CachedDataSet`? Persisted writes to Parquet should discard the memory and be garbage collected by Python.

[KED-2744] Memory Leakage - Unexpected Caching behaviour with CacheDataset

Hi @acnazarejr I don't think it should be behaving this way - how do you know the memory is not being released?

[KED-2744] Memory Leakage - Unexpected Caching behaviour with CacheDataset

Could I also ask - why not make MemoryDatasets cache by default?

[KED-2744] Memory Leakage - Unexpected Caching behaviour with CacheDataset

I guess my question is - if we can, why not make regular datasets cache by default?

[KED-2744] Memory Leakage - Unexpected Caching behaviour with CacheDataset

Got it! They say cache invalidation is the 2nd hardest problem for a reason!

Visualize hooks in Kedro-Viz

An MVP showing the names of hooks register would be a great first start Hooks fall into the following categories: https://docs.kedro.org/en/stable/kedro.framework.hooks.specs.html At a high level hooks basically fall into 3...

`kedro run --params key=1,2,3` Kedro CLI does not take list as argument

Some overlap with this recent discussion https://github.com/kedro-org/kedro/pull/3904#issuecomment-2142457119

Add documentation about the combined use of `$globals` and `$runtime_params`

[Another user touching this question](https://linen-slack.kedro.org/t/16757928/hello-team-i-d-like-to-know-if-i-can-change-the-data-directo#72f17b03-ac3f-4d30-9e0c-e929a7eec687)