Andrew Kesterson

Has there been any further discussion of this feature? I'm currently trying to map an organization with about 2 dozen accounts and the processing time is incredibly painful. I agree...

Sorry, just now seeing this. yeah I'll take a look and see what I can do re: getting it rebased and tested.

Relates to #6757

Can you assign this to me please

Is there a documented reproduction case for this? > Analyze the current implementation of materializers and identify how and where temporary directories are being used. I've examined all of the...

Looks like these are the integrations that create temporary objects in their materializer ``` zenml/src/zenml/integrations/bentoml zenml/src/zenml/integrations/huggingface zenml/src/zenml/integrations/lightgbm zenml/src/zenml/integrations/llama_index zenml/src/zenml/integrations/pillow zenml/src/zenml/integrations/polars zenml/src/zenml/integrations/pycaret zenml/src/zenml/integrations/tensorflow zenml/src/zenml/integrations/whylogs zenml/src/zenml/integrations/xgboost ``` I'll take a look

Most of those materializers are already using the `tempfile` module, they just aren't cleaning up after themselves. (Some are doing it, some are inconsistent within the same module, others have...

Sorry for the long delay here. PR is up. There are some comments around the best way to test. https://github.com/zenml-io/zenml/pull/2560

I discovered another materializer that is a special case. The tensorflow dataset materializer intentionally does not clean up after itself in the load() stage, because the dataset is lazily loaded...

Docs are updated. Fixed a couple of materializers I had missed.