xarray icon indicating copy to clipboard operation
xarray copied to clipboard

Hook for better control over copy behavior with duck-arrays?

Open SimonHeybrock opened this issue 3 years ago • 0 comments

Context

By using copy(deep=False) custom operations may avoid copying large amounts of data but can modify, e.g., coord dicts:

tmp = da.copy(deep=False)
del tmp.coords['abc']
# Use tmp

When da wraps a duck-array with substructure the current implementation is insufficient:

tmp = da.copy(deep=False)
# Imagine a duck-array similar to numpy.ma but with a dict of masks
del tmp.data.masks['abc']  # Bad: breaks `da`
# Use tmp

Describe the solution you'd like

Currently there does not appear a solution to this, unless we know details about the duck array. Therefore, I wonder if we need an additional "hook" that duck-arrays may provide, which could be called by Xarray to make a non-deep copy?

class MyDuckArray:
  def _copy_shallow_(self):  # TODO: better name
    """Copy everything except buffers"""
# in xarray.Variable
def copy(self, deep=True):
  if deep:
    data = copy.deepcopy(data)
  elif hasattr(data, '_copy_shallow_'):
   data = data._copy_shallow_()

Additional context

This is the current implementation for Variable. There is no operation for deep=False: https://github.com/pydata/xarray/blob/716973e41060184beebd64935afe196a805ef481/xarray/core/variable.py#L956-L957

SimonHeybrock avatar Sep 20 '22 08:09 SimonHeybrock