Fail to synchronize

[SangkyunLee]

For bug reports, please follow the template below. For enhancement proposals, feel free to use whatever template makes sense (major new features should be discussed in the Zarr specifications repository https://github.com/zarr-developers/zarr-specs).

Minimal, reproducible code sample, a copy-pastable example if possible

# Your code here
i
import zarr
      
      
        import numpy as np
      
      
        from dask import delayed
      
      
        

      
      
        synchronizer=zarr.ThreadSynchronizer()
      
      
        

      
      
        

      
      
        down_scale=0
      
      
        outstack_size = (1,1,1,6000,7000)
      
      
        chunksize =(1,1,1,1000,1000)
      
      
        outstack = zarr.open_array(str(down_scale),shape=outstack_size,
      
      
        chunks = chunksize,
      
      
        fill_value = 0,
      
      
        synchronizer=synchronizer,
      
      
        mode='w',
      
      
        dtype='i2')
      
      
        

      
      
        

      
      
        nimg = 6*7
      
      
        

      
      
        

      
      
        y,x=np.meshgrid(list(range(6)), list(range(7)))
      
      
        y = y.flatten()
      
      
        x = x.flatten()
      
      
        

      
      
        

      
      
        

      
      
        def cal_area(y,x, inx, chunkshape, outshape):
      
      
            yi = y[inx]
      
      
            xi = x[inx]
      
      
            y0 = max(chunkshape[0]*yi-10,0)
      
      
            y1 = min(chunkshape[0]*(yi+1)+10,outshape[0])
      
      
        

      
      
            x0 = max(chunkshape[1]*xi-10,0)
      
      
            x1 = min(chunkshape[1]*(xi+1)+10,outshape[1])
      
      
            
      
      
            return [y0,y1,x0,x1]
      
      
        

      
      
        

      
      
        chunkshape = chunksize[-2:]
      
      
        outshape = outstack_size[-2:]
      
      
        # print(cal_area(y, x, 10,chunkshape, outshape))
      
      
        

      
      
        def incr(outstack, iimg, count=0):
      
      
            slice_coord = cal_area(y, x, iimg,chunkshape, outshape)
      
      
            y0, y1, x0, x1 = slice_coord
      
      
            outstack[0,0,0, y0:y1, x0:x1] = outstack[0,0,0, y0:y1, x0:x1]+1
      
      
            return count+1
      
      
        

      
      
        lazyinc = delayed(incr)
      
      
        result =[]
      
      
        for iimg in range(nimg):
      
      
            result.append(lazyinc(outstack,iimg))
      
      
        

      
      
        import dask
      
      
        import dask.array as da
      
      
        out = dask.compute(*result)
      
      
        

      
      
        daimg = da.from_zarr(outstack)
      
      
        

      
      
        import matplotlib.pyplot as plt
      
      
        plt.matshow(daimg[0,0,0])
      
      
        <span class="pl-s1">plt</span>.<span class="pl-en">show</span>()
      
      
        <span class="pl-s1">plt</span>.<span class="pl-en">show</span>()

Problem description

When using dask delayed module to simultaneously increase the value of a patch in the zarr array, where a patch is overlaid in adjacent patches, synchronization does not properly work. After all workers complete their task, the final output should show a complete grid pattern. but the output results vary across executions by showing different incomplete grid patterns each time.

Version and installation information

Please provide the following:

• Value of zarr.__version__ 2.8.3
• Value of numcodecs.__version__ 0.7.2.
• Version of Python interpreter 3.9.1
• Operating system (Linux/Windows/Mac) Linux
• How Zarr was installed (e.g., "using pip into virtual environment", or "using conda") conda

Also, if you think it might be relevant, please provide the output from pip freeze or conda env export depending on which was used to install Zarr.

[chris-allan]

See also #857.