Alexey Gerasev

``` ---- tests::write_read stdout ---- thread '' panicked at blocking/src/tests.rs:134:9: assertion `left == right` failed left: 2300 right: 2304 note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace thread...

Zero length allocation failure

5

Hi! I'm facing an issue with zero length memory allocation (while trying to run [`candle`](https://github.com/huggingface/candle) on GTX 970). Here is the minimal reproducer: ```rust let dev = cudarc::driver::CudaDevice::new(0).unwrap(); dev.null::().unwrap(); ```...

Extend `GradStore` public functionality

5

This PR allows user to merge two `GradStore`s together (and also to create an empty one). It is helpful for collecting gradients from multiple different backward passes (e.g. with different...

Also: + Require `InterruptContext` to be mutable to avoid introducing interior mutability. + Add `higher_priority_task_woken` method to it to have a way to check the flag value.

`RmsNorm` switches to faster implementation if tensor is contiguous: https://github.com/huggingface/candle/blob/82b641fd2752e3b14db6a9c91faef70e3329f3b5/candle-nn/src/layer_norm.rs#L174-L175 But it does not support backward pass: https://github.com/huggingface/candle/blob/82b641fd2752e3b14db6a9c91faef70e3329f3b5/candle-nn/src/ops.rs#L640 Maybe it's better to implement `ModuleT` rather than `Module` for `RmsNorm` and...

Require `PrimitiveAtom::Impl` to be `Send + Sync`

1

Atomic types naturally should be `Send + Sync` but [`PrimitiveAtom::Impl`](https://docs.rs/atomig/latest/atomig/impls/trait.PrimitiveAtom.html#associatedtype.Impl) does not have these bounds that leads to necessity to write additional `where`-clauses anywhere `Atomic` is used. Maybe these bounds...