Tim Dettmers

Optimizer state loading fix for bitsandbytes 8-bit optimizers.

8

This is a fix for loading [bitsandbytes 8-bit optimizer](https://github.com/facebookresearch/bitsandbytes) with optimizer sharding. By default, DeepSpeed shards all tensors held by the optimizer state automatically. However, 8-bit optimizers also hold the...

**Describe the bug** When one uses run_simple() with different models of the same type `roberta-base` and `roberta-large` the run crashes because the code assumes they are the same model because...

Added strided elewise ops (logistic, tanh).

1

The element-wise operations are performed on the last dimension on a given index. It assumes that tensors are stacked row-wise but that their dimension is increased in the next dimension...

Error on multiple paragraphs with Spacy >=2.0

5

``` Traceback (most recent call last): File "test.py", line 23, in support=[support, support2, support3] File "/home/tim/git/jack/jack/core/reader.py", line 84, in __call__ batch = self.input_module(inputs) File "/home/tim/git/jack/jack/core/input_module.py", line 185, in __call__ annotations...

Missing Windows support

4

Currently, the library uses precompiled Linux binaries. I am unsure how compatible these are with standard PyTorch installs on Windows. It might be that the binaries need to be compiled...

Quantization functions test fail on Pascal

1

The following tests fail on Pascal: ``` tests/test_functional.py::test_estimate_quantiles[float] FAILED tests/test_functional.py::test_estimate_quantiles[half] FAILED tests/test_functional.py::test_quantile_quantization FAILED ``` My guess is this is probably due to `atomicAdd` for floats working differently.