segyges

Tests fail when run with pytest --forked

2

**Describe the bug** When tests are run with pytest --forked per the instructions in /test/README.md, a large number of tests fail with the error: ```RuntimeError: Cannot re-initialize CUDA in forked...

Remove unused requirements-sparseattention

2

- Removes this requirements file - Removes reference to it in the dockerfile - Also removes protobuf from the dockerfile since it is apparently unused and I noticed it -...

Per [this](https://github.com/EleutherAI/gpt-neox/pull/1144), my understanding is that the ```gas``` config in neox doesn't do anything, and shouldn't be used, and should be removed. We should be using ```gradient_accumulation_steps``` instead. It [appears](https://github.com/search?q=repo%3AEleutherAI%2Fpythia+gas&type=code&p=1)...

This seems advisable because the dataloader uses numpy extensively and oddities may crop up later with numpy updates.