David Hou

Clarify policy on breaking changes

3

Theres no general policy on breaking changes in CONTRIBUTING.md; I think it'd be useful to have one.

[WIP] Winograd Convolution

3

WIP. Code/progress is not interesting at the moment. Posting just for reference. Approach: - _pool 4x4 with stride=2*stride - permute kernel HW to the first 2 dims. - winograd by...

Dedup kernel args (bufs) that have the same underlying RawBuffer

8

This is part 1/k for winograd ( #1037 ). I will be splitting off orthogonal changes from that PR so that I can get some feedback / have less going...

fp16 resnet

2

this used to at least produce non-nan losses. testing it...

tricky assign tests

2

these should all pass (or assert) not comprehensive!

[poc/wip] new split reduce heuristic

1

Ensure occupancy. Optimize layout for group. Don't waste big memory. Probably really slow without beam

use f32 by default to calculate in randn

2

There are many places in the codebase where fp16 is not adequate for some particular calculation. sum() is handled well; it selects least upper dtype with float32. We might have...

initialize Tensor grad same type as self

4

when a tensor has different dtype then default_float, backward will initialize gradient to the wrong type (default instead of self)

realize unsafe pads even if they don't expand the buffer

8

a previously failing test and a quick fix. checking for unsafe pads can just be its own pass. need to think about this expand rule!

if you write t.expand(), the backwards is sum(), but if t is fp16, then the sum will be in fp16 and may not have a good time. need to measure...