NNlib.jl

NNlib.jl copied to clipboard

Add minimal infrastructure for the docs

2

This currently does not add any new doctests or improve the existing docs, as that would make this PR massive. Here are the docstrings that are missing from the manual...

Saransh-cpp

Create independent documentation for `NNlib.jl`?

5

`NNlib.jl` does not have independent documentation, rather, it has a [dedicated page](https://fluxml.ai/Flux.jl/stable/models/nnlib/) in `Flux`'s documentation. Due to the lack of stand-alone documentation, it does not have doctests, and the documentation...

Saransh-cpp

https://github.com/FluxML/NNlib.jl/blob/023cd3da63892b754b4197fe7a848093128f2bf9/src/scatter.jl#L206-L212 https://github.com/FluxML/NNlib.jl/blob/023cd3da63892b754b4197fe7a848093128f2bf9/src/gather.jl#L82-L87

vpuri3

the PR in comments is merged https://github.com/FluxML/NNlib.jl/blob/023cd3da63892b754b4197fe7a848093128f2bf9/test/test_utils.jl#L13-L19

vpuri3

Functions like gather/scatter give scalar indexing errors if used on CuArrays without remembering to load NNlibCUDA. Since there is now a very lightweight GPUArraysCore, I think NNlib should depend on...

mcabbott

Add gradients for `conv_bias_act`, and a similar `dense_bias_act`

4

This aims to add gradient definitions for the existing `conv_bias_act`. That is, however, very much WIP, and I don't recommend anyone try to read it just yet. It also adds...

mcabbott

add topk

6

Initial steps to fix #352 TODO - [ ] give more thought to the interface (which outputs do we expect?) - [ ] add rrule or write it in an...

CarloLucibello

in conjunction with https://github.com/FluxML/NNlibCUDA.jl/pull/32, add support for half-precision `gemm`, for which a special kernel is provided by Nvidia. see https://github.com/JuliaGPU/CUDA.jl/pull/1080

bjarthur

add sparsemax

6

Initial sparsemax implementation #354 TODO - [x] add support for vectors (similar to softmax) - [ ] add comprehensive testing - [x] compare gradients with PyTorch implementations

tylerjthomas9

always put the index on GPU if the soure array is there when using `gather`

2

```julia using CUDA, NNlib using Flux:gpu #this imports NNlibCUDA using BenchmarkTools CUDA.allowscalar(false) a = CUDA.rand(200,3000,64) idx = rand(1:64,500) idx_gpu = idx |> gpu @benchmark CUDA.@sync NNlib.gather(a, idx) @benchmark CUDA.@sync NNlib.gather(a,...

YichengDWu