Ben Murrell
Ben Murrell
This is now working on GPU. In combination with https://github.com/FluxML/Zygote.jl/pull/1541 I can now train a 7 billion parameter transformer on a single 48gb A6000 GPU:
Thanks @CarloLucibello - I just woke up and noticed some issues with this PR, so I was about to convert it to draft! I need to sort out `adjust` for...
FYI I've made this: https://github.com/MurrellGroup/CannotWaitForTheseOptimisers.jl which can house new optimisers until you folks are happy to have them included in Optimisers.jl (mostly because I really just need to be able...
FYI the way we handle this in [MolecularEvolution.jl](https://github.com/MurrellGroup/MolecularEvolution.jl), where a phylogeny can be thought of as a branching HMM (instead of a lattice), is to have `forward` and `backward` functions...
We don't work in the log domain.
Ah thank you! I was 100% unaware of these, and this almost certainly solves my problem (and has been immediately handy for the Stockholm sequencing folks too). For anyone else...
Ah, in case this is the sort of "small issue" you fix, XDV.1 does not appear to be a leaf node one that tree (and my software is upset by...