Kevin Yin
Kevin Yin
Kubuntu 23.04 Reproducer 1: 1. Create a folder with a UTF-8 character. For example, I created mine at "/home/a/Downloads/û/" 2. Put any image file inside. Mine is "/home/a/Downloads/û/a.jpg" 3. Open...
Kubuntu 23.04 filesystem mounted using LUKS Steps: 1. mount a LUKS filesystem. I used cryptsetup for this. 2. Put any image file in the mounted filesystem. I'll assume it's located...
I'm 7 years late; hope this repo is still alive! :smiley:
Some transformers (like x-transformers) take in a sequence of length (seq_len+1), then split it into input=x[:-1] and target=x[1:], and calculate the loss directly in forward(). This is efficient because the...
I followed https://github.com/andrewrk/libsoundio/wiki/Compiling-for-Windows and it worked, and my existing system had all the appropriate build tools pre-installed. Prior to this, I was about to give up on libsoundio before trying...
https://github.com/pytorch/torchtitan/blob/d442743fed7980392a00eecd464b6db8522d8116/torchtitan/parallelisms/__init__.py#L46 Should be PP DP TP. This matters for NUMA across nodes.
https://github.com/pytorch/torchtitan/blob/4e5ffafb6e1ebc159ca57625c875d0d44e5a654a/torchtitan/models/llama/model.py#L374 Luckily, this line is never activated, since the model is always constructed on the `meta` device in torchtitan, which makes this a no-op. In the general case, if the...
### 🐛 Describe the bug 512M parameters Mostly vanilla LM transformer. FlashAttention 2.4.2, PyTorch 2.2.0. Uses both FA and FlashRotary. Dtype: bf16 Nvidia A40. single-GPU Unfused: 85 TFLOPS Fused: 68...
https://pytorch.org/tutorials/beginner/examples_autograd/two_layer_net_custom_function.html "ctx is a context object that can be used to stash information for backward computation. You can **cache arbitrary objects** for use in the backward pass using the ctx.save_for_backward...
Firefox 116.0.3, Kubuntu 23.04, pipewire I hear a repetitive clicking noise on top of the expected sound. Clicking the pause button didn't help.