Axel Magnuson

slow transfer speeds from URL sources

5

I am working on ingesting the RPV2 dataset onto GCS buckets using GCP storage transfer jobs. Speeds seem to be incredibly slow (on the order of 100KB/s - 1MB/s), and...

Inconsistent IDs lead to distributed computing woes.

1

When trying to work with these data via Dataflow, I noticed a few things: - the ID field key is inconsistent between files. it is `id` in minhash and signals,...

I am working on a pipeline similar to FineWeb, and the time has come to start scaling up. I am curious: what were the specs for the Slurm cluster used...

Rust crate housekeeping

1

I recently found out about Steel and it seems like a great fit for a personal project of mine. However I am curious about its state of development. Development seems...

Type Error in GPTLMHeadModel

7

I am having a go at running inference and evaluation for this model, and running into a TypeError in `GPTLMHeadModel`: ``` In [1]: import torch ...: from transformers import AutoTokenizer...

FYI: HuggingFace Transformers Request

1

Hi all, just a heads up: I filed an [issue](https://github.com/huggingface/transformers/issues/29466) with `huggingface/transformers` requesting model support for BASED via their library. My engagement over the past few days has been part...

ld.lld error while building examples: archive member is neither ET_REL nor LLVM bitcode

14

When I run `zig build examples` with the latest code, I get the following error: ``` examples +- zig build-exe texture_outline Debug native failure error: warning(link): unexpected LLD stderr: ld.lld:...

How to handle utility libraries that have external dependencies

10

Today I wanted to add proper font rendering to my project, and thought I would reach for `sokol_fontstash.h`. My first approach was to try to monkey-patch it into `mod_sokol_clib` by...