Jonas Pfeiffer
Jonas Pfeiffer
We have been in close contact with people from Hugging Face, and have been heavily discussing options of either merging the two libraries, or disentangling the `adapters` from `transformers`. However,...
Yeah thats what I thought. Alternatively, how much effort does it involve to have both settings available? A user is required to set either? In the frontend we would present...
makes sense. To bypass that we could *always* translate the `reduction_factor` into `bottleneck_size`. This is *always* possible, however not the other way around: ``` hidden_size = 768 reduction_factor = 32...
No, the `reduction_factor` is always and remains dependent on the actual `hidden_size`. So for `Base` the `hidden_size = 768`, for `Large` its `hidden_size = 1024` (I think). If `reduction_factor` is...
this has not been implemented yet correct?
No problem at all, I just had the PR set for approved and was worried that someone will merge it :)
not a priority. we can get back to it later
For unseen tokens of a seen script (e.g. latin) the tokenizer has the character-level fallback option. you can learn an entirely new tokenizer for unseen scripts. See https://aclanthology.org/2021.emnlp-main.800/