wxp

[ALBERT] Fine-tune SQUAD, got error in `convert_examples_to_features()`

4

In particular, the error is Index Error: list index out of range in `_convert_index()`

[ALBERT] How to deal with `Model diverged with loss = NaN` when training from scratch?

6

hi, I tried to train `albert base` model from scratch on 64 v100 gpus. Part of my hyper parameters are : `init_lr=2.75e-5 (i.e. 0.00173/64)` ` train_batch_size=48` `train_steps=32millions, so 500K for...

Hi, What are the parameters setting for training data generation for your released models? like, what is the `dupe_factor`, whether do `whole word masking`? I just found the `max_seq_len`, mask_probability,...

Can anyone download all the files in the url list file?

13

I tried to download the bookscorpus data. So far I just downloaded around 5000 books. Can anyone get all the books? I met a lot `HTTP Error: 403 Forbidden` How...

[DocBERT] Can DocBERT bin any length document without truncaiton?

1

Does DocBERT still have the max_seq_len limitation, like BERT?

We have a separate feedback API endpoint located in the cloud. Instead of using the custom datalayer provided by Chainlit, I want to add two buttons at the end of...

Finetuned `zephyr-7b-beta` with internal data generates same reuslts as model `HuggingFaceH4/zephyr-7b-beta`

1

use huggingface pipleline to run inference task, but found finetuned `HuggingFaceH4/zephyr-7b-beta` and model `HuggingFaceH4/zephyr-7b-beta` generates exactly otuputs. Does anyone have any clue about this error?