Larry Law

Llama fast tokenizer `train_new_from_iterator` returns `TypeError: 'NoneType' object is not subscriptable`

2

### System Info accelerate==0.18.0 aiohttp==3.8.4 aiosignal==1.3.1 anyio==3.6.2 argon2-cffi==21.3.0 argon2-cffi-bindings==21.2.0 arrow==1.2.3 asttokens==2.2.1 async-timeout==4.0.2 attrs==23.1.0 backcall==0.2.0 beautifulsoup4==4.12.2 bitsandbytes==0.38.1 bleach==6.0.0 certifi==2022.12.7 cffi==1.15.1 charset-normalizer==3.1.0 cmake==3.26.3 comm==0.1.3 datasets==2.11.0 debugpy==1.6.7 decorator==5.1.1 defusedxml==0.7.1 dill==0.3.6 evaluate==0.4.0 executing==1.2.0 fastjsonschema==2.16.3...

Using `auto_map` in `tokenizer_config.json` gives `TypeError: argument of type 'NoneType' is not iterable`

2

### System Info certifi==2022.12.7 charset-normalizer==3.1.0 cmake==3.26.3 filelock==3.12.0 fsspec==2023.4.0 huggingface-hub==0.14.0 idna==3.4 Jinja2==3.1.2 lit==16.0.2 MarkupSafe==2.1.2 mpmath==1.3.0 networkx==3.1 numpy==1.24.3 nvidia-cublas-cu11==11.10.3.66 nvidia-cuda-cupti-cu11==11.7.101 nvidia-cuda-nvrtc-cu11==11.7.99 nvidia-cuda-runtime-cu11==11.7.99 nvidia-cudnn-cu11==8.5.0.96 nvidia-cufft-cu11==10.9.0.58 nvidia-curand-cu11==10.2.10.91 nvidia-cusolver-cu11==11.4.0.1 nvidia-cusparse-cu11==11.7.4.91 nvidia-nccl-cu11==2.14.3 nvidia-nvtx-cu11==11.7.91 packaging==23.1 PyYAML==6.0 regex==2023.3.23...

Accelerate support for GLM

6

### Feature request Accelerate support for [GLM](https://github.com/THUDM/GLM). ### Motivation [GLM](https://github.com/THUDM/GLM) is a SOTA chinese LLM. However, running the following code... ``` from transformers import AutoModelForSeq2SeqLM model = AutoModelForSeq2SeqLM.from_pretrained("THUDM/glm-10b", trust_remote_code=True, device_map="auto",...

Results of pre-training stage 1

2

Thank you for using the Issue submission template. Please follow the steps below to provide relevant information. We will prioritize issues with relatively complete information. Your cooperation is appreciated. *Hint:...

Deepspeed ZeRO2 + Trainer does not resume training after evaluation

9

### System Info - `transformers` version: 4.30.2 - Platform: Linux-4.18.0-305.25.1.el8_4.x86_64-x86_64-with-glibc2.35 - Python version: 3.10.6 - Huggingface_hub version: 0.15.1 - Safetensors version: 0.3.1 - PyTorch version (GPU?): 2.0.0 (False) - Tensorflow...

Hi, I've made a [PR](https://huggingface.co/THUDM/glm-10b-chinese/discussions/2) to support accelerate for GLM. Can you help review it? Example code: ``` from transformers import AutoModelForSeq2SeqLM device_map={'glm.word_embeddings': 0, 'glm.transformer.embedding_dropout': 0, 'glm.transformer.position_embeddings': 0, 'glm.transformer.block_position_embeddings': 0,...

**Motivation.** I want `OpusRead.printPairs` to be a generator for downstream task. Specifically, I intend to share Opus as a huggingface dataset (see: `DatasetBuilder._generate_examples` in [link](https://huggingface.co/docs/datasets/dataset_script#generate-samples)). **Change**. Added `yield_tuple` write mode...

Hi, thanks for releasing this dataset! I've made the dataset available on huggingface: https://huggingface.co/datasets/larrylawl/douban-dushu Do you mind adding this link to your README.md? Thanks!