Paul Richardson

4bit LoRA "--monkey-patch" breaks "--gpu-memory" Model Splitting for Multi-GPU

5

### Describe the bug Trying to apply [kuleshov/llama-65b-4bit](https://huggingface.co/kuleshov/llama-65b-4bit) to [Neko-Institute-of-Science/LLaMA-65B-4bit-128g](https://huggingface.co/Neko-Institute-of-Science/LLaMA-65B-4bit-128g) `--monkey-patch` seems to ignore `--gpu-memory` ### Is there an existing issue for this? - [X] I have searched the existing...

Question: Is alpaca-chatbot-format Made for Vicuna?

1

Probably really basic question but is alpaca-chatbot-format correspond to the ShareGPT dataset as found here: https://huggingface.co/datasets/gozfarb/ShareGPT_Vicuna_unfiltered? It needs to support conversations as examples IE sets of messages per example like...

Currently modules/training.py is padding with `0` the **token** vs `0` the **token id** token_id 0 = "``" (for llama at least) token 0 = "`0`"

## Changes This PR introduces a feature that creates a train_dataset_sample.json file in the /logs directory. The file allows users to inspect a sample (up to 10 entries) of data...