Dean Wyatte

The docs for `tfrs.tasks.Retrieval.num_hard_negatives` note If positive, the num_hard_negatives negative examples with largest logits are kept when computing cross-entropy loss. If larger than batch size **or non-positive**, all the negative...

Minimal support for SavedModel in WarmStartSettings

3

This PR adds support for using a SavedModel with `WarmStartSettings` by passing `[saved_model_export_dir]/variables` to `WarmStartSettings.ckpt_to_initialize_from` as discussed in https://github.com/tensorflow/tensorflow/issues/25400. It may be nicer to just change `WarmStartSettings` to accept the...

The original prepared SODA synthetic dialogue dataset was removed from https://drive.google.com/uc?id=1TOGQfr419n8wpzJpYLLw4nB3tSKD8zXV (referenced [here](https://github.com/LAION-AI/Open-Assistant/blob/ada91f1c37b793ff19b0d0f0197d59aa019a4375/model/model_training/custom_datasets/qa_datasets.py#L279)) https://github.com/LAION-AI/Open-Assistant/tree/ada91f1c37b793ff19b0d0f0197d59aa019a4375/data/datasets/soda_synthetic_dialogue contains the code to prepare it. Ideally it should be hosted in a HuggingFace datasets repo...

Fixes https://github.com/LAION-AI/Open-Assistant/issues/2670 Output ``` For SODA dialogue dataset found 427 faults within the total 1486896 dialogs ```

Update README to indicate that deepspeed launcher should be used

1

Was OOMing when following the instructions here. This should be more scaleproof and should still work for smaller models as far as I know

Add data loader for HF oasst1

3

Currently, data needs to be manually downloaded and a path specified via config to train on OpenAssistant datasets (both internal and https://huggingface.co/datasets/OpenAssistant/oasst1/blob/main/2023-04-12_oasst_ready.trees.jsonl.gz) so that the tree can be parsed by...

### System Info ```shell optimum main (commit e164827db4ed46c14979fdaef970865784c54412) ``` ### Who can help? @michaelbenayoun @fxmarty ### Information - [X] The official example scripts - [ ] My own modified scripts...

Release aarch64 wheels

8

With tensorflow 2.10, it looks like TensorFlow is starting to support aarch64. If I recall correctly, tensorflow-text builds for aarch64. Would it be possible to start building those wheels as...

Add parquet-specific dataset IO for reduced memory usage

1

Currently, `blocks.assemble` expects all datafile frames to be in memory simultaneously for merging https://github.com/square/blocks/blob/5a19b13dfb67c659177058a96e19b5d785449a71/blocks/core.py#L77 Unfortunately, there are some platform-dependent memory allocation issues that are likely out of blocks' control where...

CUDA: an illegal memory access was encountered with Mistral FP8 Marlin kernels on NVIDIA driver 535.216.01 (AWS Sagemaker Real-time Inference)

2

### System Info Tested with text-generation-inference 2.4.0 and 3.0.0 Docker containers running the CLI from within on Sagemaker Real-time Inference (NVIDIA driver 535.216.01) ### Information - [x] Docker - [x]...