Denis Kochetkov

Regarding tests `test_batch_generation` and `test_batch_generation_2heads`. If token initialisation class is changed form `GPT2Tokenizer` to `GPT2TokenizerFast` the test passes through until generated tokens assertion. Is it intended behaviour or the loading...

Sorry, @jlamypoirier, I meant data preparation during training, not data preparation using the `prepare` command. When I start training experiment, it takes about 6.5 minutes to process ~1.2k files. I...

It could be a mismatch between the PyTorch version used at *build time* and the one used at *runtime*. To make sure the extension is built against the PyTorch already...

I have created a debugging sandbox with manual tests for now. The results are as follows: ## Ignoring `attention_mask` and `position_ids`: | Batch Size | No Flash Attention (Float32) |...

@sohamparikh @jlamypoirier Hi, I am trying to use the cross-document attention prevention that @tscholak pointed me to (https://github.com/ServiceNow/Fast-LLM/pull/177/files) to mimic left padding for documents in a batch during generation. It...

## Current State - Implemented evaluation abstraction and `lm_eval` integration for single GPU. - Made necessary changes to `generate()`. ## Next Steps - Refactor `lm_eval` integration to rely less on...

I’ve finished working in this draft and will create 3 new PRs from it: - Generate support - Refactoring of evaluations - `lm_eval` integration In addition to the changes here,...

Work on this prototype branch has been completed and moved to other feature branches. This PR can be safely closed.

## Demonstration and Discussion of Concept This code serves as a demonstration and discussion of the proposed concept. ### Key Decisions: - **Avoiding `runnable`**: I skip using `runnable` for a...

## Created basic implementation based on feedback. - Word splitting and number detection implemented using regex. - Binary data detection is based on non-printable characters. - **Presidio Model Management:** Needs...