Oufattole

The use case I have in mind for additional data is that users may wish to extract windows of data for contrastive learning tasks. So I may wish to extract...

People expressed interest in how to use ACES on MIMICIV. Is the plan to add a MIMICIV tutorial with some explanations of what aces is doing under the hood? Does...

Ahhh good point, you can't really show much under the hood stuff due to privacy constraints. How about demonstrating 1. a MIMICIV a workflow of getting cohort stats on various...

I'm trying to do sliding window inference, but the lengths of the initial prompts are different in my transformer, so I think I should mask out the padding as that's...

Ahhh I see thank you, I'll try that! With medical data, unlike in NLP and CV, many patient trajectories are very small and you don't need a long sequence length...

Wait, actually, I think you do support masking the left padded tokens with the [seq_start_pos arg here](https://github.com/lucidrains/x-transformers/blob/abeedc8cb60180892f1c19b42548c9833abe4f5b/x_transformers/x_transformers.py#L1921C19-L1921C32) @lucidrains .

Yes, I think you already do this kv-cache slicing during generation [here](https://github.com/lucidrains/x-transformers/blob/abeedc8cb60180892f1c19b42548c9833abe4f5b/x_transformers/autoregressive_wrapper.py#L230) when restricting to the max_seq_length (i.e. in the sliding window setting). Am I correct about this? I'll send...

I think this does the trick @justin13601 : https://colab.research.google.com/drive/12YCgq3wE2lythDb3719hrMyY8B2xb_Wd?usp=sharing