PyxLSTM
PyxLSTM copied to clipboard
muditbhargava66
Efficient Python library for Extended LSTM with exponential gating, memory mixing, and matrix memory for superior sequence modeling.
The new sLSTM doesn't have the stabilizer state m. This leads to exploding gradients very easily.
## RuntimeError: Boolean value of Tensor with more than one value is ambiguous in xLSTM/slstm.py ### Issue Description I'm encountering a `RuntimeError` when attempting to execute a backward pass in...
The example given in the README.md file had a import statement that said: from xLSTM.utils import load_config, set_seed, get_device I am unable to find the implementation for the get_device function,...
https://github.com/muditbhargava66/PyxLSTM/blob/a34c2526aa2646203e6d512ed3186765bbd20b3b/xLSTM/mlstm.py#L89 This code will make a wrong in: https://github.com/muditbhargava66/PyxLSTM/blob/a34c2526aa2646203e6d512ed3186765bbd20b3b/xLSTM/mlstm.py#L68 RuntimeError: The size of tensor a (2534) must match the size of tensor b (256) at non-singleton dimension 1 Can you...
I was testing the module using this code : ``` from xLSTM.model import xLSTM import torch model = xLSTM(5, 8, 16, 5, 2, 0.1, True, 'slstm') inputs = torch.randint(low=0, high=5,size=(12,15000))...
After: ``` $ pip install . ``` do ``` $ python3 examples/language_modeling.py ... ModuleNotFoundError: No module named 'xLSTM.data' ```
The current problems fixed by this commit: Blocks have a linear layer at the end: `self.proj = nn.Linear(hidden_size, input_size)` Thus leads to an incompatibility, if you use multiple blocks: `xLSTMBlock(embedding_size...
Great job on the xLSTM repo, Mudit! It would be really helpful if you could include a sample dataset downloader to make running your examples easier.
Metadata
Owner
Metadata
Efficient Python library for Extended LSTM with exponential gating, memory mixing, and matrix memory for superior sequence modeling.