markov
markov copied to clipboard
Add iterator for tokens sorted by probability
For a compression library we need an iterator over all tokens sorted by its probability. This PR adds this feature.