preprocessor icon indicating copy to clipboard operation
preprocessor copied to clipboard

Bug when using Tokenize with the URL option

Open lezakkaz opened this issue 4 years ago • 0 comments

Describe the bug I want to replace URLs in the text with a URL tag, it generally works well but with some input, my code seems to bug with no errors whatsoever.

To Reproduce Steps to reproduce the behavior:

import preprocessor as p
p.set_options(p.OPT.URL)
line = '.......https://www.youtube.com/watch?v=DVdUHfKovn4...........................................' 
print(p.tokenize(line))

Expected behavior A clear and concise description of what you expected to happen. '.......$URL$...........................................'

Screenshots If applicable, add screenshots to help explain your problem. asdasdasfasfasd

Desktop (please complete the following information):

  • OS: Windows-10 64
  • Python Version: 3.8.5
  • preprocessor version 0.6.0

Additional context Nop

lezakkaz avatar Oct 14 '21 08:10 lezakkaz