ucto icon indicating copy to clipboard operation
ucto copied to clipboard

add possibility to add extra user-defined rules on startup

Open kosloot opened this issue 7 years ago • 0 comments

recently an --add-tokens option was introduced to Ucto to add extra 'TOKENS' to the configuration. We might consider extending this, so a user could add extra, non-default rules/items to the tokenizer.

Some caveats to consider:

  • are the extra rules additional, of do they override?
  • make it possible to disable a certain rule
  • are the additions language specific? How to express that

kosloot avatar Apr 10 '18 08:04 kosloot