Kazuma TAKAOKA

Remove prolonged sound mark

1

``` ご期待くださいーー!! ご 接頭辞,*,*,*,*,* 御 期待 名詞,普通名詞,サ変可能,*,*,* 期待 くださ 動詞,一般,*,*,五段-サ行,未然形-一般 下す いーー 感動詞,フィラー,*,*,*,* いー ! 補助記号,句点,*,*,*,* ！ ! 補助記号,句点,*,*,*,* ！ ``` Expected result: ``` ご期待くださいーー!! ご 接頭辞,*,*,*,*,* 御 期待 名詞,普通名詞,サ変可能,*,*,*...

The first column of a source of user dictionary is a headword for TRIE. Because input texts are normalized by `DefaultInputTextPlugin`, the headwords must be normalized in the same way....

Known word and OOV are different in segmentation although their word structures are the same. > 全国的 名詞,普通名詞,形状詞可能,*,*,* 全国的 > 間接 名詞,普通名詞,一般,*,*,* 間接 > 的 接尾辞,形状詞的,*,*,*,* 的 Adjust them by...

「にぎり寿司三億年」 === Before rewriting: 0: 0 24 にぎり寿司三億年(1634131) 2 5142 5154 13355 === After rewriting: 0: 0 9 にぎり(136394) 4 0 0 0 1: 9 15 寿司(700728) 4 0 0...

Change OOV to person name by context (part-of-speech or title)

- Build and upload a binary synonym dictionary - Automatic update of *-latest.zip

Update the installation script for pip

1

``` $ pip install SudachiDict-core Collecting SudachiDict-core Downloading SudachiDict-core-20221021.tar.gz (9.0 kB) Preparing metadata (setup.py) ... done Requirement already satisfied: SudachiPy=0.5 in ./.pyenv/versions/3.11.1/lib/python3.11/site-packages (from SudachiDict-core) (0.6.6) Installing collected packages: SudachiDict-core DEPRECATION:...

When inputting "〇所定勤務時間", it is outputted as a single OOV. - The kanji numeral "〇" is assigned to the character categories SYMBOL, KANJI, and KANJINUMERIC. - The OOV generation rule...