Taekyoon issues

Repositories
Issues
Comments

Results 4 issues of


                                            Taekyoon

Refactor RDD process to Dataframe process

enhancement

Need to add ignore null or empty text during korean text process

### Agenda - Some raw datasets can have empty or null text. - Using `filter` method in RDD or DF, text like `""` need to be ignored during this process....

bug

Add scripts to run hadoop cluster

enhancement

Add build news paper dataset as long text data form

### Context - Build news paper dataset as long text form - The data comes from (modu dataset)[https://corpus.korean.go.kr/main.do] - 신문 말뭉치 2021

add job