Cloud9

Cloud9 copied to clipboard

Changes to Aquaint2 code

1

Optimizations to support indexing English Gigaword 5th ed (10M docs).

earljwagner

More wikipedia language support, wiki disambiguation entity extraction, and output wiki docId in plaintext dump

3

- Increased language support for Wikipedia for top 24 languages by # of articles - Added disambiguation patterns for each of the 24 supported languages - ExtractWikipediaDisambiguations lets you extract...

gragtah

Some error checks for parsing Wikipedia dumps and English wikipedia pages.

seweissman

With this change, one should be able to process a bzip2 directly. Let me know if you have any comment.

goiri

Very minor change on line 211 to make sure AWS EMR doesn't throw errors.

aligajani