linear-dmlc: segmentation fault for large train file

[iamzhyuyang]

Hi,

Thank you very much for such a great tools! Recently I'm trying to use linear-dmlc base on the provided demo. However, when I change the train file to a real-world file as large as 5GB, I got segmentation fault:

Core was generated by `wormhole/bin/linear.dmlc news.conf'. Program terminated with signal SIGSEGV, Segmentation fault. #0 dmlc::Localizer::CountUniqIndex (

idx_frq=0x0, uniq_idx=0x7ff50c000c40, blk=..., this=0x7ff51f1de090)
at ../base/localizer.h:141

141 I curr = pair_[0].k; [Current thread is 1 (Thread 0x7ff51f1df700 (LWP 22147))]

I checked the size of "pair_", and it returned "0". If I split the 5GB file to 5 1GB small files, linear-dmlc works as expected.

Any idea on the problem? Thanks!