THULAC-Python icon indicating copy to clipboard operation
THULAC-Python copied to clipboard

thu1 = thulac.thulac() memory error

Open HappyShadowWalker opened this issue 8 years ago • 5 comments

Traceback (most recent call last): File "", line 1, in File "C:\Python27\lib\site-packages\thulac_init_.py", line 58, in init self.__tagging_decoder.init((self.__prefix+"model_c_model.bin"),(self.__prefix+"model_c_dat.bin"),(self.__prefix+"model_c_label.txt")) File "C:\Python27\lib\site-packages\thulac\character\CBTaggingDecoder.py", line 36, in init self.model = CBModel(modelFile) File "C:\Python27\lib\site-packages\thulac\character\CBModel.py", line 58, in init self.fl_weights = struct.unpack("<"+str(self.f_size * self.l_size)+"i", temp) MemoryError

HappyShadowWalker avatar Apr 20 '17 03:04 HappyShadowWalker

感谢您对thulac的支持,由于我们之前的开发没有针对windows进行,所以Windows上面python2.7兼容目前有些问题。不过从之前的反馈来看,windows上python3.6不会有这个问题

MaJunhua avatar Apr 20 '17 06:04 MaJunhua

编辑CBModel.py,找到__init__方法,找到下面这个代码:

temp = inputfile.read(4 * self.l_size * self.l_size)
self.ll_weights = struct.unpack("<"+str(self.l_size * self.l_size)+"i", temp)
self.ll_weights = tuple(self.ll_weights)

temp = inputfile.read(4 * self.f_size * self.l_size)
self.fl_weights = struct.unpack("<"+str(self.f_size * self.l_size)+"i", temp)

改成

temp = inputfile.read(4 * self.l_size * self.l_size)
self.ll_weights = array.array('i')
self.ll_weights.fromstring(temp)
self.ll_weights = tuple(self.ll_weights)

temp = inputfile.read(4 * self.f_size * self.l_size)
self.fl_weights = array.array('i')
self.fl_weights.fromstring(temp)

再加上import

import array

python2.7适用,3.6有这个API,也能兼任

PaniniGelato avatar Jul 11 '17 05:07 PaniniGelato

PaniniGelato's answer works for me on windows. Python 2.7.13.

Pacific73 avatar Jul 24 '17 03:07 Pacific73

PaniniGelato's answer saved ~330MB when initialize thulac, python 2.7

LiXiaoB avatar Apr 28 '19 06:04 LiXiaoB

python3.6,仍然出现了上述问题;采用了PaniniGelato的方法后同样会出现

KisAaki avatar Jun 10 '22 13:06 KisAaki