python-arpa icon indicating copy to clipboard operation
python-arpa copied to clipboard

The efficiency of computing backoff

Open huangruizhe opened this issue 6 years ago • 0 comments

https://github.com/sfischer13/python-arpa/blob/2284b815866aeb08f65f786da416e78d7937ee1d/arpa/models/base.py#L34-L45

This try...catch mechanism to implement the backoff may not be efficient enough. According to the python documentation:

A try/except block is extremely efficient if no exceptions are raised. Actually catching an exception is expensive.

However, it is common in a language model to have unseen ngrams and to backoff to lower orders. Thus, I guess it may be more appropriate to implement this using if...else (as also suggested in the documentation) instead of try...catch.

huangruizhe avatar Jul 30 '19 20:07 huangruizhe