Alex

Tokenizer fixes and span_tokenize method

2

Tokenizer from #15 had issues like not splitting a dot at the end of a sentence as a separate token ``` 40006,40007c40017 < community < . --- > community. 41148,41149c41158...

OSError: cannot load library libnanomsg.so.5.0.0

5

`nnpy` install fails with the following traceback ``` $ sudo pip install nnpy Downloading/unpacking nnpy Downloading nnpy-1.3.tar.gz Running setup.py (path:/tmp/pip_build_root/nnpy/setup.py) egg_info for package nnpy Traceback (most recent call last): File...

Fix set_level side effects

3

On test case completion all modifications to loggers levels are reversed. Fixes #60

caplog.set_level has side effects if logger name is passed explicitly

2

If I pass logger name to caplog.set_level like this ``` caplog.set_level(logging.ERROR, logger='package.utils') ``` then for all following test cases that logger will have level set to ERROR, which is not...

Add --max-tags-count parameter

2

Limit number of tags per docker repository

Add CRAWLER setting to be able to set custom crawler via settings file

4

In my case it's needed for ScrapyRT - we are using custom Crawler class there. I started working on this PR before I noticed #1147, so now I have doubts...