Off-by-One Issue?

[janstarke]

Hi

I've just pulled and built bleeding-jumbo with rexgen. I've not observed any sign of memory leak and there is no sign of missing that "missing characters" bug but... There is difference between two commands: $ echo pass|./john --stdin --stdout --regex=case="(?i:y|n)\0(?i:y|n)"

and

$ echo pass|./john --stdin --stdout --regex=alpha="(?i:y|n)\0(?i:y|n)"

Just using option "alpha" with or without alphabet will cause that any text after \0 will be copied one place to early. I'm glad that there is a debug print buf, because buf is "pass" but passwords are e.g. "npasn". If someone will use any alphabet e.g. case or leet than this bug is causing this e.g.

$ echo pass|./john --stdin --stdout --regex=alpha:leet="(?i:y|n)\0(?i:y|n)" Using default input encoding: UTF-8 buf=p[a4@][s$5][s$5] error: syntax error, unexpected T_BEGIN_GROUP, expecting T_END_CLASS Error, invalid regex expression. John exiting now base_word=pass Regex= (?i:y|n)p[a4@][s$5][s$5(?i:y|n)

Furthermore alpha:case without case option is doing nothing.

$ echo pass|./john --stdin --stdout --regex=alpha:case="(?i:y|n)\0(?i:y|n)" Using default input encoding: UTF-8 buf=pass pass 1p 0:00:00:00 0.00% 5.555p/s pass

$ echo pass|./john --stdin --stdout --regex=case=alpha:case="(?i:y|n)\0(?i:y|n)" Using default input encoding: UTF-8 buf=[pP][aA][sS][sS] error: syntax error, unexpected T_BEGIN_GROUP, expecting T_END_CLASS Error, invalid regex expression. John exiting now base_word=pass Regex= (?i:y|n)[pP][aA][sS][sS(?i:y|n)

Of course there is workaround if you know what character should be at the end of buf and add it as the first character after \0 e.g.

$ echo pa|./john --stdin --stdout --regex=case=alpha:case="(?i:y)\0](?i:n)" Using default input encoding: UTF-8 buf=[pP][aA] ypan Ypan yPan YPan ypAn YpAn yPAn YPAn ypaN YpaN yPaN YPaN ypAN YpAN yPAN YPAN 16p 0:00:00:00 0.00% 94.11p/s YPAN

Maybe this will help to find this off-by-one error.

Best Regards

[frank-dittrich]

I think this could be a john issue instead of a rexgen issue, no? This is similar, I guess:

(bleeding-jumbo)run $ echo pass|./john --stdin --stdout --regex=alpha:leet='\0\0'
Using default input encoding: UTF-8
buf=p[a4@][s$5][s$5]
error: syntax error, unexpected T_BEGIN_CLASS, expecting T_END_CLASS
Error, invalid regex expression.  John exiting now  base_word=pass  Regex= p[a4@][s$5][s$5p[a4@][s$5][s$5]

(bleeding-jumbo)run $ echo pass|./john --stdin --stdout --regex='\0\0'=alpha:leet
Using default input encoding: UTF-8
buf=p[a4@][s$5][s$5]
error: syntax error, unexpected T_BEGIN_CLASS, expecting T_END_CLASS
Error, invalid regex expression.  John exiting now  base_word=pass  Regex= p[a4@][s$5][s$5p[a4@][s$5][s$5=

[frank-dittrich]

This is the workaround for the first command:

echo pass|./john --stdin --stdout --regex=alpha:leet='\0]\0'

This for the second:

echo pass|./john --stdin --stdout --regex='\0]\0]'=alpha:leet

Two ] need to be added for the second command, to also avoid the error: syntax error, unexpected $end, expecting T_END_CLASS error.

[janstarke]

I currently have time for working on this issue. Do you still know if this was a john-issue or a rexgen-issue?