lux icon indicating copy to clipboard operation
lux copied to clipboard

Large multi-line regular expressions do not match

Open pillravi opened this issue 8 years ago • 6 comments

This seems to be a similar issue to #17, but on the expect side rather than the send side.

I am cat-ing a file multiple times to search for several different blocks of text. (I cannot use a file compare utility such as diff because the text blocks may appear in any order.)

Script 3 passes:

!cat test.txt
"""??
<48 bytes (5 lines)>
"""
!cat test.txt
"""??
<48 bytes (5 lines)>
"""
!cat test.txt
"""??
<4057 bytes (113 lines)>
"""
!cat test.txt
"""??
<4057 bytes (113 lines)>
"""

while script 4 fails:

!cat test.txt
"""??
<Block 1: 48 bytes (5 lines)>
"""
!cat test.txt
"""??
<Block 2: 48 bytes (5 lines)>
"""
!cat test.txt
"""??
<Block 3: 3875 bytes (108 lines)>
"""
!cat test.txt
"""??
<Block 4: 3875 bytes (108 lines)>
"""

with the following output:

result            : FAIL at 40:21:-1:22:121:1834
expected
	<Block 3: 3875 bytes (108 lines)>
actual match_timeout

	<Block 3: 3875 bytes (108 lines)>
	<Block 4: 3875 bytes (108 lines)>
	SH-PROMPT:cat test.txt
	cat test.txt
	<Block 1: 48 bytes (5 lines)>
	<Block 2: 48 bytes (5 lines)>
	<Block 3: 3875 bytes (108 lines)>
	<Block 4: 3875 bytes (108 lines)>
	SH-PROMPT:cat test.txt
	<Block 1: 48 bytes (5 lines)>
	<Block 2: 48 bytes (5 lines)>
	<Block 3: 3875 bytes (108 lines)>
	<Block 4: 3875 bytes (108 lines)>
	SH-PROMPT:

successful        : 0
failed            : 1
script_04.lux:40:21:-1:22:121:1834 - match_timeout
summary           : FAIL

There is no particular length that causes it to fail; from the examples we see that an expect of 4057 bytes succeeds but an expect of 3875 bytes fails. However this problem does only manifest on expects that are qualitatively "long."

I am using the latest Lux from the non-blocking-pty branch.

pillravi avatar Mar 16 '17 17:03 pillravi

Can you try tor reproduce it with the develop branch?

/Håkan

On Thu, Mar 16, 2017 at 6:42 PM, pillravi [email protected] wrote:

This seems to be a similar issue to #17 https://github.com/hawk/lux/issues/17, but on the expect side rather than the send side.

I am cat-ing a file multiple times to search for several different blocks of text. (I cannot use a file compare utility such as diff because the text blocks may appear in any order.)

Script 3 passes:

!cat test.txt """?? <48 bytes (5 lines)> """ !cat test.txt """?? <48 bytes (5 lines)> """ !cat test.txt """?? <4057 bytes (113 lines)> """ !cat test.txt """?? <4057 bytes (113 lines)> """

while script 4 fails:

!cat test.txt """?? <Block 1: 48 bytes (5 lines)> """ !cat test.txt """?? <Block 2: 48 bytes (5 lines)> """ !cat test.txt """?? <Block 3: 3875 bytes (108 lines)> """ !cat test.txt """?? <Block 4: 3875 bytes (108 lines)> """

with the following output:

result : FAIL at 40:21:-1:22:121:1834 expected <Block 3: 3875 bytes (108 lines)> actual match_timeout

<Block 3: 3875 bytes (108 lines)> <Block 4: 3875 bytes (108 lines)> SH-PROMPT:cat test.txt cat test.txt <Block 1: 48 bytes (5 lines)> <Block 2: 48 bytes (5 lines)> <Block 3: 3875 bytes (108 lines)> <Block 4: 3875 bytes (108 lines)> SH-PROMPT:cat test.txt <Block 1: 48 bytes (5 lines)> <Block 2: 48 bytes (5 lines)> <Block 3: 3875 bytes (108 lines)> <Block 4: 3875 bytes (108 lines)> SH-PROMPT:

successful : 0 failed : 1 script_04.lux:40:21:-1:22:121:1834 - match_timeout summary : FAIL

There is no particular length that causes it to fail; from the examples we see that an expect of 4057 bytes succeeds but an expect of 3875 bytes fails. However this problem does only manifest on expects that are qualitatively "long."

I am using the latest Lux from the non-blocking-pty branch.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/hawk/lux/issues/22, or mute the thread https://github.com/notifications/unsubscribe-auth/AAJpsG3KPI85vNEk5iPG1Nnuse4YRIJkks5rmXR5gaJpZM4Mfpf0 .

hawk avatar Mar 16 '17 22:03 hawk

Yes, the issue is reproducible on the develop branch. I see the new diff feature added with 1.16, but other than that the output is the same. Lux's diff seems to think there is something different in code block 3, but when I diff the expected and the actual match_timeout myself, they are identical.

pillravi avatar Mar 17 '17 16:03 pillravi

Thanks.

How do I do to reproduce it? Do you have a executable test case?

/Håkan

On Mar 17, 2017 17:36, "pillravi" [email protected] wrote:

Yes, the issue is reproducible on the develop branch. I see the new diff feature added with 1.16, but other than that the output is the same. Lux's diff seems to think there is something different in code block 3, but when I diff the expected and the actual match_timeout myself, they are identical.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/hawk/lux/issues/22#issuecomment-287405461, or mute the thread https://github.com/notifications/unsubscribe-auth/AAJpsNJSNehQ6JkTrMfPMApdMCgQj-hbks5rmraFgaJpZM4Mfpf0 .

hawk avatar Mar 17 '17 16:03 hawk

Here is an example test case: failing lux test.zip

pillravi avatar Mar 23 '17 19:03 pillravi

Thanks for the test case. It reproduced the error.

It looks that there is an extra carriage return causing the problem. The runpty program (in lux) reads and writes chunks of data. The extra CR occurs when runpty reads a full buffer and its last char happens to be a CR. Then the first char in the next read will be a spurious CR. I have not figured out why that happens though.

/Håkan

On Thu, Mar 23, 2017 at 8:27 PM, pillravi [email protected] wrote:

Here is an example test case: failing lux test.zip https://github.com/hawk/lux/files/865928/failing.lux.test.zip

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/hawk/lux/issues/22#issuecomment-288834216, or mute the thread https://github.com/notifications/unsubscribe-auth/AAJpsK31zlU3kbgxeKoH_S2y-HUFWkEiks5rose8gaJpZM4Mfpf0 .

hawk avatar Mar 24 '17 14:03 hawk

It may work now. Can you re-run using lux on the develop-branch?

/Håkan

On Thu, Mar 23, 2017 at 8:27 PM, pillravi [email protected] wrote:

Here is an example test case: failing lux test.zip https://github.com/hawk/lux/files/865928/failing.lux.test.zip

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/hawk/lux/issues/22#issuecomment-288834216, or mute the thread https://github.com/notifications/unsubscribe-auth/AAJpsK31zlU3kbgxeKoH_S2y-HUFWkEiks5rose8gaJpZM4Mfpf0 .

hawk avatar Apr 04 '17 08:04 hawk