coreutils icon indicating copy to clipboard operation
coreutils copied to clipboard

tr: correctly handle multibyte octal sequences

Open andrewliebenow opened this issue 1 year ago • 6 comments

andrewliebenow avatar Oct 12 '24 05:10 andrewliebenow

Partly resolves the last issue listed in https://github.com/uutils/coreutils/issues/6777.

I'm not sure how to perform logging inside the nom parsing code without the messages being printed twice.

andrewliebenow avatar Oct 12 '24 05:10 andrewliebenow

GNU testsuite comparison:

GNU test failed: tests/timeout/timeout. tests/timeout/timeout is passing on 'main'. Maybe you have to rebase?

github-actions[bot] avatar Oct 12 '24 06:10 github-actions[bot]

First and last issues in #6777 are now addressed by this

andrewliebenow avatar Oct 12 '24 06:10 andrewliebenow

GNU testsuite comparison:

Skip an intermittent issue tests/tail/inotify-dir-recreate (fails in this run but passes in the 'main' branch)

github-actions[bot] avatar Oct 12 '24 09:10 github-actions[bot]

GNU testsuite comparison:

Skipping an intermittent issue tests/rm/rm1 (passes in this run but fails in the 'main' branch)
Congrats! The gnu test tests/timeout/timeout is no longer failing!

github-actions[bot] avatar Oct 18 '24 00:10 github-actions[bot]

GNU testsuite comparison:

GNU test failed: tests/timeout/timeout. tests/timeout/timeout is passing on 'main'. Maybe you have to rebase?
Congrats! The gnu test tests/tail/assert is no longer failing!

github-actions[bot] avatar Oct 20 '24 15:10 github-actions[bot]

Hi, I've played a bit with octal parsing as well, and I come to the same conclusion as you, which is that the logging of a warning is impossible to do because it is executed several times.

The solutions I see are the following:

  • Use some sort of global value to memoize the fact that this warning was already displayed (which is a terrible idea)
  • Introduce a lexer before the parser to handle escape sequences (which is nice, but is also a much bigger change)

What do you think ?

(my changes for reference)

RenjiSann avatar Oct 24 '24 13:10 RenjiSann

@RenjiSann The least-bad idea I could think of was changing the return type of all of these functions to allow returning a warning to print, but that would be kind of ugly considering that it's only this one path that would actually return one.

I should probably remove my commented-out code, so we can at least get the main fix merged in (albeit without the warning).

andrewliebenow avatar Oct 24 '24 17:10 andrewliebenow

@RenjiSann The least-bad idea I could think of was changing the return type of all of these functions to allow returning a warning to print, but that would be kind of ugly considering that it's only this one path that would actually return one.

I am not convinced this would efficiently solve the problem, it might be over-complicated to check for different warning messages that should both be displayed

I should probably remove my commented-out code, so we can at least get the main fix merged in (albeit without the warning).

Let's do this, and maybe open an issue for handling the warning correctly later

RenjiSann avatar Oct 24 '24 17:10 RenjiSann

GNU testsuite comparison:

Skip an intermittent issue tests/timeout/timeout (fails in this run but passes in the 'main' branch)

github-actions[bot] avatar Oct 30 '24 04:10 github-actions[bot]