posixutils-rs icon indicating copy to clipboard operation
posixutils-rs copied to clipboard

testing: show extra information on failure

Open andrewliebenow opened this issue 1 year ago • 2 comments

andrewliebenow avatar Oct 25 '24 22:10 andrewliebenow

Old:

---- tr::tr_octal_above_one_byte_value stdout ----
thread 'tr::tr_octal_above_one_byte_value' panicked at plib/src/testing.rs:85:5:
assertion `left == right` failed
  left: "(Ł)"
 right: "Ł)"
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

New:

---- tr::tr_octal_above_one_byte_value stdout ----
thread 'tr::tr_octal_above_one_byte_value' panicked at plib/src/testing.rs:229:9:
------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------
Test failed with 2 total failure types
--------------------------------------------------------------------------------

Command executed
--------------------------------------------------------------------------------
tr -d \501
--------------------------------------------------------------------------------

Failure 1 of 2: stdout differs from what was expected
--------------------------------------------------------------------------------

Actual
--------------------------------------------------------------------------------
(Ł)
--------------------------------------------------------------------------------

Expected
--------------------------------------------------------------------------------
Ł)
--------------------------------------------------------------------------------

Failure 2 of 2: stderr differs from what was expected
--------------------------------------------------------------------------------

Actual
--------------------------------------------------------------------------------

--------------------------------------------------------------------------------

Expected
--------------------------------------------------------------------------------
tr: warning: the ambiguous octal escape \501 is being interpreted as the 2-byte sequence \050, 1

--------------------------------------------------------------------------------


stderr, for diagnosing failure
--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------------

note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

andrewliebenow avatar Oct 25 '24 22:10 andrewliebenow

The new format might look a bit verbose, but I've repeatedly run up against:

  1. Not remembering what left/right correspond to (actual/expected)
  2. Not knowing if the comparison is in stdout or stderr
  3. Not having access to any additional information beyond that which is provided by the first failed assertion

andrewliebenow avatar Oct 25 '24 23:10 andrewliebenow