Invalid hashcat rules

[n0kovo]

When running hashcat using the rules generated with the --hashcat option, a lot of them aren't processed as they are invalid. For me, the tool output 5658 rules, 1557 of which were invalid.

Truncated hashcat output:

Skipping invalid or unsupported rule in file [REDACTED] on line 1: E?w
Skipping invalid or unsupported rule in file [REDACTED] on line 2: E?w$1
Skipping invalid or unsupported rule in file [REDACTED] on line 4: E?w$2
Skipping invalid or unsupported rule in file [REDACTED] on line 5: E?w$1$2$3
Skipping invalid or unsupported rule in file [REDACTED] on line 6: om1
Skipping invalid or unsupported rule in file [REDACTED] on line 8: E?p$1$2
Skipping invalid or unsupported rule in file [REDACTED] on line 10: E?w$7
Skipping invalid or unsupported rule in file [REDACTED] on line 13: E?p$3
Skipping invalid or unsupported rule in file [REDACTED] on line 14: E?p$5
Skipping invalid or unsupported rule in file [REDACTED] on line 15: E?w$1$1
Skipping invalid or unsupported rule in file [REDACTED] on line 17: E?p$8
Skipping invalid or unsupported rule in file [REDACTED] on line 18: E?p$4
Skipping invalid or unsupported rule in file [REDACTED] on line 19: E?w$1$3
Skipping invalid or unsupported rule in file [REDACTED] on line 20: E?w$1$0
Skipping invalid or unsupported rule in file [REDACTED] on line 21: E?w$0$1
Skipping invalid or unsupported rule in file [REDACTED] on line 22: E?p$6
Skipping invalid or unsupported rule in file [REDACTED] on line 23: E?w$2$2
Skipping invalid or unsupported rule in file [REDACTED] on line 24: E?w$0
Skipping invalid or unsupported rule in file [REDACTED] on line 25: E?p$\x21
Skipping invalid or unsupported rule in file [REDACTED] on line 28: E?w$2$1
Skipping invalid or unsupported rule in file [REDACTED] on line 29: E?p$6$9
Skipping invalid or unsupported rule in file [REDACTED] on line 32: E?p$1$9
Skipping invalid or unsupported rule in file [REDACTED] on line 34: E?p$1$4
Skipping invalid or unsupported rule in file [REDACTED] on line 36: E?w$0$9
Skipping invalid or unsupported rule in file [REDACTED] on line 37: E?p$7$7
Skipping invalid or unsupported rule in file [REDACTED] on line 40: om2
Skipping invalid or unsupported rule in file [REDACTED] on line 44: E?w$1$6
Skipping invalid or unsupported rule in file [REDACTED] on line 45: E?p$2$4

[SimonMarechal]

So I did not reply anything but I started working on this a while ago, it's almost done :)

[n0kovo]

How's it coming along? Very much looking forward to testing it out!

[n0kovo]

@SimonMarechal BUMP

[SimonMarechal]

Should be good now, do you still have issues?

[superevr]

It still generates a lot of invalid rules for hashcat.

Or I get this error message:

thread 'main' panicked at 'should not happen : invalid rule to be displayed: [Command(Swap(Val(0), Val(0))), Command(InsertString(Val(0), [35, 51, 38, 52, 37, 35, 33]))]', src/main.rs:226:25

[SimonMarechal]

@superevr I'll take a look into the panic, but do you mind sharing an invalid rule?

[SimonMarechal]

Also at first glance it looks like this shouldn't panic that way if executed un hashcat mode, do you mind sharing how I can reproduce this?

[superevr]

Sure.

I'm running this in MacOS, not sure if it would make a difference. And I'll note that I did get these errors when compiling via cargo install --git https://github.com/synacktiv/rulesfinder:

warning: variant `Hashcat` is never constructed
 --> src/rules.rs:6:5
  |
4 | enum ToolSupport {
  |      ----------- variant in this enum
5 |     JtR,
6 |     Hashcat,
  |     ^^^^^^^
  |
  = note: `ToolSupport` has derived impls for the traits `Clone` and `Debug`, but these are intentionally ignored during dead code analysis
  = note: `#[warn(dead_code)]` on by default

warning: function `john_rule` is never used
   --> src/rules.rs:993:8
    |
993 | pub fn john_rule(r: &Rule) -> bool {
    |        ^^^^^^^^^

warning: `rulesfinder` (bin "dumper") generated 2 warnings

For the input cleartext, I'll use the Hashmob mini list that can be downloaded at https://hashmob.net/resources/hashmob

I'll generate a wordlist from the Hashmob mini list using maskcat:

cat hashmob.net_2023-08-13.mini.found | maskcat tokens 99 | sort -u >! hashmob.mini.tokens

Then I'll run rulesfinder (this time with backtracking enabled:

RUST_BACKTRACE=full rulesfinder -w hashmob.mini.tokens --cleartexts hashmob.net_2023-08-13.mini.found -n 50 -t 7 --minsize 6 --hashcat | tee hashmob.mini.rule                                                                                    [11:58 AM]
[ETA: 00:00:00] ████████████████████████████████████████████████████████████ 56997/56997 - 350902 fragments inserted
[ETA: 00:00:00] ████████████████████████████████████████████████████████████ 2414/2414 - 6820 rules retained
thread 'main' panicked at 'should not happen : invalid rule to be displayed: [Command(Swap(Val(0), Val(0))), Command(Append(49))]', src/main.rs:226:25
stack backtrace:
   0:        0x104bba5a0 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::hf56058dac04a8100
   1:        0x104be40e8 - core::fmt::write::h887ee594c50d2f6b
   2:        0x104bc7688 - std::io::Write::write_fmt::h8e068fbe7ba944b7
   3:        0x104bba3b0 - std::sys_common::backtrace::print::hefee1a8be582057a
   4:        0x104bc4fc4 - std::panicking::default_hook::{{closure}}::h7ba479390a999ae5
   5:        0x104bc4c8c - std::panicking::default_hook::hc4ff20421fd3aa8b
   6:        0x104bc55d4 - std::panicking::rust_panic_with_hook::h97e5266e8ce2f24f
   7:        0x104bba88c - std::panicking::begin_panic_handler::{{closure}}::h5c38d4c71a65b53e
   8:        0x104bba698 - std::sys_common::backtrace::__rust_end_short_backtrace::h9c79ccffe3575672
   9:        0x104bc51f0 - _rust_begin_unwind
  10:        0x104bf8cd0 - core::panicking::panic_fmt::h8d86c61b68da2636
  11:        0x104b0ec18 - rulesfinder::main::h203ae0abfc802bba
  12:        0x104b16698 - std::sys_common::backtrace::__rust_begin_short_backtrace::h26cf9747c13607db
  13:        0x104b16868 - std::rt::lang_start::{{closure}}::hd84e0dc5df520cee
  14:        0x104bc5114 - std::panicking::try::hd27950ade386a4c1
  15:        0x104bcbf00 - std::rt::lang_start_internal::h4c832f79c4cb8516
  16:        0x104b0effc - _main

There is no panic when not using --hashcat

[superevr]

When it's actually able to generate a rule in hashcat format, it returns stuff like this:

cAz"1!"
c$1
cAz"12"
cAz"123"
cAz"#1"
cAz"01"
*00$1

I don't think "A" is a valid rule. The z rule is not formatted right the string should be in the format $1 $2 $3

[SimonMarechal]

Everything looks really wrong! Lemme check that.

[SimonMarechal]

It should be much better now. Do you mind testing it out?

[superevr]

much better! Here's a sample of rules it generated this time, which ran without error.

*00$1
*00$1$2$3
*00$1$2
*00$1$2$3$4
*00$1$1
*00$2
*00$0$1
*00$1$0
*00$1$3
*00$2$2
*00$1$2$3$4$5
*00$9$9
*00$2$3
*00$7
*00$2$1
*00$3
sA4$5
*00$1$4

There were still a lot of *00 rules, which means "Swap character at position 0 with character at position 0, which won't do anything.

[SimonMarechal]

Well spotted, I filtered those out, and also y0 and Y0.