response: fix connect tunneling bug - v3

[cccs-sadugas]

The response was emitting partial body data depending on how you fed the parser with inbound and outbound data chunks. It seems the intended behavior is to not emit body data if HTP_STREAM_TUNNEL will eventually be entered (please correct if mistaken).

The fix was to allow htp_connp_REQ_CONNECT_WAIT_RESPONSE to resume in order to enter the HTP_STREAM_TUNNEL or complete the request.

The tunneling transaction was also incomplete because the request side wasn't being finalized after entering HTP_STREAM_TUNNEL.

See test case for example.

[catenacyber]

Thanks @cccs-sadugas

Would you have a redmine ticket for this ? Suricata-verify test ?

[catenacyber]

Running suricata on the pcap created by the test, I see this diff

{                                                                       {
  "event_type": "http",                                                   "event_type": "http",
  "tx_id": 0,                                                             "tx_id": 0,
  "http": {                                                               "http": {
    "hostname": "www.ssllabs.com",                                          "hostname": "www.ssllabs.com",
    "http_port": 443,                                                       "http_port": 443,
    "url": "www.ssllabs.com:443",                                           "url": "www.ssllabs.com:443",
    "http_method": "CONNECT",                                               "http_method": "CONNECT",
    "protocol": "HTTP/1.0",                                                 "protocol": "HTTP/1.0",
    "status": 200,                                                          "status": 200,
    "length": 0                                                 |           "length": 15
  }                                                                       }
}                                                                       }
{                                                                       {
  "event_type": "flow",                                                   "event_type": "flow",
  "src_ip": "127.0.0.1",                                                  "src_ip": "127.0.0.1",
  "src_port": 57207,                                                      "src_port": 57207,
  "dest_ip": "127.0.0.1",                                                 "dest_ip": "127.0.0.1",
  "dest_port": 8001,                                                      "dest_port": 8001,
  "proto": "TCP",                                                         "proto": "TCP",
  "app_proto": "http",                                          |         "app_proto": "failed",
                                                                >         "app_proto_orig": "http",

I am not sure the fix is correct, "app_proto": "failed", looks better to me...

As for the length, the PR looks to improve it...

So, I guess I need to dig into this app_proto diff

[catenacyber]

This delays the callback HTPCallbackResponseComplete

[simdugas]

Thanks for the initial feedback! I will try to create a redmine ticket, look at comments, and do more testing with Suricata in the next week or two.

[catenacyber]

Hello @simdugas will you work on this again ?

[simdugas]

Hello @simdugas will you work on this again ?

Yes, as soon as I find spare time to do so. Sorry for the delay!

[catenacyber]

@simdugas did you not do a CV PR with the PCAP you showed me last week ? Also as discussed do you have a pcap with SSH coming after the CONNECT ?

[catenacyber]

Suricata now uses libhtp rust, let me know if you have a test for this problem...