DTLS handshake started with invalid message sequence numbers

[pfg666]

Update

Check the comment below as the message sequence numbers may be generated by conforming clients in rare scenarios. This relates to both this and issue 4029.

Description

• Type: Bug
• Priority: Minor

Non-conformance Bug

OS
Linux, Ubuntu 20

build:
Version: 4.7.1r

Affected Versions DTLS 1.2

Related issues This issue is similar to the one I posted earlier, issue 4029. The cause may very well be the same.

Expected behavior According to the DTLS 1.2. RFC:

The first message each side transmits in each handshake always has message_seq = 0. Whenever each new message is generated, the message_seq value is incremented by one.

This means the server should only carry out the handshake if the first and second ClientHello messages have message_seq 0 and 1, respectively, assuming that cookie exchange is performed.

Actual Behavior Our testing shows that the WolfSSL server does not properly validate the message_seq field. It completes the cookie exchange and generates the ServerHello flight following a ClientHello message with invalid message_seq (2 instead of 1).

Steps to Reproduce I attached files necessary for reproduction (see reproduction.tar.gz) using DTLS-Fuzzer, a Java-based tool for testing DTLS libraries based on TLS-Attacker. Also included in the archive is a capture of the interaction, generated on my machine. DTLS-Fuzzer requires the JDK for Java 8. On Ubuntu, this can be installed by running: sudo apt-get install openjdk-8-jdk

Unpack the archive, cd to resulting folder and run bash reproduce.sh, while running an instance of Wireshark on the side. The reproduction script will:

• setup WolfSSL 4.7.1r and DTLS-Fuzzer;
• launch an WolfSSL server
• launch DTLS-Fuzzer, execute test.

If everything works as planned, Wireshark should show an interaction similar to that in the image below:

Notice that the second ClientHello prompts the server to generate a new ServerHello flight. Looking at the contents of this ClientHello, we see that its message_seq field is 2 instead of 1. Further testing reveals that message_seq can take an even higher number (e.g. 4) and the handshake will still proceed.

Thanks! reproduction.tar.gz

[pfg666]

I opened a similar issue for Scandium, which also exhibited this behavior. From the discussions I had there, it appears that, there are rare situations when a conforming client may generate ClientHello messages with increased message sequence numbers.

This could be a reason for the server to accept these messages . I am leaving this issue open so that a developer can look at it, and close it if this is intended behavior. The same applies for the related issue I posted.

[ejohnstown]

I believe this is not an issue at this point for the same reasons as for issues #4029 and #5224. Thank you for the bug report.