bc-java icon indicating copy to clipboard operation
bc-java copied to clipboard

Regression in Bouncy Castle bctls (1.74+) with Java 6 for TLS 1.2 GCM Cipher Suites via JSSE

Open KNB52 opened this issue 7 months ago • 4 comments

We are currently utilizing Bouncy Castle bctls-jdk15on and bcprov-jdk15on version 1.59 with Java 6. Our current project requires us to upgrade our system to accommodate a communication partner who is switching to RSA/PSS certificates. To address this, we attempted to upgrade Bouncy Castle to bctls-jdk15to18, bcprov-jdk15to18, and bcutil-jdk15to18 version 1.80.

However, after the upgrade, we consistently encounter a "cannot reuse nonce for GCM encryption" exception. This exception originates from the JceAEADCipherImpl's private cipher variable, which encapsulates bcprov's GCMBlockCipher. This issue arises when the negotiated cipher suite is TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256.

For context, we are using this via SSLSocketFactory as a JSSE provider.

While we acknowledge that Java 6 usage is a diminishing demographic, we believe there's still a significant segment of Bouncy Castle users who rely on it for external services, especially given the recent push towards TLS 1.2 enforcement.

We suspect the core problem lies in JceAEADCipherImpl#doFinal(byte[], int, int, byte[], int). It appears to be calling init on its private cipher variable without resetting the nonce. We find it questionable to call cipher.init before cipher.doFinal in the first place. If the cipher instance is intended for reuse, init should ideally be invoked after cipher.doFinal and only after the nonce has been properly re-initialized. Alternatively, if the instance is not meant for reuse, init shouldn't be called at all in this context. Fundamentally, updateAAD (for Java versions that support it) and cipher.init (for Java versions where updateAAD isn't supported in the same manner) serve distinct purposes and, in our view, should not be called concurrently or in the same execution path.

We have examined the source code for Bouncy Castle versions ranging from 1.73 to 1.80. Our analysis indicates that versions 1.74 and later consistently fail to operate correctly on Java 6. This represents a clear regression, as the functionality that worked reliably in version 1.73 is now broken from 1.74 onwards.

As a temporary workaround, we are currently reverting to version 1.73, which allows us to maintain communication. However, we earnestly request a fundamental and permanent resolution in a future release.

※I'm not from an English-speaking country, so I had Gemini translate it for me. I hope you understand.

KNB52 avatar Jun 06 '25 06:06 KNB52

Thanks for reporting this issue. I can see how the problem arises from a double-init and it should be fixable. The code is quite constrained though e.g. in TLS 1.3, the ciphertext length is included in the additional data (AAD) for the AES/GCM cipher. However calling Cipher.getOutputSize to calculate the ciphertext length needs Cipher.init to be called first, and pre-1.7 AAD can only be passed via init (because no updateAAD).

peterdettman avatar Jun 10 '25 07:06 peterdettman

Apologies for the delayed response.

We are, of course, not keen on continuing to use version 1.73 due to its known security vulnerabilities. Therefore, we would greatly appreciate a fix in a near-future version if at all possible.

KNB52 avatar Jun 16 '25 02:06 KNB52

This is now fixed. When updateAAD is not available we use Cipher.init twice; first with a dummy nonce (this call allows getOutputSize to then work correctly), then with the true nonce and the additional data for the doFinal call.

peterdettman avatar Aug 05 '25 05:08 peterdettman

My apologies for the delayed response. I understand that it will be included in the next version, 1.82. I'll give it a try once it's launched.

Thank you very much.

KNB52 avatar Sep 03 '25 05:09 KNB52