Signature verification fails using an OpenSSL-generated ML-DSA-87 Private Key with BouncyCastle1.8.0

[feuxfollets1013]

Hello developers,
I encountered an issue where signature verification fails when trying to create a certificate using the ML-DSA-87 . I guess the following issues and discussions might be related, but I could not find a solution:

• https://github.com/bcgit/bc-java/issues/1936
• https://github.com/openssl/openssl/issues/26652

I'm wondering if you could tell me how to resolve this issue?

---

Observed Behavior

Signing a CSR generated by BouncyCastle 1.8.0 using an ML-DSA-87 private key generated by OpenSSL 3.5.0 fails signature verification.

Comparing the private keys generated by OpenSSL and BouncyCastle reveals the following differences:

• Private Key Generated by OpenSSL 3.5.0:

  • privateKey.getFormat(): PKCS#8
  • privateKey.getEncoded().length: 7517
  • privateKey.getAlgorithm(): ML-DSA-87
  • Seed inside privateKey instance: null

• Private Key Generated by BouncyCastle 1.8.0:

  • privateKey.getFormat(): PKCS#8
  • privateKey.getEncoded().length: 52
  • privateKey.getAlgorithm(): ML-DSA-87
  • Seed inside privateKey instance: Randomly generated value

---

Versions

• BouncyCastle: 1.8.0
• OpenSSL: 3.5.0
• Java: OpenJDK 21

---

Steps to Reproduce

1. Generate an Encrypted CA Private Key Using OpenSSL 3.5.0
   Use the following command to generate an encrypted private key. The generated private key is attached to this issue.

   openssl req -x509 -new -newkey "ML-DSA-87" -keyout ca.key -out ca.crt -subj "/CN=TestCA" -days "365"
   

2. Load the Private Key Using readSignerSecretKey
   Load the private key using the readSignerSecretKey method:

   CertificateGenerator generator = new CertificateGenerator("ML-DSA-87", parameterSpec);
   generator.readSignerSecretKey("ca.key", "test");
   

3. Check the Loaded Private Key's Properties
   Print the properties of the loaded private key:

   
   System.out.println("Format(OpenSSL): " + keyPair.getPrivate().getFormat());
   System.out.println("Encoded Length(OpenSSL): " + keyPair.getPrivate().getEncoded().length);
   System.out.println("Algorithm(OpenSSL): " + keyPair.getPrivate().getAlgorithm());
   

4. Generate a Root Certificate and Check the Generated Private Key's Properties
   Generate a root certificate and print the properties of the generated private key:

   System.out.println("Format(BC): " + keyPair.getPrivate().getFormat());
   System.out.println("Encoded Length(BC): " + keyPair.getPrivate().getEncoded().length);
   System.out.println("Algorithm(BC): " + keyPair.getPrivate().getAlgorithm());
   

5. Verify the Certificate in generateCertificateByBC
   The verification succeeds in this method.

6. Verify the Certificate in generateCertificateByCAFile
   The verification fails in this method.

---

Test Code

Below is the test code used to reproduce the issue:

import java.io.FileReader;
import java.math.BigInteger;
import java.security.KeyPair;
import java.security.KeyPairGenerator;
import java.security.PrivateKey;
import java.security.SecureRandom;
import java.security.Security;
import java.security.SignatureException;
import java.security.cert.X509Certificate;
import java.security.spec.AlgorithmParameterSpec;
import java.util.Date;

import javax.security.auth.x500.X500Principal;

import org.bouncycastle.asn1.x500.X500Name;
import org.bouncycastle.asn1.x509.BasicConstraints;
import org.bouncycastle.asn1.x509.Extension;
import org.bouncycastle.cert.X509CertificateHolder;
import org.bouncycastle.cert.X509v3CertificateBuilder;
import org.bouncycastle.cert.jcajce.JcaX509CertificateConverter;
import org.bouncycastle.cert.jcajce.JcaX509ExtensionUtils;
import org.bouncycastle.cert.jcajce.JcaX509v3CertificateBuilder;
import org.bouncycastle.jce.provider.BouncyCastleProvider;
import org.bouncycastle.openssl.PEMParser;
import org.bouncycastle.openssl.jcajce.JcaPEMKeyConverter;
import org.bouncycastle.openssl.jcajce.JceOpenSSLPKCS8DecryptorProviderBuilder;
import org.bouncycastle.operator.ContentSigner;
import org.bouncycastle.operator.InputDecryptorProvider;
import org.bouncycastle.operator.jcajce.JcaContentSignerBuilder;
import org.bouncycastle.operator.jcajce.JcaContentVerifierProviderBuilder;
import org.bouncycastle.pkcs.PKCS10CertificationRequest;
import org.bouncycastle.pkcs.PKCS10CertificationRequestBuilder;
import org.bouncycastle.pkcs.PKCS8EncryptedPrivateKeyInfo;
import org.bouncycastle.pkcs.jcajce.JcaPKCS10CertificationRequestBuilder;

public class CertificateGenerator {
    private KeyPairGenerator keyPairGenerator;
    private KeyPair keyPair;
    private X509Certificate signerCert;
    private PrivateKey signerPrivateKey;
    private static final String BC_PROVIDER = "BC";

    static {
        Security.addProvider(new BouncyCastleProvider());
    }

    public static void main() {
        try {
            final String ALGO = "ML-DSA-87";
            final String DN = "CN=ML-DSA.example.com";
            final String CA_PATH = "ca.key";
            final String CA_PASSWORD = "test";
            CertificateGenerator certGen = new CertificateGenerator(ALGO, MLDSAParameterSpec.ml_dsa_87);
            certGen.generateCertificateByBC(DN, CA_PATH, CA_PASSWORD);
            certGen.generateCertificateByCAFile(DN, CA_PATH, CA_PASSWORD);
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    public CertificateGenerator(String keyGenAlgorithm, AlgorithmParameterSpec parameterSpec) throws Exception {
        this.keyPairGenerator = KeyPairGenerator.getInstance(keyGenAlgorithm, BC_PROVIDER);
        this.keyPairGenerator.initialize(parameterSpec);
    }

    private KeyPair generateKeyPair() {
        this.keyPair = this.keyPairGenerator.generateKeyPair();
        return this.keyPair;
    }

    private PKCS10CertificationRequest generateCSR(String dn, String signingAlgorithm) throws Exception {
        this.generateKeyPair();
        PKCS10CertificationRequestBuilder pkcs10Builder = new JcaPKCS10CertificationRequestBuilder(
                new X500Principal(dn), this.keyPair.getPublic());
        JcaContentSignerBuilder signerBuilder = new JcaContentSignerBuilder(signingAlgorithm).setProvider(BC_PROVIDER);
        ContentSigner signer = signerBuilder.build(this.keyPair.getPrivate());
        PKCS10CertificationRequest csr = pkcs10Builder.build(signer);
        boolean isValid = csr
                .isSignatureValid(new JcaContentVerifierProviderBuilder().build(csr.getSubjectPublicKeyInfo()));
        if (isValid) {
            return csr;
        } else {
            throw new SignatureException("CSR signature is invalid.");
        }
    }

    private void generateRootCertificate() throws Exception {
        KeyPair keyPair = this.keyPairGenerator.generateKeyPair();
        BigInteger rootSerialNum = new BigInteger(Long.toString(new SecureRandom().nextLong()));

        X500Name rootCertIssuer = new X500Name("CN=root-cert");
        X500Name rootCertSubject = rootCertIssuer;
        ContentSigner rootCertContentSigner = new JcaContentSignerBuilder("ML-DSA-87").setProvider(BC_PROVIDER)
                .build(keyPair.getPrivate());
        X509v3CertificateBuilder rootCertBuilder = new JcaX509v3CertificateBuilder(rootCertIssuer, rootSerialNum,
                new Date(System.currentTimeMillis()),
                new Date(System.currentTimeMillis() + 10L * 365 * 24 * 60 * 60 * 1000), rootCertSubject,
                keyPair.getPublic());

        // Add Extensions
        // A BasicConstraint to mark root certificate as CA certificate
        JcaX509ExtensionUtils rootCertExtUtils = new JcaX509ExtensionUtils();
        rootCertBuilder.addExtension(Extension.basicConstraints, true, new BasicConstraints(true));
        rootCertBuilder.addExtension(Extension.subjectKeyIdentifier, false,
                rootCertExtUtils.createSubjectKeyIdentifier(keyPair.getPublic()));

        X509CertificateHolder rootCertHolder = rootCertBuilder.build(rootCertContentSigner);
        X509Certificate rootCert = new JcaX509CertificateConverter().setProvider(BC_PROVIDER)
                .getCertificate(rootCertHolder);
        this.signerPrivateKey = keyPair.getPrivate();
        this.signerCert = rootCert;
        rootCert.verify(rootCert.getPublicKey());

        System.out.println("Format(BC): " + keyPair.getPrivate().getFormat());
        System.out.println("Encoded Length(BC): " + keyPair.getPrivate().getEncoded().length);
        System.out.println("Algorithm(BC): " + keyPair.getPrivate().getAlgorithm());
    }

    public X509Certificate generateCertificateByCAFile(String dn, String caP12PEMPath, String caPrivateKeyPassword)
            throws Exception {
        this.readSignerSecretKey(caP12PEMPath, caPrivateKeyPassword);
        this.generateRootCertificate();
        PKCS10CertificationRequest csr = this.generateCSR(dn, this.signerPrivateKey.getAlgorithm());

        X509v3CertificateBuilder certBuilder = new X509v3CertificateBuilder(
                new X500Name(this.signerCert.getSubjectX500Principal().getName()),
                new BigInteger(Long.toString(new SecureRandom().nextLong())), new Date(System.currentTimeMillis()),
                new Date(System.currentTimeMillis() + 10L * 365 * 24 * 60 * 60 * 1000), csr.getSubject(),
                csr.getSubjectPublicKeyInfo());

        JcaContentSignerBuilder signerBuilder = new JcaContentSignerBuilder(this.signerPrivateKey.getAlgorithm())
                .setProvider(BC_PROVIDER);
        ContentSigner signer = signerBuilder.build(this.signerPrivateKey);
        X509CertificateHolder certHolder = certBuilder.build(signer);
        X509Certificate cert = new JcaX509CertificateConverter().setProvider(BC_PROVIDER).getCertificate(certHolder);

        cert.verify(this.signerCert.getPublicKey(), "BC"); // verify error
        return cert;
    }

    public X509Certificate generateCertificateByBC(String dn, String caP12PEMPath, String caPrivateKeyPassword)
            throws Exception {
        this.generateRootCertificate();
        PKCS10CertificationRequest csr = this.generateCSR(dn, this.signerPrivateKey.getAlgorithm());

        X509v3CertificateBuilder certBuilder = new X509v3CertificateBuilder(
                new X500Name(this.signerCert.getSubjectX500Principal().getName()),
                new BigInteger(Long.toString(new SecureRandom().nextLong())), new Date(System.currentTimeMillis()),
                new Date(System.currentTimeMillis() + 10L * 365 * 24 * 60 * 60 * 1000), csr.getSubject(),
                csr.getSubjectPublicKeyInfo());

        JcaX509ExtensionUtils certExtUtils = new JcaX509ExtensionUtils();
        certBuilder.addExtension(Extension.basicConstraints, true, new BasicConstraints(false));
        certBuilder.addExtension(Extension.subjectKeyIdentifier, false,
                certExtUtils.createSubjectKeyIdentifier(csr.getSubjectPublicKeyInfo()));
        certBuilder.addExtension(Extension.authorityKeyIdentifier, false,
                certExtUtils.createAuthorityKeyIdentifier(signerCert));

        JcaContentSignerBuilder signerBuilder = new JcaContentSignerBuilder(this.signerPrivateKey.getAlgorithm())
                .setProvider(BC_PROVIDER);
        ContentSigner signer = signerBuilder.build(this.signerPrivateKey);
        X509CertificateHolder certHolder = certBuilder.build(signer);
        X509Certificate cert = new JcaX509CertificateConverter().setProvider(BC_PROVIDER).getCertificate(certHolder);

        cert.verify(this.signerCert.getPublicKey(), "BC"); // verify ok
        return cert;
    }

    private void readSignerSecretKey(String filePath, String keyPassword) throws Exception {
        PEMParser pemParser = new PEMParser(new FileReader(filePath));

        PKCS8EncryptedPrivateKeyInfo privateKeyInfo = (PKCS8EncryptedPrivateKeyInfo) pemParser.readObject();
        pemParser.close();
        InputDecryptorProvider decryptorProvider = new JceOpenSSLPKCS8DecryptorProviderBuilder()
                .build(keyPassword.toCharArray());
        this.signerPrivateKey = new JcaPEMKeyConverter()
                .getPrivateKey(privateKeyInfo.decryptPrivateKeyInfo(decryptorProvider));
        System.out.println("Format(OpenSSL): " + keyPair.getPrivate().getFormat());
        System.out.println("Encoded Length(OpenSSL): " + keyPair.getPrivate().getEncoded().length);
        System.out.println("Algorithm(OpenSSL): " + keyPair.getPrivate().getAlgorithm());
    }
}

---

test-ca-cert.zip

[dghgit]

Try the beta at https://downloads.bouncycastle.org/betas

[feuxfollets1013]

@dghgit I appeciate your prompt reply. I ran it using v.1.8.1-beta and it worked.

[feuxfollets1013]

Hi, @dghgit Does this problem also occur in Dilithium5? I tried this same issue with Dilithium5 and it failed. So for analysis, I added the following test code to CrystalDilithiumTest.java and ran it. Then, in the signVerifyInternal method of DilithiumEngine.java, it appears that signature verification is failing due to a mismatch between siglen and CryptoBytes. What is the expected siglen, I understand that the signature size for Dilithium5 is 4595bytes, but the CryptoBytes size expected here is 4627bytes.

	public void testOpenSSLSignAndBcVerify() {
		try {
			byte[] publicKey = readPublicKeyFromPEMFile("src/test/resources/qsc.pub");
			// openssl dgst -sha256 -sign qsc.key -out qsc.sign message
			FileInputStream fis = new FileInputStream("src/test/resources/qsc.sign");
			byte[] sign = fis.readAllBytes();
			fis.close();
			FileInputStream fis2 = new FileInputStream("src/test/resources/message");
			byte[] msg = fis2.readAllBytes();
			fis2.close();

			MessageDigest md = MessageDigest.getInstance("SHA-256");
			byte[] digest = md.digest(msg);
			DilithiumPublicKeyParameters pkParameters = new DilithiumPublicKeyParameters(DilithiumParameters.dilithium5,
					publicKey);
			DilithiumSigner verifier = new DilithiumSigner();
			verifier.init(false, pkParameters);
			boolean result = verifier.verifySignature(digest, sign);

			if (result) {
				assertTrue(true);
			} else {
				fail();
			}
		} catch (Exception e) {
			fail();
		}
	}

	private byte[] readPublicKeyFromPEMFile(String filePath)
			throws IOException, NoSuchAlgorithmException, InvalidKeySpecException, NoSuchProviderException {
		StringBuilder keyBuilder = new StringBuilder();
		try (BufferedReader reader = new BufferedReader(new FileReader(filePath))) {
			String line;
			while ((line = reader.readLine()) != null) {
				// Skip PEM headers and footers
				if (line.startsWith("-----BEGIN") || line.startsWith("-----END")) {
					continue;
				}
				keyBuilder.append(line.trim());
			}
		}
		// Decode Base64 content
		byte[] subjectPublicKeyInfo = Base64.getDecoder().decode(keyBuilder.toString());
		byte[] rawPublicKey = Arrays.copyOfRange(subjectPublicKeyInfo, 24, subjectPublicKeyInfo.length);
		return rawPublicKey;
	}

Versions

• BouncyCastle: 1.8.0
• Java: OpenJDK 21
• OpenSSL: 3.5.0
• oqs-provider: 0.8.0
• liboqs: 0.12.0

Test Resources

test-resources.zip

[dghgit]

It's definitely 4627, although you may be looking at an older version of Dilithium. We'll be deprecating Dilithium out shortly, I imagine everyone else will as well, for this I would just stick to ML-DSA and it's 3 parameter sets from here on in.