SSL certificate authentication: shutdown while in init

[gayoso]

Hi, I am developing both server and client for an application using ACE and OpenSSL. I am using the latest version of ACE, and OpenSSL 1.1.0f.

This code runs in the server, initializing the SSL Context:

ACE_SSL_Context *context = ACE_SSL_Context::instance();
context->set_mode(ACE_SSL_Context::SSLv23_server);
context->certificate("server_cert.pem", SSL_FILETYPE_PEM);
context->private_key("server_key.pem", SSL_FILETYPE_PEM);
if (context->load_trusted_ca("trusted.pem", 0, false) == -1) {
	ACE_ERROR_RETURN((LM_ERROR, "%p\n", "load_trusted_ca"), -1);
}

context->set_verify_peer(1, 1, 0);

if (context->have_trusted_ca() <= 0) {
	ACE_ERROR_RETURN((LM_ERROR, "%p\n", "have_trusted_ca"), -1);
}

This code runs in the client, initializing the SSL Context (so far, server certificate is not authenticated yet):

ACE_SSL_Context *context = ACE_SSL_Context::instance();
context->set_mode(ACE_SSL_Context::SSLv23_client);
context->certificate("C:/adq/client_cert.pem", SSL_FILETYPE_PEM);
context->private_key("C:/adq/client_key.pem", SSL_FILETYPE_PEM);

Client authentication works, and if I change the client certificates the connection is refused. However, the client starts throwing SSL errors after the first connection. The error shown is: "ACE_SSL error code: 336462231 - error:140E0197:SSL routines:SSL_shutdown:shutdown while in init".

Client side, there is an ACE_SSL_SOCK_Stream member that is reused by creating a connection, and then closing it, then creating a connection again, etc. This is the only thing that I imagine could be causing the issue. However, if I remove the line "context->set_verify_peer(1, 1, 0);" from the server program, the client works properly. Is the SSL_SOCK_Stream not reusable when authentication is enabled?

Anyways, a mutual authentication example would be appreciated since ACE_SSL offers a wrapper for mutual authentication but most SSL examples are quite simple.

Let me know if there is any more info I should provide.

[jwillemsen]

Maybe the error is given due to an hange in OpenSSL (see https://github.com/openssl/openssl/issues/710), can you analyze this further and create a PR where the code checks SSL_in_init() before invoking SSL_shutdown()

[gayoso]

I will see if i can do that in a couple days.

For now, a working fix seems to be not reusing the ACE_SSL_SOCK_Stream object. Insead, I call connect and close when done, and then create a new ACE_SSL_SOCK_Stream for a new connect.

[gayoso]

Coming back to this years later, the problem seems to be resolved by adding a call to SSL_CTX_set_session_id_context server side:

ACE_SSL_Context *context = ACE_SSL_Context::instance();
srand(time(NULL));
int session_id = rand();
if (SSL_CTX_set_session_id_context(context->context(), (const unsigned char*)&session_id, sizeof(session_id)) != 1) {
	return -1;
}

https://www.openssl.org/docs/man1.0.2/man3/SSL_CTX_set_session_id_context.html

"If the session id context is not set on an SSL/TLS server and client certificates are used, stored sessions will not be reused but a fatal error will be flagged and the handshake will fail."

After this change the ACE_SSL_SOCK_Stream object can be reused for new connections. If you find this is correct feel free to close this issue.