SSL verification always fails using host_name_verification

[HaraldurSam]

If I use the host_name_verification callback function to verify the ssl certificate it always return false, due to logical error in the code. Let my try to explain why?

I setup the ssl verification process like this, following examples given the boost:asio documents.

asio::ssl::context ctx(asio::ssl::context::sslv23);
ctx.set_default_verify_paths();

// Verify the remote server's certificate
ctx.set_verify_mode(asio::ssl::verify_peer);
ctx.set_verify_callback(asio::ssl::host_name_verification(host.c_str()));

Then lets look at the host_name_verification code.

bool host_name_verification::operator()(bool preverified, verify_context& ctx) const
{
  using namespace std; // For memcmp.

 // Don't bother looking at certificates that have failed pre-verification.
 if (!preverified)
    return false;

 ...

This function is called from the context.

int context::verify_callback_function(int preverified, X509_STORE_CTX* ctx)
{
  if (ctx)
  {
    if (SSL* ssl = static_cast<SSL*>(::X509_STORE_CTX_get_ex_data(ctx, ::SSL_get_ex_data_X509_STORE_CTX_idx())))
    {
      if (SSL_CTX* handle = ::SSL_get_SSL_CTX(ssl))
      {
        if (SSL_CTX_get_app_data(handle))
        {
          detail::verify_callback_base* callback = static_cast<detail::verify_callback_base*>(SSL_CTX_get_app_data(handle));

          verify_context verify_ctx(ctx);
          return callback->call(preverified != 0, verify_ctx) ? 1 : 0;
        }
      }
    }
  }

  return 0;
}

And the verify_callback_function is called by openssl function in the class x509_vfy.c

static int verify_cb_cert(X509_STORE_CTX *ctx, X509 *x, int depth, int err)
{
    ctx->error_depth = depth;
    ctx->current_cert = (x != NULL) ? x : sk_X509_value(ctx->chain, depth);
    if (err != X509_V_OK)
        ctx->error = err;
    return ctx->verify_cb(0, ctx);
} 

As one can see calling the verify_callback_function with preverified = 0 as the openSsl function does, causes the verify_callback_function function to call the callback function with preverified = false, which causes the host_name_verification to always return false.

I don't know what I am missing but this logic always fails the verification of the ssl certificate.

I workaround is to intercept the callback function and call the host_name_verification with preverified = true and then the verification process is done.

Workaround:

ctx.set_verify_callback(make_verbose_verification(asio::ssl::host_name_verification(host.c_str())));


///@brief Auxiliary function to make verbose_verification objects.
template <typename Verifier> verbose_verification<Verifier> make_verbose_verification(Verifier verifier)
{
	return verbose_verification<Verifier>(verifier);
}


template <typename Verifier> class verbose_verification
{
public:
	verbose_verification(Verifier verifier) 
		: myVerifier(verifier) 
	{
	}

	bool operator()(bool preverify_ok, asio::ssl::verify_context& context)
	{
		return myVerifier(true, context);
		return verified;
	}

private:
	Verifier myVerifier;
};

Regards, Haraldur

[brjoha]

I came across this because I'm attempting to switch from rfc2818_verification() to host_name_verification(), but finding that my unit tests pass with the former but not the latter.

For a minute I thought this was a bug like you outlined, but I found that rfc2818_verification() is implemented the same way.

Only explanation is that OpenSSL invokes the callback in more than one location and passing a a non-zero value in that case.

[neongreen-sc]

I spent a lot of time debugging this, but in the end it turned out that one cert in the chain was actually invalid, and I simply didn't see the error message because either asio or websocketpp ate it :(

[DarthGandalf]

@HaraldurSam the workaround you proposed simply disables verification of the certificate, opening the door for MITM attack. It still checks CN, but even self-signed certificate with that CN will be accepted.

For me preverified was 0 because I didn't enable SNI, so the server was returning invalid certificate.

So the fix is this:

asio::ssl::stream<asio::ip::tcp::socket> sock(...);
if (!SSL_set_tlsext_host_name(sock.native_handle(), "hostname.com")) {
    // handle error
}
sock.async_handshake(...);

With SNI configured, server sends valid certificate, and asio::ssl::host_name_verification works fine.