Feature flag to automatically use webpki-roots?

[joshtriplett]

rustls requires using a matching version of webpki-roots that uses the same webpki types. Numerous library crates that build on rustls often have feature flags to enable one or both of rustls-native-certs or webpki-roots, and then do the same code sequence to add those roots to the rustls client config. Users of those library crates in turn need to request such features, and then enable them in the library crate.

Would it potentially be reasonable to add a feature flag to rustls, which when set, causes rustls to automatically add webpki-roots to the client config? That would allow numerous library crates building on rustls to drop their code doing so, and prevent many more such library crates from needing to add their own options for that. Instead, users of such library crates can just enable the appropriate feature flag in rustls to get the behavior they want.

[djc]

For webpki, webpki-roots and rustls-native certs, since they are already semver-locked together, I think it would be useful to add re-exports of those crates based no a configuration. We could perhaps add methods to the new config builders that then leverages this support directly, but I don't think we should set the root cert store implicitly based on a feature flags, since this tends to have surprising effects due to the additivity constraints of Cargo features (as in https://github.com/rustls/hyper-rustls/pull/134).

[briansmith]

For webpki, webpki-roots and rustls-native certs, since they are already semver-locked together, I think it would be useful to add re-exports of those crates based no a configuration.

I'd rather not encourage the use of webpki-roots or rustls-native-certs as they are now. I have tried to use them on some projects and I'm working on a detailed write-up about why (IMO) we should be doing something else. Short version: webpki-roots contains too many CAs for applications that want to be conservative and avoid the system root store, but also most applications probably should be integrating with the operating system's root store. The way rustls-native-certs works on non-Linux platforms isn't what applications need. Note that the the main branch of rustls contains changes to the certificate validation traits to improve this situation.

[joshtriplett]

In practice, just about every crate in the ecosystem is already doing that: either using webpki-roots or rustls-native-certs or offering a choice between the two.

[djc]

@briansmith until there is a viable alternative (that is not an upgrade to webpki-roots or rustls-native-certs), it's not clear to me that it makes sense to make these options harder to use than necessary. Do you have reasons to assume the issues with webpki-roots and rustls-native-certs cannot be fixed in the respective crates?

[briansmith]

Do you have reasons to assume the issues with webpki-roots and rustls-native-certs cannot be fixed in the respective crates?

Yes. Give me a couple of weeks and I'll share why in detail.

[jsha]

I wrote up my understanding of some of the issues here: https://github.com/rustls/rustls-native-certs/issues/25. Summary: "In general, root programs maintain both a trust store and a platform verifier, and the behavior of the two is linked. Distrusts may be implemented first in the verifier (subject to various conditions), and only much later by removal from the trust store."

[briansmith]

@jsha That is an excellent summary, and the thinking there has driven the changes I've already made to the certificate validation APIs in Rustls, so that the right native integration can be implemented.

[joshtriplett]

Ah! That makes sense. Would that also help cover cases where the certificate store needs an explicit negative entry for a CA or sub-CA, or a "must use CT" entry?

[jsha]

Would that also help cover cases where the certificate store needs an explicit negative entry for a CA or sub-CA, or a "must use CT" entry?

Presumably yes. By the way, worth mentioning that "must use CT" is no longer a great example, because "must use CT" is now enforced as a blanket policy in Chrome (and Apple devices), rather than something enforced for a subset of CAs.

https://groups.google.com/a/mozilla.org/g/dev-security-policy/c/QphQiDwBlC8/m/MdRiOy_qBgAJ

[briansmith]

I think we should close this. I filed https://github.com/rustls/rustls/issues/940 to document why we can't do things like this.

[bdaehlie]

Closing on the basis of Brian's last comment.

[djc]

I completely agree that a Cargo feature flag, by itself, is tricky because of the additive nature of Cargo features. However, a proven concept is to offer a Cargo feature that can only be used/enabled by an explicit API call. For example, in this case, we could offer with_webpki_roots() and with_native_roots() and maybe with_platform_verifier() as an alternative to the with_root_certificates() API. This would make life quite a bit easier for downstream users without the risks of just a Cargo feature.

Personally I think this would be an improvement, as currently the conversions required to do any of the above manually are kinda annoying to write out (there's a bunch of boilerplate that's duplicated across a whole lot of low-level crates).