Chef Server's all-or-nothing IPv6 configuration is problematic

[stevendanna]

Currently, to support IPv6, we've provided users with a single configuration option:

ip_version

This configuration option has an impact on a variety of different internal service configuration options. As a result, it is very difficult to support a mixed environment. Such mixed environments might be especially more common for users that are using external PostgreSQL or external Elasticsearch.

We should investigate how much of this ip-version-specific configuration we can simply remove. Ideally, I would like the following behavior:

• The user does not specify any particular IP version configuration by default. If an IPv6 address is used as an IP address in their chef-server.rb, then the correct configuration is generated automatically. If the user specifies hostnames, then we prefer the IPv4 address returned for that name if it exists.

• We provide a new set of configuration items items that would allow you to express the following ideas:

• prefer_ipv4: Prefer IPv4 to IPv6, but use IPv6 if that is all that is available (the default)

• prefer_ipv6: Prefer IPv6 to IPv4, but use IPv4 addresses if that is all that is available.

• only_ipv6: Only allow IPv6 (perhaps with an exclusion for 127.0.0.1)

• only_ipv4: Only allow IPv4

We should research how other tools handle this configuration to provide a set of options that feel ergonomic to our users.

[markan]

One pain point in the whole IPv6 saga is the difference between things resolvable as IPv6 and those reachable as IPv6. Not everything that advertises IPv6 DNS can be reached via IPv6, many servers list both but only are accessible by one. The ibrowse library isn't sophisticated in handling this; it makes a choice based on resolvability, which may fail later as a timeout because it wasn't reachable. My hack on the chef ibrowse fork got us past a short term issue, but didn't help the situation. Waiting for one to time out and then trying the other seems unviable. From my modern browsers seem to have a bunch of heuristics, including trying both and taking the first to respond, or various nudges to try to get to IPv6 if possible. So the prefer_* strategies might require a little work; possibly including having something outside of ibrowse arbitrating which to chose, and probably caching the choice.

I'm pretty sure that only_ipv6 will require an exception for 127.0.0.1; I hit some snags getting things on ::1. In any event, if we are hardcoding an IP we we've embedded an opinion and probably should respect it.

[stevendanna]

My hack on the chef ibrowse fork got us past a short term issue, but didn't help the situation.

Yeah, I was looking at this today and it is definitely one of the trickier cases.

So the prefer_* strategies might require a little work; possibly including having something outside of ibrowse arbitrating which to chose, and probably caching the choice.

Yup I'm envisioning that if we want to go as far as the end-goal I outlined here we'd have to do a lot of the heavy lifting ourselves.

I think a stop along the path to it though is at the very least breaking out some of the configurables so that we could do things like have external PG using one IP version but communication with a backend using another.

Another thing I need to write up is that there appear to be some eldap bugs around IPv6 as well (IPv6 support was intended from what I can see in the code, but them some input validation was added that makes it impossible to use).