Maintain order of Documents within the global Document Set throughout processing

[gcheadle-vmware]

What steps did you take: tests in the tkg-cli-provider repo CI

What happened: The list of line numbers is non-deterministic, failure from the CI shows:

-Error: : unable to get template: Overlaying (in following order: overlay.yaml, aws-overlay.yaml, 01_plans/prod.yaml, 
03_customizations/autoscaler/autoscaler_overlay.yaml, 03_customizations/http_proxy.yaml): Document on line 
03_customizations/http_proxy.yaml:62: Expected number of matched nodes to be 1, but was 3 (lines: base-
template.yaml:125, aws-overlay.yaml:150, aws-overlay.yaml:163)
+Error: : unable to get template: Overlaying (in following order: overlay.yaml, aws-overlay.yaml, 01_plans/prod.yaml, 
03_customizations/autoscaler/autoscaler_overlay.yaml, 03_customizations/http_proxy.yaml): Document on line 
03_customizations/http_proxy.yaml:62: Expected number of matched nodes to be 1, but was 3 (lines: aws-overlay.yaml:150, 
aws-overlay.yaml:163, base-template.yaml:125)

What did you expect: Expected (lines: base-template.yaml:125, aws-overlay.yaml:150, aws-overlay.yaml:163) to be in the same order for every run.

Anything else you would like to add: Accepted #256 as a temporary fix until the Carvel team can determine and fix the upstream source of non-determinism. Slack link with further context.

Environment:

• ytt version (use ytt --version):
• OS (e.g. from /etc/os-release):

[pivotaljohn]

@gcheadle-vmware and I reviewed this together:

• consider whether this solution is "sufficient" — sorting the output directly, while addresses the symptom, leaves open the possibility that any future output relying on the order of documents in the output of templates may accidentally also be non-deterministic.
• non-determinism comes from iterating over a map — looks like the heart of the non-deterministic behavior comes from how we're storing the total library of YAML Documents: in a map. When document matchers go searching over the total set, we're iterating over that map.
• be conscious of any performance trade-off — if we do decide to sacrifice some performance for more centralized determinism, please have data to quantify the performance difference (e.g. benchmark a large-ish library before and after).