assert: Fix Subset/NotSubset when map is missing keys from the subset

[danielwhite]

Summary

This ensures that assert.Subset and assert.NotSubset consistently fails for tests against maps a key in the subset map is not contained in the map under test.

Changes

This ensures that the value obtained by reflect/Value.MapIndex from the map under test is checked for validity with reflect/Value.IsValid.

MapIndex returns the value associated with key in the map v. [... ] It returns the zero Value if key is not found in the map

This now aligns more closely with InDeltaMapValues.

The use of recover() was hiding this error by setting the result of the test to be "false" despite the test suite passing. This led to flapping tests where they would succeed if the panic occurred on a missing key before comparing key values that didn't match!

I've ensured the test suite now asserts on the expected error message and added another example where the subset has keys not found on the map under test.

Motivation

The following demonstrates a test that would pass more often than fail:

func TestSubset(t *testing.T) {
	x := map[string]int{
		"match":     1,
		"different": 20,
	}
	subset := map[string]int{
		"match":     1,
		"different": 2,
		"not-found": 3,
	}
	mockT := new(mockTestingT)
	isSubset := assert.Subset(mockT, x, subset)

	assert.False(t, isSubset)
	require.Len(t, mockT.Errors, 1, "expected mock test to report error")
	assert.Contains(t, mockT.Errors[0], "does not contain")
}

type mockTestingT struct {
	Errors []string
}

func (t *mockTestingT) Errorf(format string, args ...interface{}) {
	t.Errors = append(t.Errors, fmt.Sprintf(format, args...))
}

Without the change, we see a mixture of results due to the random order of map keys:

$ go test -run '^TestSubset$' -count=100 -json | jq --slurp --compact-output 'map(select(has("Test") and (.Action | test("pass|fail")))) | group_by(.Action) | map({Action: .[0].Action, Count: . | length }) | .[]'
{"Action":"fail","Count":14}
{"Action":"pass","Count":86}

With this change applied, the test passes consistently:

$ go test -run '^TestSubset$' -count=100 -json | jq --slurp --compact-output 'map(select(has("Test") and (.Action | test("pass|fail")))) | group_by(.Action) | map({Action: .[0].Action, Count: . | length }) | .[]'
{"Action":"pass","Count":100}

Related issues