DATA RACE for `LocalFile.index`

[informalict]

version: v0.20.0

There is a DATA RACE for index field in LocalFile struct in file the index/rolodex_file_loader.go.

line 194:

l.index = index

and line 231

if l.index != nil && l.index.root == nil {

So reading from index (line 231) can happen at the same time as writing to it (line 194).

Details:

WARNING: DATA RACE
Read at 0x00c000316300 by goroutine 638:
  github.com/pb33f/libopenapi/index.(*LocalFile).GetContentAsYAMLNode()
      /home/runner/go/pkg/mod/github.com/pb33f/[email protected]/index/rolodex_file_loader.go:231 +0x444
  github.com/pb33f/libopenapi/index.(*rolodexFile).GetContentAsYAMLNode()
      /home/runner/go/pkg/mod/github.com/pb33f/[email protected]/index/rolodex_file.go:79 +0xd2
  github.com/pb33f/libopenapi/index.(*SpecIndex).lookupRolodex()
      /home/runner/go/pkg/mod/github.com/pb33f/[email protected]/index/find_component.go:151 +0x1be
  github.com/pb33f/libopenapi/index.(*SpecIndex).FindComponent()
      /home/runner/go/pkg/mod/github.com/pb33f/[email protected]/index/find_component.go:31 +0x147
  github.com/pb33f/libopenapi/index.(*SpecIndex).ExtractComponentsFromRefs.func1()
      /home/runner/go/pkg/mod/github.com/pb33f/[email protected]/index/extract_refs.go:657 +0x5f1
  github.com/pb33f/libopenapi/index.(*SpecIndex).ExtractComponentsFromRefs()
      /home/runner/go/pkg/mod/github.com/pb33f/[email protected]/index/extract_refs.go:721 +0x319
  github.com/pb33f/libopenapi/index.createNewIndex()
      /home/runner/go/pkg/mod/github.com/pb33f/[email protected]/index/spec_index.go:96 +0x584
  github.com/pb33f/libopenapi/index.NewSpecIndexWithConfig()
      /home/runner/go/pkg/mod/github.com/pb33f/[email protected]/index/spec_index.go:56 +0x71a
  github.com/pb33f/libopenapi/index.(*LocalFile).Index()
      /home/runner/go/pkg/mod/github.com/pb33f/[email protected]/index/rolodex_file_loader.go:191 +0x118
  github.com/pb33f/libopenapi/index.(*Rolodex).IndexTheRolodex.func1.1()
      /home/runner/go/pkg/mod/github.com/pb33f/[email protected]/index/rolodex.go:253 +0x1f2
  github.com/pb33f/libopenapi/index.(*Rolodex).IndexTheRolodex.func1.gowrap1()
      /home/runner/go/pkg/mod/github.com/pb33f/[email protected]/index/rolodex.go:278 +0x6e

Previous write at 0x00c000316300 by goroutine 639:
  github.com/pb33f/libopenapi/index.(*LocalFile).Index()
      /home/runner/go/pkg/mod/github.com/pb33f/[email protected]/index/rolodex_file_loader.go:194 +0x190
  github.com/pb33f/libopenapi/index.(*Rolodex).IndexTheRolodex.func1.1()
      /home/runner/go/pkg/mod/github.com/pb33f/[email protected]/index/rolodex.go:253 +0x1f2
  github.com/pb33f/libopenapi/index.(*Rolodex).IndexTheRolodex.func1.gowrap1()
      /home/runner/go/pkg/mod/github.com/pb33f/[email protected]/index/rolodex.go:278 +0x6e

Goroutine 638 (running) created at:
  github.com/pb33f/libopenapi/index.(*Rolodex).IndexTheRolodex.func1()
      /home/runner/go/pkg/mod/github.com/pb33f/[email protected]/index/rolodex.go:278 +0x489
  github.com/pb33f/libopenapi/index.(*Rolodex).IndexTheRolodex.gowrap1()
      /home/runner/go/pkg/mod/github.com/pb33f/[email protected]/index/rolodex.go:301 +0xa4

Goroutine 639 (finished) created at:
  github.com/pb33f/libopenapi/index.(*Rolodex).IndexTheRolodex.func1()
      /home/runner/go/pkg/mod/github.com/pb33f/[email protected]/index/rolodex.go:278 +0x489
  github.com/pb33f/libopenapi/index.(*Rolodex).IndexTheRolodex.gowrap1()
      /home/runner/go/pkg/mod/github.com/pb33f/[email protected]/index/rolodex.go:301 +0xa4

[informalict]

BTW please run your tests with go test -race so then you will see probably more DATA RACE

[daveshanley]

I am aware of this. Thank you for reporting however.

[daveshanley]

I have put some cycles into this, but it goes pretty deep, it's not a trivial thing to fix.

[AtomicTroop]

Just ran into this too. As a temporary workaround, setting ExtractRefsSequentially to true in your DocumentConfiguration sidesteps the issue in exchange for a reasonable performance hit.

In any case, it keeps our spec validation tests from failing.

[daveshanley]

Race condition has been fixed in v0.23.0 It required a significant lift, across the module, but clean, correct locking and async code is now in place.