featurebase icon indicating copy to clipboard operation
featurebase copied to clipboard
verified publisher icon FeatureBaseDB

race in memberlist

Open jaffee opened this issue 7 years ago • 2 comments

This doesn't seem to reproduce every time, but this data race seems to be entirely in memberlist code during TestClusteringNodesReplica2.

WARNING: DATA RACE
Write at 0x00c4208feec9 by goroutine 475:
  github.com/pilosa/pilosa/vendor/github.com/hashicorp/memberlist.(*Memberlist).aliveNode()
      /Users/jaffee/go/src/github.com/pilosa/pilosa/vendor/github.com/hashicorp/memberlist/state.go:962 +0xe9e
  github.com/pilosa/pilosa/vendor/github.com/hashicorp/memberlist.(*Memberlist).mergeState()
      /Users/jaffee/go/src/github.com/pilosa/pilosa/vendor/github.com/hashicorp/memberlist/state.go:1150 +0x390
  github.com/pilosa/pilosa/vendor/github.com/hashicorp/memberlist.(*Memberlist).mergeRemoteState()
      /Users/jaffee/go/src/github.com/pilosa/pilosa/vendor/github.com/hashicorp/memberlist/net.go:985 +0x661
  github.com/pilosa/pilosa/vendor/github.com/hashicorp/memberlist.(*Memberlist).handleConn()
      /Users/jaffee/go/src/github.com/pilosa/pilosa/vendor/github.com/hashicorp/memberlist/net.go:233 +0xaf8

Previous read at 0x00c4208feec9 by goroutine 467:
  github.com/pilosa/pilosa/vendor/github.com/hashicorp/memberlist.(*Memberlist).rawSendMsgPacket()
      /Users/jaffee/go/src/github.com/pilosa/pilosa/vendor/github.com/hashicorp/memberlist/net.go:615 +0x636
  github.com/pilosa/pilosa/vendor/github.com/hashicorp/memberlist.(*Memberlist).gossip()
      /Users/jaffee/go/src/github.com/pilosa/pilosa/vendor/github.com/hashicorp/memberlist/state.go:512 +0x417
  github.com/pilosa/pilosa/vendor/github.com/hashicorp/memberlist.(*Memberlist).(github.com/pilosa/pilosa/vendor/github.com/hashicorp/memberlist.gossip)-fm()
      /Users/jaffee/go/src/github.com/pilosa/pilosa/vendor/github.com/hashicorp/memberlist/state.go:106 +0x41
  github.com/pilosa/pilosa/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc()
      /Users/jaffee/go/src/github.com/pilosa/pilosa/vendor/github.com/hashicorp/memberlist/state.go:130 +0x11b

Goroutine 475 (running) created at:
  github.com/pilosa/pilosa/vendor/github.com/hashicorp/memberlist.(*Memberlist).streamListen()
      /Users/jaffee/go/src/github.com/pilosa/pilosa/vendor/github.com/hashicorp/memberlist/net.go:192 +0x7a

Goroutine 467 (running) created at:
  github.com/pilosa/pilosa/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule()
      /Users/jaffee/go/src/github.com/pilosa/pilosa/vendor/github.com/hashicorp/memberlist/state.go:106 +0x3fb
  github.com/pilosa/pilosa/vendor/github.com/hashicorp/memberlist.Create()
      /Users/jaffee/go/src/github.com/pilosa/pilosa/vendor/github.com/hashicorp/memberlist/memberlist.go:165 +0xc3
  github.com/pilosa/pilosa/gossip.(*memberSet).Open()
      /Users/jaffee/go/src/github.com/pilosa/pilosa/gossip/gossip.go:59 +0xa4
  github.com/pilosa/pilosa/server.(*Command).setupNetworking()
      /Users/jaffee/go/src/github.com/pilosa/pilosa/server/server.go:339 +0x573
  github.com/pilosa/pilosa/server.(*Command).Start()
      /Users/jaffee/go/src/github.com/pilosa/pilosa/server/server.go:135 +0x20e
  github.com/pilosa/pilosa/test.Cluster.Start()
      /Users/jaffee/go/src/github.com/pilosa/pilosa/test/pilosa.go:201 +0x20f
  github.com/pilosa/pilosa/server_test.TestClusteringNodesReplica2()
      /Users/jaffee/go/src/github.com/pilosa/pilosa/server/server_test.go:435 +0x160
  testing.tRunner()
      /usr/local/Cellar/go/1.10.1/libexec/src/testing/testing.go:777 +0x16d

I ran make test TESTFLAGS="-count=1 -race -timeout=0"

d3ae1cdf5e2dbaf37aaf8c8f6054b3b12f7f472c which is jaffee/1609-log-race.

jaffee avatar Aug 21 '18 18:08 jaffee

just saw this again running go test -race in pilosa/server - just merged master in the roaring-import branch 495af54c10c85facf96aacfe5fdb32d47ecdf033

WARNING: DATA RACE
Write at 0x00c000b54fc9 by goroutine 363:
  github.com/pilosa/pilosa/vendor/github.com/hashicorp/memberlist.(*Memberlist).aliveNode()
      /Users/jaffee/go/src/github.com/pilosa/pilosa/vendor/github.com/hashicorp/memberlist/state.go:961 +0xf16
  github.com/pilosa/pilosa/vendor/github.com/hashicorp/memberlist.(*Memberlist).mergeState()
      /Users/jaffee/go/src/github.com/pilosa/pilosa/vendor/github.com/hashicorp/memberlist/state.go:1149 +0x3b2
  github.com/pilosa/pilosa/vendor/github.com/hashicorp/memberlist.(*Memberlist).mergeRemoteState()
      /Users/jaffee/go/src/github.com/pilosa/pilosa/vendor/github.com/hashicorp/memberlist/net.go:985 +0x6ab
  github.com/pilosa/pilosa/vendor/github.com/hashicorp/memberlist.(*Memberlist).handleConn()
      /Users/jaffee/go/src/github.com/pilosa/pilosa/vendor/github.com/hashicorp/memberlist/net.go:233 +0x994

Previous read at 0x00c000b54fc9 by goroutine 435:
  github.com/pilosa/pilosa/vendor/github.com/hashicorp/memberlist.(*Memberlist).rawSendMsgPacket()
      /Users/jaffee/go/src/github.com/pilosa/pilosa/vendor/github.com/hashicorp/memberlist/net.go:615 +0x62c
  github.com/pilosa/pilosa/vendor/github.com/hashicorp/memberlist.(*Memberlist).gossip()
      /Users/jaffee/go/src/github.com/pilosa/pilosa/vendor/github.com/hashicorp/memberlist/state.go:518 +0x635
  github.com/pilosa/pilosa/vendor/github.com/hashicorp/memberlist.(*Memberlist).gossip-fm()
      /Users/jaffee/go/src/github.com/pilosa/pilosa/vendor/github.com/hashicorp/memberlist/state.go:106 +0x41
  github.com/pilosa/pilosa/vendor/github.com/hashicorp/memberlist.(*Memberlist).triggerFunc()
      /Users/jaffee/go/src/github.com/pilosa/pilosa/vendor/github.com/hashicorp/memberlist/state.go:130 +0x10d

Goroutine 363 (running) created at:
  github.com/pilosa/pilosa/vendor/github.com/hashicorp/memberlist.(*Memberlist).streamListen()
      /Users/jaffee/go/src/github.com/pilosa/pilosa/vendor/github.com/hashicorp/memberlist/net.go:192 +0x7a

Goroutine 435 (running) created at:
  github.com/pilosa/pilosa/vendor/github.com/hashicorp/memberlist.(*Memberlist).schedule()
      /Users/jaffee/go/src/github.com/pilosa/pilosa/vendor/github.com/hashicorp/memberlist/state.go:106 +0x425
  github.com/pilosa/pilosa/vendor/github.com/hashicorp/memberlist.Create()
      /Users/jaffee/go/src/github.com/pilosa/pilosa/vendor/github.com/hashicorp/memberlist/memberlist.go:165 +0xc3
  github.com/pilosa/pilosa/gossip.(*memberSet).Open()
      /Users/jaffee/go/src/github.com/pilosa/pilosa/gossip/gossip.go:60 +0xaa
  github.com/pilosa/pilosa/server.(*Command).setupNetworking()
      /Users/jaffee/go/src/github.com/pilosa/pilosa/server/server.go:339 +0x58c
  github.com/pilosa/pilosa/server.(*Command).Start()
      /Users/jaffee/go/src/github.com/pilosa/pilosa/server/server.go:135 +0x1d0
  github.com/pilosa/pilosa/test.Cluster.Start()
      /Users/jaffee/go/src/github.com/pilosa/pilosa/test/pilosa.go:201 +0x23e
  github.com/pilosa/pilosa/test.runCluster()
      /Users/jaffee/go/src/github.com/pilosa/pilosa/test/pilosa.go:260 +0xaa
  github.com/pilosa/pilosa/test.MustRunCluster()
      /Users/jaffee/go/src/github.com/pilosa/pilosa/test/pilosa.go:268 +0x77
  github.com/pilosa/pilosa/server_test.TestClusteringNodesReplica1()
      /Users/jaffee/go/src/github.com/pilosa/pilosa/server/server_test.go:381 +0x79
  testing.tRunner()
      /usr/local/Cellar/go/1.11/libexec/src/testing/testing.go:827 +0x162
...
--- FAIL: TestClusteringNodesReplica1 (1.02s)
    testing.go:771: race detected during execution of test

jaffee avatar Sep 13 '18 21:09 jaffee

I got the same error on TestClusteringNodesReplica1 today, so it's not just Replica2.

seebs avatar Jan 18 '19 19:01 seebs