Bryan Boreham comments

Results 1068 comments of


                                            Bryan Boreham

Don't log "repeated timestamp but different value" message as error/warning

@MadhavJivrajani sounds good so far. However Cortex depends on weaveworks/common; we can't export a string the other way round. Maybe Cortex could pass in a filtering function (via an extension...

Does Cortex query rolled up timeseries?

No, that's not implemented. See #4322 which I think is similar.

Distributor instrumentation reports "success" to requests returning 4xx when called via HTTPgRPC

I don't use httpgrpc for calls into the distributor, so don't care what you do in that regard. I guess we'd need to think about what happens in the places...

Distributor instrumentation reports "success" to requests returning 4xx when called via HTTPgRPC

This was changed in https://github.com/cortexproject/cortex/commit/f807690176ec98508a03c5ba29cbd8740c309ebd#diff-e58545b15a7caeb6b1290e4a6f2f41825141a8bfe028afe1736cfe9d2e4ac4f5R69, copied across to https://github.com/weaveworks/common/pull/117/files#diff-3e32bcfdd79c52ac49dbb25b172d9e2d3010cec4078f6f1748e4d588114d974dR69 So I guess this had something to do with the desired behaviour on retries. We should figure out what that was...

Distributor instrumentation reports "success" to requests returning 4xx when called via HTTPgRPC

Note what I said [here](https://github.com/cortexproject/cortex/issues/3949#issuecomment-925886977): > copied across to https://github.com/weaveworks/common/pull/117/files#diff-3e32bcfdd79c52ac49dbb25b172d9e2d3010cec4078f6f1748e4d588114d974dR69 was a misunderstanding. That `err` was always nil. _That_ came in at https://github.com/weaveworks/common/pull/36, which was a fix to https://github.com/weaveworks/common/issues/35. That...

the ring never removes old ingester even if the ingester pod is evicted

If the ingester shut down cleanly, even on eviction, then it would not be in the ring. So, the first task is to find out why it did not shut...

the ring never removes old ingester even if the ingester pod is evicted

Actually I don’t understand `could only find 0`, since your ring shows 1 active.

the ring never removes old ingester even if the ingester pod is evicted

I mean the ingester went through its exit sequence, rather than being abruptly terminated from outside. There are two main cases: hand-over to another ingester, and flush to store. In...

the ring never removes old ingester even if the ingester pod is evicted

That sounds like the same problem as #1290 @tomwilkie can you remember what that check is for? https://github.com/cortexproject/cortex/blob/7cf06900f6fea998724a46a8534ce13ee94c1b67/pkg/ring/replication_strategy.go#L20

the ring never removes old ingester even if the ingester pod is evicted

The current design requires that you set `terminationGracePeriodSeconds` long enough to shut down the first ingester cleanly. Your point 1 seems the same as #1290 Point 2 because we don't...