like-inspur
like-inspur
没有,只是修改了prometheus的config,再次测试了依然复现,cooridnator日志也打印第二次分配shard的过程: ``` level=info ts=2021-06-24T11:10:55.967Z caller=kubernetes.go:263 component="discovery manager scrape" discovery=kubernetes msg="Using pod service account via in-cluster config" level=info ts=2021-06-24T11:10:55.969Z caller=kubernetes.go:263 component="discovery manager scrape" discovery=kubernetes msg="Using pod service account via in-cluster config"...
同时发现删除prometheus的某个job配置后,series总数减少了,但是新建的prometheus个数却比删除前还多一个(原来4个,现在5个),查看某些prometheus,根本没有target。应该是coordiantor分配target还有问题
根据series的分配,启动时2个prometheus就够了;但是coordinator分配了3个prometheus,查看每个prometheus发现,prometheus-1并没有target,应该是cooridnator判断prometheus-1异常,因此扩展一个副本创建prometheus-2并分配target;因为配置了允许缩容,过了一段时间,prometheus-2的target迁移到proemtheus-1,prometheus的副本数从3降为2 本次问题原因出现在coordinator对prometheus-1判断异常,版本0.1.4,cooridnator日志如下: ``` time="2021-06-17T11:22:37Z" level=info msg="need space 64107" component=coordinator time="2021-06-17T11:22:37Z" level=info msg="change scale to 2" component="shard manager" sts=prometheus time="2021-06-17T11:22:47Z" level=info msg="need space 37517" component=coordinator time="2021-06-17T11:22:47Z" level=info msg="prometheus-0 need update targets"...
请问sidecar和coordinator都会暴露指标吗
sidecar暴露的指标和prometheus原生的一样,coordinator没有暴露指标
可以标记每个target所在的正确分片,即当出现2个分片采集同一个target时,删除多余分片的target,保证target在正确分片上
可以标记每个target所在的正确分片,即当出现2个分片采集同一个target时,删除多余分片的target,保证target在正确分片上
> 如果分片恢复,会自动去掉一个采集 最近发一个新版本吗?
I also meeth this problem on kubernetes 1.8,with prometheus configmap add cadvisor job like this: - job_name: 'kubernetes-cadvisor' metrics_path: /metrics/cadvisor tls_config: ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token kubernetes_sd_configs: - role: node relabel_configs:...
yes, add it on https://github.com/line/promgen/pull/319