alertmanager icon indicating copy to clipboard operation
alertmanager copied to clipboard

feat: add common threaded state feature with webex message threads implemented

Open gameloser opened this issue 1 year ago • 2 comments

Ref: https://github.com/prometheus/alertmanager/issues/3202 The PR adds a new field in nflog's Entry for storing common threaded notifying state, and provides one implementation for webex notifier. I think the others can be similar.

The approach is to make the ctx pass through func Notify so that the state passes through DedupStage to the RetryStage and all the way to the SetNotifiesStage. In the RetryStage, each notifier can set to and get from ctx.

The webex receiver is added a new field for switching threaded (bool) state between on/off.

I tested for webex threaded messages in cluster mode and by reloading threaded receiver field between true/false. Following are testing results:

global:
  resolve_timeout: 10m
  webex_api_url: https://webexapis.com/v1/messages
receivers:
  - name: default-receiver
  - name: webex
    webex_configs:
      - http_config:
          authorization:
            credentials: NWUX
            type: Bearer
        room_id: Y2lzX
        send_resolved: true
        threaded: true

route:
  group_wait: 10s
  group_interval: 1m
  repeat_interval: 1m
  receiver: default-receiver
  group_by:
    - alertname
  routes:
    - receiver: webex
      match:
        alertname: a

templates:
  - templates/*.tmpl
Screenshot 2024-07-05 at 14 43 32

Logs:

14:40:55 a2 | ts=2024-07-05T06:40:55.118Z caller=notify.go:922 level=debug component=dispatcher receiver=webex integration=webex[0] aggrGroup="{}/{alertname=\"a\"}:{alertname=\"a\"}" alerts=[a[d258efc][active]] msg="Notify success" attempts=1 duration=1.822621s
14:40:55 a3 | ts=2024-07-05T06:40:55.139Z caller=nflog.go:538 level=debug component=nflog msg="gossiping new entry" entry="entry:<group_key:\"{}/{alertname=\\\"a\\\"}:{alertname=\\\"a\\\"}\" receiver:<group_name:\"webex\" integration:\"webex\" > timestamp:<seconds:1720161655 nanos:118545000 > firing_alerts:11193240310645098077 threaded_state:<key:\"id\" value:\"Y2lzY2\" > > expires_at:<seconds:1720161775 nanos:118545000 > "
14:40:55 a1 | ts=2024-07-05T06:40:55.318Z caller=nflog.go:538 level=debug component=nflog msg="gossiping new entry" entry="entry:<group_key:\"{}/{alertname=\\\"a\\\"}:{alertname=\\\"a\\\"}\" receiver:<group_name:\"webex\" integration:\"webex\" > timestamp:<seconds:1720161655 nanos:118545000 > firing_alerts:11193240310645098077 threaded_state:<key:\"id\" value:\"Y2lzY2\" > > expires_at:<seconds:1720161775 nanos:118545000 > "

gameloser avatar Jul 05 '24 08:07 gameloser

@simonpasquier @gotjosh Any comments? thanks.

gameloser avatar Jul 11 '24 00:07 gameloser

@simonpasquier any comments? thx

gameloser avatar Sep 06 '24 02:09 gameloser