Route案例

一. prometheus rule新增监控项

pwd:/opt/app/monitor/prometheus/rule/alert-environment.yml
  - alert: instance_diskUsage
    expr: sum by(device) (node_filesystem_avail_bytes{device!~".*fs.*|shm"}) / 1024 / 1024 / 1024 < 20
    for: 1m
    labels:
      alertname: 服务器磁盘
      serverity: 非常严重
    annotations:
      summary: "服务器磁盘告警"
      description: "\{\{ $labels.ip \}\} 服务器系统盘剩余可用空间为\{\{$value | printf \"%.2f\" \}\}%"

二. alertmanager 配置修改

2.1 新增告警路由

pwd:/opt/app/monitor/alertmanager/alertmanager.yml
  receiver: wechat // 当告警数据label未匹配到"instance_memory_use"、"instance_load5"时,通过此渠道进行告警
  routes:
  - receiver: wechat2  // 通过"wechat2"告警渠道进行告警
    group_wait: 1m
    match:
      alertname: instance_memory_use // 告警数据label中匹配到"instance_memory_use"，则走次路由进行告警
  - receiver: wechat3  // 通过"wechat3"告警渠道进行告警
    group_wait: 1m
    match:
      alertname: instance_load5  // 告警数据label中匹配到"instance_load5"，则走次路由进行告警

2.2 新增路由告警接收人

pwd:/opt/app/monitor/alertmanager/alertmanager.yml
receivers:
- name: 'wechat'
  wechat_configs:
  - corp_id: 'corp_id'
    agent_id: 'agentid'
    api_secret: 'api_secret'
    to_user: 'TianCiwang'
    send_resolved: true

- name: 'wechat2'
  wechat_configs:
  - corp_id: 'corp_id'
    agent_id: 'agentid'
    api_secret: 'api_secret'
    to_user: 'TianCiwang'
    send_resolved: true

- name: 'wechat3'
  wechat_configs:
  - corp_id: 'corp_id'
    agent_id: 'agentid'
    api_secret: 'api_secret'
    to_user: 'TianCiwang'
    send_resolved: true