Enable Alerts

1. Open the Expert Editor.
2. In the following we will activate alerting for XCC. Repeat all steps below in the monitored_mta folder instead of monitoring to activate MTA alerts.
3. Open the file /monitoring/alertmanager/alertmanager.yml. Create it if it does not exist. All alert configuration will be done in this file.
4. Choose the desired alert channel and configure it in alertmanager.yml. Here we provide examples for email, webhook and Slack integration. You can find the full list of supported integrations on the official Prometheus Documentation.
   • Email receiver (basic)
   • Email receiver (with routing)
   • Webhook receiver
   • Slack receiver
   • No alerts (default)

   This example configuration sends all alerts to a specified email address, see also prometheus.io - email_config.

   global:
     smtp_smarthost: localhost:25
     smtp_from: alertmanager@xorlab.example.com
     smtp_auth_username: <alertmanager>     # Only required if smtp auth is used
     smtp_auth_password: <password>         # Only required if smtp auth is used
     smtp_require_tls: true
     smtp_hello: localhost
     resolve_timeout: 2m
   
   templates:
   - /config/monitoring/alertmanager/template/email.tmpl
   #  - /config/monitoring/alertmanager/template/custom.tmpl
   
   receivers:
   - name: email
     email_configs:
     - to: <alert@on-call.example.com>
       from: <alertmanager@xorlab.example.com>
   

   This example configuration sends alerts to different email addresses, based on the severity of the alerts, see also prometheus.io - email_config. There are four levels of alert severities used by the default alerts: normal, medium, high, and critical.

   global:
     smtp_smarthost: <localhost:25>
     smtp_from: <alertmanager@xorlab.example.com>
     smtp_auth_username: <alertmanager>     # Only required if smtp auth is used
     smtp_auth_password: <password>         # Only required if smtp auth is used
     smtp_require_tls: true
     smtp_hello: localhost
     resolve_timeout: 2m
   
   templates:
   - /config/monitoring/alertmanager/template/email.tmpl
   #  - /config/monitoring/alertmanager/template/custom.tmpl
   
   route:
     receiver: email-normal                 # default receiver
     group_by:
     - alertname
     group_wait:       1m                   # initial wait to ensure alerts can be grouped
     group_interval:   5m                   # subsequent alerts wait 30m for new alerts if in an active group
     repeat_interval: 30m                   # wait for 'repeat_interval' before resending active alerts
     continue: false                        # whether an alert should continue matching against subsequent routes
     routes:
     - receiver: email-medium
       matchers:
       - severity="medium"
     - receiver: email-high
       matchers:
       - severity="high"
     - receiver: email-critical
       matchers:
       - severity="critical"
   
   receivers:
   - name: email-normal
     email_configs:
     - to: '<user1@example.com>, <user2@example.com>'
       from: <alertmanager@xorlab.example.com>
   - name: email-medium
     email_configs:
     - to: '<user1@example.com>, <user2@example.com>'
       from: <alertmanager@xorlab.example.com>
   - name: email-high
     email_configs:
     - to: <on-call@example.com>
       from: <alertmanager@xorlab.example.com>
   - name: email-critical
     email_configs:
     - to: <on-call@example.com>
       from: <alertmanager@xorlab.example.com>
   

   This example configuration sends all alerts to a specified webhook, see also prometheus.io - Receiver integration settings

   route:
     receiver: webhook
     group_by:
     - alertname
     group_wait: 1m
     group_interval: 5m
     repeat_interval: 30m
   
     routes:
     - receiver: webhook
   
   receivers:
   - name: webhook
     webhook_configs:
     - url: https://example.com/webhook
   

   This example configuration sends all alerts to a specified Slack channel, see also prometheus.io - Receiver integration settings and the Slack documentation.

   global:
     # REPLACE the following slack_api_url with a valid one.
     slack_api_url: https://hooks.slack.com/services/XXXXXXXXXXX/YYYYYYYYYYY/ZZZZZZZZZZZZZZZZZZZZZZZZ
     resolve_timeout: 2m
   
   receivers:
   - name: slack-alert
     slack_configs:
       channel: <alert>
       text: '{{ template "slack.text.alert" . }}'
   

   By default, Alertmanager does not send alerts to any real receivers, all alerts are routed to a no-op / blackhole receiver.

   route:
     receiver: blackhole
     routes:
     - receiver: blackhole
   
   receivers:
   - name: blackhole
   
5. Optional: You also find some predefined examples in the same folder. Only the file alertmanager.yml will be considered, so they can be safely ignored or deleted if not used.
   • alertmanager.email.yml - Email notification examples, one basic example and one with severity based routing.
   • alertmanager.webhook.yml - Generic webhook example
   • alertmanager.slack.yml - Slack integration example.
   Mixing receivers

   The different examples can be mixed by configuring different receivers and adjusting the route configuration accordingly, similar to the Email receiver (with routing) example. Alertmanager supports more receivers and allows for more advanced configuration (e.g. alerts being sent to more than one receiver, complex routing, etc.), see also prometheus.io - Route-related settings.
6. Click Publish. The alerting becomes active within about one minute. Have a look at Test and Troubleshoot Alerts to verify them.

​

Optional: Enable notifications for resolved alerts

1. Open your active alertmanager.yml config file (once for the XCC and once for the MTA).
2. Find the receivers group, and under the chosen receiver config add send_resolved: true. For example, the receivers config for email alerts should look like this:
   (...)
   receivers:
     - name: email-normal
       email_configs:
         - to: alerts@example.com
           from: alertmanager@xorlab.example.com
           html: '{{ template "email.default.html" . }}'
           send_resolved: true
   (...)
   

3. Click Publish. The monitoring configuration becomes active within about one minute.