title: Alertmanager not sending out alert to slack
body: <p>I configured alertmanager with prometheus and according to the prometheus interface the alerts are firing. However there is not slack message showing up and I am wondering if maybe ufw needs to be configured or if there is any other config I missed.</p>
  <p>The alertmanager service is running and prometheus shows &quot;firing&quot;.</p>
  <p>Here are my config files:</p>
  <p>alertmanager.yml:</p>
  <pre><code>global:
   slack_api_url: 'https://hooks.slack.com/services/my_id_removed...'
  route:
    group_by: ['alertname']
    group_wait: 30s
    group_interval: 5m
    repeat_interval: 1h
    receiver: 'slack_general'
  receivers:
  #- name: 'web.hook'
  #  webhook_configs:
  #  - url: 'http://127.0.0.1:5001/'
  - name: slack_general
    slack_configs:
    - channel: '#alerts'
      send_resolved: true
  inhibit_rules:
    - source_match:
        severity: 'critical'
      target_match:
        severity: 'warning'
      equal: ['alertname', 'dev', 'instance']
  </code></pre>
  <p>prometheus.yml:</p>
  <pre><code>global:
    scrape_interval: 10s
    evaluation_interval: 15s # Evaluates rules every 15s. Default is 1m
  alerting:
    alertmanagers:
    - static_configs:
      - targets: ['localhost:9093']
  rule_files:
    - rules.yml
  scrape_configs:
    - job_name: 'prometheus'
      scrape_interval: 5s
      static_configs:
        - targets: ['localhost:9090', 'localhost:9104']
  
    - job_name: 'node_exporter_metrics'
      scrape_interval: 5s
      static_configs:
        - targets: ['leo:9100', 'dog:9100']
  
    - job_name: 'alertmanager'
      static_configs:
        - targets: ['localhost:9093']
  </code></pre>
  <p>alerts.yml:</p>
  <pre><code>groups:
   - name: test
     rules:
     - alert: InstanceDown
       expr: up == 0
       for: 1m
     - alert: HostHighCpuLoad
       expr: 100 - (avg by(instance) (rate(node_cpu_seconds_total{mode=&quot;idle&quot;}[2m])) * 100) &gt; 80
       for: 0m
       labels:
         severity: warning
       annotations:
         summary: Host high CPU load (instance {{ $labels.instance }})
         description: &quot;CPU load is &gt; 80%\n  VALUE = {{ $value }}\n  LABELS = {{ $labels }}&quot;
     - alert: HostOutOfMemory
       expr: node_memory_MemAvailable_bytes / node_memory_MemTotal_bytes * 100 &lt; 10
       for: 2m
       labels:
         severity: warning
       annotations:
         summary: Host out of memory (instance {{ $labels.instance }})
         description: &quot;Node memory is filling up (&lt; 10% left)\n  VALUE = {{ $value }}\n  LABELS = {{ $labels }}&quot;
     # Please add ignored mountpoints in node_exporter parameters like
     # &quot;--collector.filesystem.ignored-mount-points=^/(sys|proc|dev|run)($|/)&quot;.
     # Same rule using &quot;node_filesystem_free_bytes&quot; will fire when disk fills for non-root users.
     - alert: HostOutOfDiskSpace
       expr: (node_filesystem_avail_bytes * 100) / node_filesystem_size_bytes &lt; 8 and ON (instance, device, mountpoint) node_filesystem_readonly == 0
       for: 2m
       labels:
         severity: warning
       annotations:
         summary: Host out of disk space (instance {{ $labels.instance }})
         description: &quot;Disk is almost full (&lt; 8% left)\n  VALUE = {{ $value }}\n  LABELS = {{ $labels }}&quot;
  </code></pre>
  <p><a href="https://i.stack.imgur.com/TmAWA.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/TmAWA.png" alt="enter image description here" /></a></p>
  <pre><code> sudo systemctl status alertmanager.service
  ● alertmanager.service - Alertmanager for prometheus
       Loaded: loaded (/etc/systemd/system/alertmanager.service; enabled; vendor preset: enabled)
       Active: active (running) since Sat 2022-03-05 22:02:31 CET; 6min ago
     Main PID: 9398 (alertmanager)
        Tasks: 30 (limit: 154409)
       Memory: 21.8M
       CGroup: /system.slice/alertmanager.service
               └─9398 /opt/alertmanager/alertmanager --config.file=/opt/alertmanager/alertmanager.yml --storage.path=/opt/alertmanager/data
  
  Mar 05 22:02:31 leo alertmanager[9398]: level=info ts=2022-03-05T21:02:31.094Z caller=main.go:225 msg=&quot;Starting Alertmanager&quot; version=&quot;(version=0.23.0, branch=HEAD, revision=61046b17771a57cfd4&gt;
  Mar 05 22:02:31 leo alertmanager[9398]: level=info ts=2022-03-05T21:02:31.094Z caller=main.go:226 build_context=&quot;(go=go1.16.7, user=root@e21a959be8d2, date=20210825-10:48:55)&quot;
  Mar 05 22:02:31 leo alertmanager[9398]: level=info ts=2022-03-05T21:02:31.098Z caller=cluster.go:184 component=cluster msg=&quot;setting advertise address explicitly&quot; addr=192.168.0.2 port=9094
  Mar 05 22:02:31 leo alertmanager[9398]: level=info ts=2022-03-05T21:02:31.099Z caller=cluster.go:671 component=cluster msg=&quot;Waiting for gossip to settle...&quot; interval=2s
  Mar 05 22:02:31 leo alertmanager[9398]: level=info ts=2022-03-05T21:02:31.127Z caller=coordinator.go:113 component=configuration msg=&quot;Loading configuration file&quot; file=/opt/alertmanager/alertma&gt;
  Mar 05 22:02:31 leo alertmanager[9398]: level=info ts=2022-03-05T21:02:31.128Z caller=coordinator.go:126 component=configuration msg=&quot;Completed loading of configuration file&quot; file=/opt/alertma&gt;
  Mar 05 22:02:31 leo alertmanager[9398]: level=info ts=2022-03-05T21:02:31.131Z caller=main.go:518 msg=Listening address=:9093
  Mar 05 22:02:31 leo alertmanager[9398]: level=info ts=2022-03-05T21:02:31.131Z caller=tls_config.go:191 msg=&quot;TLS is disabled.&quot; http2=false
  Mar 05 22:02:33 leo alertmanager[9398]: level=info ts=2022-03-05T21:02:33.099Z caller=cluster.go:696 component=cluster msg=&quot;gossip not settled&quot; polls=0 before=0 now=1 elapsed=2.000022377s
  Mar 05 22:02:41 leo alertmanager[9398]: level=info ts=2022-03-05T21:02:41.101Z caller=cluster.go:688 component=cluster msg=&quot;gossip settled; proceeding&quot; elapsed=10.002298352s
  </code></pre>