I configured Nagios alert notifications to post to a Slack channel using Slack’s Incoming WebHooks.

Slack has a prepared Nagios Integration, so normally you could just use that.

However, this time I wanted to keep to the Lite plan, and to avoid hitting the 5 external integrations limit when using various integrations, I implemented it myself using Incoming WebHooks. Since there seems to be no limit on issuing Webhook URLs for Incoming WebHooks, it’s very useful.

I was going to write a shell script right away, but when I googled it, I found someone had published code on Gist, so I decided to fork it and modify it slightly for use.

Below is the content of the shell script published on gist.

#!/bin/bash

# This script is used by Nagios to post alerts into a Slack channel
# using the Incoming WebHooks integration. Create the channel, botname
# and integration first and then add this notification script in your
# Nagios configuration.
#
# All variables that start with NAGIOS_ are provided by Nagios as 
# environment variables when an notification is generated.
# A list of the env variables is available here: 
#   http://nagios.sourceforge.net/docs/3_0/macrolist.html
#
# More info on Slack
# Website: https://slack.com/
# Twitter: @slackhq, @slackapi

#Modify these variables for your environment
MY_NAGIOS_HOSTNAME="nagios.yourdomain.com"
MY_NAGIOS_STATUS_URL="http://${MY_NAGIOS_HOSTNAME}/nagios/cgi-bin/status.cgi?host=${NAGIOS_HOSTNAME}"
SLACK_HOSTNAME="yourslack.slack.com"
SLACK_TOKEN="xyxyxyourslackkey"
SLACK_CHANNEL="#alert"
SLACK_BOTNAME="Nagios"


#Set the message icon based on Nagios service state
if [ "$NAGIOS_SERVICESTATE" = "CRITICAL" ]
then
    ICON=":exclamation:"
    COLOR="danger"
elif [ "$NAGIOS_SERVICESTATE" = "WARNING" ]
then
    ICON=":warning:"
    COLOR="warning"
elif [ "$NAGIOS_SERVICESTATE" = "OK" ]
then
    ICON=":white_check_mark:"
    COLOR="good"
elif [ "$NAGIOS_SERVICESTATE" = "UNKNOWN" ]
then
    ICON=":question:"
    COLOR="#00FFFF"
else
    ICON=":white_medium_square:"
    COLOR="#FFFFFF"
fi

#Send message to Slack
curl -X POST --data "payload={\\"channel\\": \\"${SLACK_CHANNEL}\\", \\"username\\": \\"${SLACK_BOTNAME}\\", \\"icon_url\\": \\"https://slack.com/img/services/nagios_48.png\\", \\"text\\": \\"${ICON} HOST: ${NAGIOS_HOSTNAME}   SERVICE: ${NAGIOS_SERVICEDISPLAYNAME}     MESSAGE: ${NAGIOS_SERVICEOUTPUT} <${MY_NAGIOS_STATUS_URL}|See Nagios>\\"}" https://${SLACK_HOSTNAME}/services/hooks/incoming-webhook?token=${SLACK_TOKEN}

Place this in the plugins directory or similar on the server running Nagios and make it executable.

chmod +x /usr/lib64/nagios/plugins/slack_nagios.sh

Then add a definition to execute slack_nagios.sh to Nagios commands.

vim /etc/nagios/objects/commands.cfg

define command {
    command_name    notify-service-by-slack
    command_line    /usr/lib64/nagios/plugins/slack_nagios.sh
}

Also add Slack to contacts.

vim /etc/nagios/objects/contacts.cfg

define contact {
  contact_name slack
  use generic-contact
  service_notification_commands   notify-service-by-slack
  email   /dev/null
}

After that, register the services you want to notify to the Slack #alert channel, check the configuration, and reload to complete.

service nginx configtest
service nagios reload

The result of test notification of Nagios alerts to Slack looks like this:

Since Slack allows you to configure whether to receive app notifications for each channel, this way you can notice Nagios alerts even in the middle of the night and respond immediately. Great!

That’s all from the Gemba.