Checking the Splunk Process from Nagios

Now its time to have Nagios check to make sure that splunk is running. For version 3 of Splunk there was a app / plugin you could get for Splunk that would work with Nagios. It appears to be gone. But I did find a snippet that some one posted here. Several things have changed so that script doesn’t work 100% out by cut and paste, but it was an excellent jumping off point and it took very few modifications to get going. Due to the fact that there is a copy right on this script, then I can’t put it here with out permission. But I will note that you can do the same thing almost by using the default nagios check_procs command.

So copy that script as check_splunk and stick it in your libexec folder. If your playing along with my setup thats: /opt/local/libexec/nagios

Once you have it downloaded you can ./check_splunk ports or procs
I didn’t worry about checking or trying to edit the search portion as for what I’m doing I don’t really need it right now, but I will revisit it if the need arises.

Now you have the script, its time todo the normal nagios setup stuff.
1. Add it to your commands
vi /opt/local/etc/nagios/objects/commands.cfg

define command {
command_name check_splunk
command_line $USER1$/check_splunk $ARG1$
}

2. Add it to your localhost
vi /opt/local/etc/nagios/objects/localhost.cfg

define service{
use generic-service
host_name localhost
service_description Splunk Port
check_command check_splunk!ports
}

define service{
use generic-service
host_name localhost
service_description Splunk Procs
check_command check_splunk!procs
}

Now restart nagios and you should be good to go