Building Performance Monitoring Solution with Nagios and NDOUtils

In my previous post i talked about Application Performance Management - APM tools and discussed possible requirements of it. I also talked about Nagios being wonderful and proven monitoring and solution. In this post i will show how to build monitoring solution with Nagios comparable to any commercial monitoring system.

We will be using following components along with Nagios.

• NDOUtils - Storing runtime information to database


• NRPE - Remote Plugin Agent


• NSCA - Remote Plugin with Passive Checks


• Nagvis - Visualization Addon


• Nagios Business Process Addons - Custom Bird's Eye View of Business Application


• NSClient++ - For monitoring Windows Hosts

Tough part is that these are all discrete components requires good knowledge. This series will focus on all these system and build good monitoring solution. This part will be focusing on NDOUtils and performance data. Once complete I will release the package with installation script so that it will be easy for installation.

I assume you have nagios installed and working. If not visit quick-start guides Ubuntu, Fedora, OpenSUSE. They work perfectly. Go through nagios console and explore what nagios provide out of the box.

Lets first see how to setup nagios and NDOUtils. NDOUtils is an implementation of Nagios Event Broker module(NEB). NEB has shared object which is loaded by nagios and can register for listening events. NDOUtils passes this event to file or socket. Second part of NDOUtils is C-daemon which saves the information to database.

Download ndoutils tar from nagios site. When you try to compile it you may face problem with mysql library. On Ubuntu first install mysql-dev library. Off course you have to have mysql server installed first.

sudo apt-get install libmysqlclient15-dev

Start mysql client and login as root. Create Database and nagios user.


create database nagios;
use nagios;
grant all on nagios.* to 'nagios'@'' identified by 'nagios'
grant all on nagios.* to 'nagios'@'localhost' identified by 'nagios'


Go to db directory and create necessary tables.

./installdb -u nagios -p nagios -h localhost -d nagios

Now go ndoutil deirctory and compile the source

./configure --with-mysql-lib=/usr/lib/mysql

Copy binaries to nagios. This assumes that you have installed nagios in /opt/nagios (Defualt is /usr/local/nagios)


cp ndo2db-3x /opt/nagios/bin
cp ndomod-3x.o /opt/nagios/bin


Also copy config files - ndomod.cfg and ndo2db.cfg from config directory to /opt/nagios/etc
Modify Following lines.

ndomod.cfg

output_type=tcpsocket
output=127.0.0.1
buffer_file=/opt/nagios/var/ndomod.tmp


ndo2db.cfg

socket_type=tcp
db_name=nagios
db_user=nagios
db_pass=nagios


Now open nagios.cfg file and modify following keys


event_broker_options=-1
broker_module=/opt/nagios/bin/ndomod-3x.o config_file=/opt/nagios/etc/ndomod.cfg


Now start ndo2db daemon before restarting nagios


/opt/nagios/bin/ndo2db-3x -c /opt/nagios/etc/ndo2db.cfg


Restart Nagios


/opt/nagios/bin/nagios -d /opt/nagios/etc/nagios.cfg


Now start mysql client and check nagios_hosts and nagios_services table to see that data is getting saved.

You can put following lines /etc/rc.local so that all events fired by Nagios ( which is started from /etc/init.d/nagios script) are picked up and processed. This makes everything started when machine boots up.

/opt/nagios/bin/ndo2db-3x -c /opt/nagios/etc/ndo2db.cfg

Once this is done all your data is saved to database. Now we can build our Performance Parser on top of this. I will come-up with shell or perl script for all installation once complete.

..
Tushar