How to Monitor Your Rails/Passenger App with Munin

11 February 2010

[caption id="attachment_70" align="alignleft" width="300" caption=" "][/caption]

Munin is a great tool to monitor resources on your server, showing graphs over time, so that you can analyze resource trends to find what's killing your server before it causes major problems. It is also very configurable and can be made to profile and graph just about anything via plugins. And with a couple tricks, you can get it to monitor your Phusion Passenger application with ease.

Update: If you have RVM installed on your server and need Munin to work with RVM's Passenger installation, follow all of the instructions below, and then make the changes described in Monitor Passenger with Munin when using RVM.*

* These changes will still use your server's non-RVM-installed default Ruby to run the Munin plugin, which will in turn use your RVM-installed Ruby to run the `passenger-status` and `passenger-memory-stats` commands.

If you already have Munin installed and working, and just want to know how to get the Passenger Plugins working, you can skip directly to Install Munin Passenger Plugins, Configure Munin for Passenger Stats, and be sure to read the Gotcha.

Install Munin and Munin-node

The first step is to install Munin. If you have your server running Ubuntu, this is pretty easy. One you've SSH'd into your server, enter:

sudo apt-get install munin munin-node -y

If you're running another flavor of Linux, see the Munin's Linux install instructions. For Mac OSX, see Mac install instructions.

Install Munin Passenger Plugins

The next step is to install the Passenger plugins. The first is passenger_status:

wget http://gist.github.com/20319.txt
sudo mv 20319.txt /usr/share/munin/plugins/passenger_status
sudo chmod a+x /usr/share/munin/plugins/passenger_status
sudo ln -s /usr/share/munin/plugins/passenger_status /etc/munin/plugins/passenger_status

The second plugin is passenger_memory_stats:

wget http://gist.github.com/21391.txt
sudo mv 21391.txt /usr/share/munin/plugins/passenger_memory_stats
sudo chmod a+x /usr/share/munin/plugins/passenger_memory_stats
sudo ln -s /usr/share/munin/plugins/passenger_memory_stats /etc/munin/plugins/passenger_memory_stats

No go ahead and restart Munin-node (this is the process that runs Munin at regular intervals):

sudo /etc/init.d/munin-node restart

Configure Munin

Now here are where those couple of tricks come in to get Munin playing nicely with your Rails application. First we want to tell Munin where to store the html and graph images that you can access through the browser.

sudo nano /etc/munin/munin.conf

And change the following lines:

htmldir /path/to/your/rails/shared/directory/munin
[yoursite.com]

Note that the htmldir can really be any directory, but make sure it's persistent (i.e. if you're using Capistrano to keep revisions of your app on the server, make sure the munin directory is in the shared directory outside of your rails app root directory).

Also note that the [yoursite.com] part is only a descriptive name, so it really doesn't matter what you call it. If you have a more complex application that runs from multiple directories or multiple servers, you can group Munin stats, and in this case, you actually have to put some thought into this line. But that's outside the scope of this article, so you can read up on customizing Munin Master on your own time if you'd like.

Now if you haven't already, you need to actually create that directory you just told Munin about:

cd /path/to/your/rails/shared/directory
mkdir -p munin
sudo chown munin:munin munin

Configure Munin For Passenger Stats

And finally, you need to allow the munin user to run the passenger-status and passenger-memory-stats commands without a password, since they both require sudo powers to run properly.

sudo visudo

And at the bottom of the file add:

munin   ALL=(ALL) NOPASSWD:/usr/bin/passenger-status, /usr/bin/passenger-memory-stats

Gotcha

At this point, Munin is suppose to start doing it's stuff and all is happy in the ruby-munin marriage. For me, however, this was not the case. After combing the Munin error logs and digging through the Munin documentation and code more than I care to admit, I realized that Munin-node needs to preface the passenger stat commands with ruby. So, here's how we fix that:

sudo nano /etc/munin/plugin-conf.d/munin-node

And then add this to the bottom of that file:

[passenger_*]
user munin
command ruby %c

Final Munin Restart

Now we'll give Munin-node one more restart and we're in business.

sudo /etc/init.d/munin-node restart

After waiting a few minutes, you should start to see .html files and graphs and whatnot in the /path/to/your/rails/shared/directory/munin directory. If not, you want to check out the Munin-node error logs to see what's going on. On Ubuntu, this is found at /var/log/munin/munin-node.log.

View Munin From Your Browser

So now, Munin is running, but what good is it if you can't view the pretty output? Typically the best way to do this is through a sub-domain. If you have yoursite.com, let's set up Munin to be viewable from munin.yoursite.com. All you need to do is point to the /path/to/your/rails/shared/directory/munin directory for munin.yoursite.com in your site's server conf file. For example, if you're on Apache, you'd just do this in your sites-available/yoursite.conf

<VirtualHost *:80>
ServerName munin.yoursite.com
DocumentRoot /path/to/your/rails/shared/directory/munin
</VirtualHost>

And make sure you have the proper A-name record setup for munin.yoursite.com with your NameServers.

Once domain propogation magic happens, you should now be able to see this when you navigate to munin.yoursite.com

[caption id="attachment_75" align="aligncenter" width="300" caption="Munin main page"][/caption]

The Passenger stats will be in the "App" section.

Have fun now monitoring your Passenger Rails application with passenger usage trending. Remember to use your newfound power for good.

(irqstats patch)

If you find that the irqstats graph doesn't work, you may need to patch the irqstats plugin. Basically you just need to sudo nano /etc/munin/plugins/irqstats and change

if ($irq =~ /^\d+$/) {

to

# numeric values or something like NMI/RES/CAL are accepted
if (($irq =~ /^\d+$/) || ($irq =~/^[A-Z]{3}$/)) {

About the author:

Steve Schwartz // Owner of Alfa Jango Web-based Software, creator of RateMyStudentRental & LeadNuke, engineer, hacker, rubyist, guitarist, aspiring racecar driverist.



Comments are loading...