This is Article #4 of a 4-part series. For a good primer, check out the first two articles listed below. For the reasoning and analysis behind the “Recommended” option in this article, check out Part 3, How to Combine GZip + CDN for Fastest Page Loads. Otherwise, jump right in!

• The Importance of Page Load Speed
• Improve Page Load Speed (by 80%) by Improving Component Load Speed
• How to Combine GZip + CDN for Fastest Page Loads
• Caching, Zipping, and (Amazon CloudFront) CDN For A Rails App
  • Prerequisites
    1. Cached stylesheets and javascripts
    2. Creating an Amazon AWS Account
  • Setup S3 Buckets
  • Setup CloudFront Distributions
  • Create CNAME records (optional)
  • Install Rails S3 Synch Plugin
    1. Installing AWS-S3 Gem
    2. Configure S3 Synch Plugin
    3. Add S3 Synch to Deployment
  • Option A: Compressible Assets from App Server, Images from CloudFront (recommended)
    1. Configure Rails Asset Host
    2. Create A-name Record
    3. Configure Apache
  • Option B: Serve Everything from CloudFront (easier, but not recommended)
    1. Configure Rails Asset Host
    2. Pre-compile Cached Stylesheet and Javascript File
  • Conclusion

In this article, we’re going to speed up our Rails application by up to 75%, simply by optimizing our Rails asset host. We’re going to serve our components (stylesheets, javascripts, images, etc.) from a combination of our app’s server and Amazon CloudFront (Option A, recommended), or entirely from CloudFront (Option B – easier).

The best option for you may depend on your specific needs, but I’ll cover both processes below. For a an in-depth analysis of why Option A is recommended over Option B, see the last article in this series, How to Combine GZip + CDN for Fastest Page Loads.

Prerequisites

Cached Stylesheets and Javascripts

Another way to reduce page load time is to combine all of your components into as few files as possible. In other words, combine all of your stylesheets into a single css file, and likewise with your javascripts. Remember from the last article, that each request takes 50-150ms, not including the response and download time. If you have 10 separate javascripts, this equates to 0.5-1.5 seconds just to request the files (not to mention all the time to download them). If you can combine all of the files into one, that means you need just one request to get the same amount of data.

Luckily in Rails, this is easy, simply add :cache => 'cached-file-name' to your stylesheet_link_tag and javascript_include_tag in your application layout. For example:

Now, as long as the following line is set to true in your environment.rb, or more likely in production.rb, Rails will either load your combined files in the layout, or create and load them if they don’t already exist.

Simply packing all stylesheets and javascripts into one file each reduced page load time of one of our production applications from 10.1 to 8.3 seconds (an 18% reduction in load time alone).

Creating an Amazon AWS Account

If you do not yet have an Amazon AWS account, you will need to create that and enable S3 and CloudFront services. See this writeup on creating and setting up an Amazon S3 account. for helpful instructions.

Setup S3 Buckets

Once you’ve signed up for your Amazon AWS account and activated S3 and CloudFront, you’ll want to setup 4 S3 buckets for your application, using Amazons S3 management console.

We’re going to setup 4 buckets and CDN distributions because some old browsers still have an artificial limitation that only allow 2 concurrent connections to each domain, meaning our components will take longer to download from Amazon if they can only be downloaded 2 at a time. By creating 4 different domains pointing to 4 different buckets/distributions, we’re allowing our components to download up to 8 at a time from those browsers that still enforce this limitation.

Setup CloudFront Distributions

Once your S3 buckets are created, click over to the CloudFront tab and create one distribution for each S3 bucket as shown. You can type any comment to help you quickly identify each distribution.

Create CNAME Records (optional)

Once you’ve created your 4 CloudFront distributions, you may create a CNAME record for each distribution. This allows you to serve files from CloudFront using your own asset subdomains, like cdn0.yourapp.com, instead of raNDomString1234.cloudfront.net. We’ll use the following format of cdn%d.yourapp.com, where %d stands for digits 0-3:

Install Rails S3 Synch Plugin

This plugin adds some Capistrano recipes to synch our application’s public directory with our four S3 buckets automatically every time we deploy our app. See Spatten Design’s documentation for more information. I’ve made some updates to their original plugin to properly set the Cache-control and Expires headers for our assets on S3, as well as to properly set the Content-encoding header for Gzipped assets.

Update: I’ve updated the S3 Synch Plugin further; it can now handle unique S3 buckets for different Rails environments (e.g. one set of buckets for production and another for staging). Be sure to update your synch_s3_asset_host.yml file as shown below.

Installing AWS-S3 Gem

The synch_s3_asset_host plugin requires the AWS-S3 gem, so add the following to your environment.rb:

…and then run the following from the terminal to install the S3 Synch plugin’s gem dependency:

Configure S3 Synch Plugin

Create a config/synch_s3_asset_host.yml file like this:

Update: The “production” part in the file above has been added for my latest update of the S3 Asset Synch Plugin.

Add S3 Synch to Deployment

Now, in your Capistrano deploy.rb script, add the following line to the :deploy namespace:

…and then add the :asset_host_syncher => true flag to the :web role:

Option A: Compressible Assets from App Server, Images from CloudFront (recommended)

For more detail about why this method is recommended, see the last article in this series.

Configure Rails Asset Host

Use the following configuration in your production.rb file to configure the way Rails writes the URLs for asset_tags:

Create A-name Record

We will also need to create an A-name record for the cache.yourapp.com subdomain, which points to your application server’s IP address.

Configure Apache

Now we need to configure Apache to accept incoming requests to our “cache” subdomain, setting the appropriate far-future Expires and Cache-control headers. We also need to tell Apache to automatically compress and serve any compressible filetype on the fly. Add this to your site’s Apache conf file:

Now we need to make sure the appropriate Apache modules are enabled and restart Apache.

Option B: Serve Everything from CloudFront (easier, but not recommended)

For more detail about why this is not recommended, see the last article in this series. Basically, though, it’s because it requires you to make one of the following compromises:

• a) Serve all files uncompressed, resulting in file sizes up to 4x bigger than necessary.
• b) Serve Gzipped assets from CloudFront without first detecting whether or not the visitor’s browser support Gzip encoding.

That being said, if this is acceptable for you, this method is simpler to set up and configure.

Configure Rails Asset Host

Add the following to your production.rb:

Pre-compile Cached Stylesheet and Javascript File

If you’re serving every component from CloudFront, you will need to pre-compile your stylesheets and javascripts on every deploy. Otherwise, Rails will try to compile and save the files to your application server, but try to serve them from S3 (where they won’t exist).

To solve this, we’ll add some Capistrano scripts to our deploy.rb to compile our files for us before the synch_s3_asset_host plugin copies our public directory over to our S3 buckets. But this means, we’d have to copy the list of asset files to be compiled into our Capistrano script, as well as having them listed in our application.html.erb layout. To DRY things up a little, we’re going to create some project-wide constants:

lib/assets_for_cache.rb

And then replace your javascript_include_tag and stylesheet_link_tag in your application layout with the following:

Add this to your deploy.rb script:

…and then add this to the :deploy namespace in your deploy.rb file, before calling the s3_asset_host sync script:

Conclusion

Now simply save your project and deploy it! The first deploy will take quite a while, as your entire /public directory will be copied to all 4 buckets on Amazon S3, one at a time. But after that, it’s a painless process.

Whether you chose the “recommended” or the “easier” option, you should immediately notice a significant increase in the performance of your Rails app. Thanks for sticking with me through this 4-part series! Please let me know if you have any thoughts, questions, or feedback in the comments.

For an introduction to caching with Rails, check out this post: Caching With Rails: An Overview

The Appearance That Caching Works

Here is what I mean by giving the appearance that caching is working:

1. you add caches_page :index to your controller, for example
2. configure and restart your server
3. load the index page
4. check your rails cache directory on your server, and see the index.html
5. it works! (hint: this step is wrong, you’re not quite there yet)

Just because Rails is generating your cached page does not mean that your server is subsequently serving the cached .html file instead of sending the request to Rails again.

Ensure It’s Serving the Cached Page

To make sure it’s subsequently serving the cached page, the easiest method is to look at the created date on the cached file on your server, for instance:

You will get an output similar to this:

Now wait a minute, and then reload the page in your browser. Once again type ls -l myapp/public/cache, and make sure that the file still shows the same created date/time. If it instead shows this:

…then you have a problem. Your server is sending every request back to Rails, where Rails is then re-generating the cached page.

Alternative 1: You can also use Apache’s RewriteLog to watch what is being served. Just add this to your site’s .conf file and then tail -f the rewrite_log:

Alternative 2: You can also use the top command to watch your running processes and make sure the rails process doesn’t rise to the top while reloading your page. This is obviously not very scientific, though, and won’t work if you have any external traffic on the site.

The Solution: Proper Server Configuration

Happy caching! For more tips for getting the best performance out of your server, check out Performance Tuning for Phusion Passenger (An Introduction), and How to Monitor Your Rails/Passenger App with Munin.

Update: Proper Nginx Configuration

For extra completeness, thank you to atambo on Hacker News for pointing me to this proper configuration for Rails caching on Nginx. You can read the comments for an explanation.

Why this doesn’t work: You proceed with creating a custom theme for your Wordpress blog that looks exactly like your Rails application. Then over the next couple years, you begin to realize how tedious this solution is. You have to copy over your CSS stylesheets from your Rails app, your Javascript files (if your standard layout contains any Javascript elements, like a Twitter widget in the footer or whatever), and your layout images and assets. Then, every time you change or update the layout of your Rails app, you have to copy the changes in your custom Wordpress theme. And you cannot just copy the changes, because if you have any Ruby/Rails functionality in the layout, you have to translate them into PHP functionality in your Wordpress layouts.

And then there’s the impossible stuff… if anything in your Rails app layout loads information from the database in your Rails app, then you cannot have those elements in your Wordpress layout, unless you do something like create an XML or JSON feed in your Rails app and then import it in your Wordpress layout. But this is quite tedious still if you want to change any of it!

There’s an easier way.

Include Your Rails Layout Directly In Your Wordpress Layout

The solution I found turns out to be quite simple. The basic idea is to create a :partial in your Rails app layout, make it publicly accessible via your config/routes.rb, have it generate and simply return the appropriate html using your controller method, and then import that html (and CSS) in your Wordpress layout.

For this to really work well, you’ll want to make sure that you make your Rails app cache the partials to be imported into your Wordpress layout, so that a lot of traffic on your blog doesn’t cause your Rails app server to explode.

1. Break Your Rails Layout into Partials

The first step is to break as much of your app/views/layouts/application.html.erb into partials as possible/desired. At the very least, this means a header and a footer partial. Typically, you would not include the stuff in the <head> part of the layout, because your Wordpress will likely require additional stylesheets and a different set of javascripts (not to mention you’ll want to make your meta information specific for your blog pages).

A simple Rails layout using this method would look something like this:

I’ve also included a widgets partial to show that this can be done with static javascript widgets as well.

Note that your layout and partials can have any and all Ruby/Rails code in them that you want. Since your Wordpress theme is importing this stuff from your Rails application, the Rails app will run as usual and just return the end result, which is HTML, so your Wordpress will have no problem interpreting it.

A simple app/views/static/_header.html.erb partial may look something like this:

Notice that the normal links in the header are “active” simply if the current URL matches the link path, but the “Blog” link is active if the @blog variable is set and true. This is because, as you will see in the next section, the URL for the header when it is called from your blog won’t be the blog URL.

2. Make Your Layout Partials Publicly Accessible

The next step is to make your layout partials publicly accessible. I.e. make it so that you can access your layout partials from the browser using a direct URL, like http://mysite.com/static/header. The actual URL you use doesn’t really matter, because users won’t ever need to see or use it.

The way we do this is to explicitly set a path for each layout partial in our config/routes.rb and then create a controller method for each that renders the HTML generated by the partial.

A path in your routes for your header partial might look like this:

Then in the app/controllers/static_controller.rb, your corresponding action would look like this:

Now, hopefully you see why, in the header partial, we couldn’t just say that the blog menu item is :class => ('active' if request.path == blog_path). The path that the header partial will be seeing when the Wordpress layout requests it will be the path you defined in your routes.rb, in this case "static/header". This is why we instead say :class => ('active' if @blog) and then set @blog = true in the controller action.

Include the Rails Partials In the Wordpress Layout

Here’s the real trick. In order to include the Rails partials in your Wordpress layout, you might be inclined to use on of these PHP functions: include() or require(). This won’t work though for two reasons.

1) include() and require() both read .php files. However, your publicly-accessible layout partials are going to return straight HTML.

2) By default, neither method allows absolute URLs due to security precautions. You can change your PHP setup to allow this, but again, this is a security issue. Not to mention, for our purposes, you still would have to deal with reason 1 above.

Instead, we can use PHP’s function to retreive HTML code, file_get_contents().

In your wp-content/themes/my_theme/header.php file, you would then include this line:

Import the Rails CSS Stylesheet for Wordpress

Of course, you’ll also want to import the base stylesheet that styles your layout HTML for your Wordpress theme. You can do this a few different ways. The easiest is to just include it in your header.php file in the <head> area like any other stylesheet, just using the absolute URL to the CSS file.

Notice that we include the Rails app stylesheet before the blog stylesheet. This is so that it’s easy to override any of the styles we need to specifically in the blog stylesheet.

An alternative way we can do this is using the CSS @import method.

Cache the Layout Partials On Your Rails Server

Luckily this is easy to fix with Rails’s built-in page-caching.

In your app/controllers/static_controller.rb file, include the following.

And make sure that caching is turned on in your production app, in config/environments/production.rb:

And/Or Cache Your Wordpress Pages (recommended)

Of course, if you install and setup caching in your Wordpress installation (highly recommended), then the complete layout files will be cached and never even need to make calls to your Rails application after the cached pages are generated the first time. This is even faster for your Wordpress blog, because it’ll load faster now that it doesn’t have to import portions of the layout from an external URL every time a page is loaded.

Note that when you do this, you’ll need to expire/delete the cached Wordpress files anytime you update the layout in your Rails app.

Done!

Now you have a great looking Wordpress blog that looks like you’re never leaving your Rails application. And best of all, you don’t have to tediously duplicate efforts every time you update your Rails app layout! How’s that for DRY design?

Munin is a great tool to monitor resources on your server, showing graphs over time, so that you can analyze resource trends to find what’s killing your server before it causes major problems. It is also very configurable and can be made to profile and graph just about anything via plugins. And with a couple tricks, you can get it to monitor your Phusion Passenger application with ease.

If you already have Munin installed and working, and just want to know how to get the Passenger Plugins working, you can skip directly to Install Munin Passenger Plugins, Configure Munin for Passenger Stats, and be sure to read the Gotcha.

Install Munin and Munin-node

The first step is to install Munin. If you have your server running Ubuntu, this is pretty easy. One you’ve SSH’d into your server, enter:

If you’re running another flavor of Linux, see the Munin’s Linux install instructions. For Mac OSX, see Mac install instructions.

Install Munin Passenger Plugins

The next step is to install the Passenger plugins. The first is passenger_status:

The second plugin is passenger_memory_stats:

No go ahead and restart Munin-node (this is the process that runs Munin at regular intervals):

Configure Munin

And change the following lines:

Note that the htmldir can really be any directory, but make sure it’s persistent (i.e. if you’re using Capistrano to keep revisions of your app on the server, make sure the munin directory is in the shared directory outside of your rails app root directory).

Also note that the [yoursite.com] part is only a descriptive name, so it really doesn’t matter what you call it. If you have a more complex application that runs from multiple directories or multiple servers, you can group Munin stats, and in this case, you actually have to put some thought into this line. But that’s outside the scope of this article, so you can read up on customizing Munin Master on your own time if you’d like.

Now if you haven’t already, you need to actually create that directory you just told Munin about:

Configure Munin For Passenger Stats

And finally, you need to allow the munin user to run the passenger-status and passenger-memory-stats commands without a password, since they both require sudo powers to run properly.

And at the bottom of the file add:

Gotcha

At this point, Munin is suppose to start doing it’s stuff and all is happy in the ruby-munin marriage. For me, however, this was not the case. After combing the Munin error logs and digging through the Munin documentation and code more than I care to admit, I realized that Munin-node needs to preface the passenger stat commands with ruby. So, here’s how we fix that:

And then add this to the bottom of that file:

Final Munin Restart

Now we’ll give Munin-node one more restart and we’re in business.

After waiting a few minutes, you should start to see .html files and graphs and whatnot in the /path/to/your/rails/shared/directory/munin directory. If not, you want to check out the Munin-node error logs to see what’s going on. On Ubuntu, this is found at /var/log/munin/munin-node.log.

View Munin From Your Browser

So now, Munin is running, but what good is it if you can’t view the pretty output? Typically the best way to do this is through a sub-domain. If you have yoursite.com, let’s set up Munin to be viewable from munin.yoursite.com. All you need to do is point to the /path/to/your/rails/shared/directory/munin directory for munin.yoursite.com in your site’s server conf file. For example, if you’re on Apache, you’d just do this in your sites-available/yoursite.conf

And make sure you have the proper A-name record setup for munin.yoursite.com with your NameServers.

Once domain propogation magic happens, you should now be able to see this when you navigate to munin.yoursite.com

Munin main page

The Passenger stats will be in the “App” section.

Have fun now monitoring your Passenger Rails application with passenger usage trending. Remember to use your newfound power for good.

(irqstats patch)

If you find that the irqstats graph doesn’t work, you may need to patch the irqstats plugin. Basically you just need to sudo nano /etc/munin/plugins/irqstats and change

to