The "Tech. Arch."

Architecting Forward ::>

Visualize Application Metrics on NewRelic for your Ruby Camping Application

Monitoring Web Applications

Traditionally monitoring a web application or a web site involves a lot of discrete activities such as
  • Reviewing web server logs either manually for 500 errors
  • Reviewing web application logs
  • Reviewing database logs
  • Reviewing stats produced by custom timing code around key transactions
The last point related to “custom timing code” is really hard and time consuming (ironically!). What would be best would be to leverage a framework which could auto-magically wrap specific call (or methods in the case of Ruby) with timing code. That code would also create tracking data points in a database for future graphing and analysis.

What would be better is an overall dashboard system presenting all key activities across service tiers (web tier, web services, database). And the value of such a system would be even greater if your app is hosted on a cloud across a server farm (like my application: myskillsmap.com on Heroku)

The NewRelic Instrumentation Framework

Enters “NewRelic”, an application performance management platform company. NewRelic has partnered with Heroku to provide various levels of service based on the sophistication you need. NewRelic’s instrumentation framework is called RPM and has traditionally supported Ruby On Rails, as well as Rack and Sinatra on an experimental basis until last week’s new release.

Since my current flagship site (mySkillsMap) is based on the Ruby Camping framework, a couple months ago I started to tinker with the RPM framework to create an experimental plug-in for a simple Camping test app.

The general idea is to attach the RPM instrumentation to all controllers of that Camping app.

  class TodaysShoutoutsList < R '/shoutouts'
    def get
		@shoutouts = Campingdemo::Models::Shoutout.find :all,
			:conditions => [ 'created_at >= ?', Date.today.to_s ]
		render :list
    end
  end

In this example, I would like the TodaysShoutoutsList controller to be instrumented automatically.
So I reviewed the basic implementation of the Rack plugin and looked into how I could monkey-patch the custom service method defined by Rack but overridden in the Camping framework.
What I needed was to have the service method be executed within the context of the NewRelic perform_action_with_newrelic_trace method.

This turned out to be a pretty interesting exercise, as I had to really study the meta behavior in Camping as well as to leverage the power of the module_eval Ruby method. Like everything in Ruby the final implementation always ends up being very concise:

require 'new_relic/agent/instrumentation/controller_instrumentation'

module NewRelic
  module Agent
    module Instrumentation
      module Camping

        def self.included(mod)
          (Kernel.const_get(mod.name)::Base).module_eval do
            def service_with_newrelic(*args)
              perform_action_with_newrelic_trace(:category => :rack) do
                service_without_newrelic(*args)
              end
            end
            
            alias service_without_newrelic service
            alias service service_with_newrelic
          end
        end
        
      end
    end
  end
end
  

Once experimental support was working and integrated in a test site,I submitted the plugin to NewRelic. Then NewRelic’s Bill Kayser merged it and tweaked it further to make it fit with the new RPM_Contrib gem.

Adding NewRelic Support to a Camping App, Step By Step

At a high-level, the implementation consists of the following steps:

  1. Get an RPM application account
  2. Install the RPM gems
  3. Configure RPM .yml file
  4. Integrate RPM’s plugin in your Camping app module

Let’s get started!

Get an RPM application account

If you are an Heroku customer, just go to your application control panel and select the NewRelic add-on. You can also do this from the command line:

$ heroku addons:add newrelic:bronze
  

If you are self-hosted or use another host, sign-up for RPM Lite.

Install the RPM gems

If you are an Heroku customer, add the 2 NewRelic gems to your .gems file:

newrelic_rpm  --version '>= 2.10.6'
rpm_contrib  --version '>= 1.0.2'
  

If you are self-hosted or use another host, install the 2 NewRelic gems the standard way:

gem install newrelic_rpm  --version '>= 2.10.6'
gem install rpm_contrib  --version '>= 1.0.2'
  

Configure RPM .yml file

Before you proceed you need to find out what your license key is.
If you are using Heroku run the following command:

heroku config -all
  

You will need the value of NEW_RELIC_LICENSE_KEY in a minute.

RPM is configured using a newrelic.yml configuration file. If you don’t already have a config folder under your root application directory, create it now. Then create a newrelic.yml file with the following contents:

common: &default_settings
  license_key: 'PASTE THE VALUE OF NEW_RELIC_LICENSE_KEY HERE'
  agent_enabled: true
  app_name: PASTE THE NAME OF YOUR CAMPING APP HERE
  enabled: true

production:
  <<: *default_settings
  enabled: true

  

Save the file.

Integrate RPM's plugin in your Camping app module

Now we just need a couple tweaks and we're on our way to get application monitoring!

  1. At the top of your Camping app source file, require the rpm_contrib gem
    require "rpm_contrib"
      
  2. Right, before the end of your main module add an include of the RPMContrib::Instrumentation::Camping module. Note that you may add a conditional statement to only include the module if you are running in your production environment.
    module Campingdemo
      #
      # ...
      #
      include RPMContrib::Instrumentation::Camping
    end
      
  3. At the end of your source file, add an instruction to start the agent:
    module Campingdemo
      # ...
    end
    # ... other modules ...
    
    NewRelic::Agent.manual_start 
      

    Note that may also add a conditional statement to only start the agent if you are running in your production environment.

All right, now you're ready to push your updates to your production site and test the dashboard.

Testing the Dashboard

In my simple test app, I have 4 controllers, which I will test in order: , NewShoutout, AddShoutout, TodaysShoutoutsList. So

  1. Index - the main controller for the app
  2. NewShoutout - displays a form to enter a shoutout - it will POST to AddShoutOut
  3. AddShoutOut - inserts a new shoutout in the database and returns to the home page
  4. TodaysShoutoutsList - displays a table of all shoutouts created today

Testing the AddShoutOut controller

Testing the TodaysShoutoutsList controller

Let's look at the Overview tab of the NewRelic dashboard. We should see the overall activity of the app during my test:

NewRelic Overview tab

On the overview you can quickly see the response times of your app as well as the Apdex score, a useful metric measuring customer satisfaction-level performance.

When selecting the Web Transactions tab, data associated with each controller is presented. You can choose from different types of stats, in this case I chose "Most Time Consuming".

NewRelic Web Transactions Overview

If I select the AddShoutOut controller, I will see the following detailed stats:

Details for the AddShoutOut controller

On these graphs you can see not only the performance and the Apdex data for the controller, but you can also drill down into the breakdown of the controller in terms of SQL calls.

When selecting the Database tab, we can see performance data for each type of SQL queries performed by the controllers:

NewRelic Database tab

So as you can see in this brief demo, it is pretty easy to identify bottlenecks in your apps and tune them appropriately. Empowered with performance data across all tiers, you can also make appropriate decisions in terms of how to scale the cloud platform your application is running on.

So What?

NewRelic RPM makes it trivial to get powerful application performance monitoring. If you are hosting your app on Heroku, just add the NewRelic resource and update your app to integrate the NewRelic Camping plugin. If you are hosting your app somewhere else you can try RPM Lite to experience its potential.

You don't have to envy Rails, Rack, and Sinatra for getting NewRelic support. The NewRelic Camping plugin now allows you to integrate with NewRelic. So now you no longer have an excuse and you can finally enjoy the benefit of the application performance monitoring dashboard! :-)

References and Resources

March 7th, 2010 Posted by | performance mgt, Ruby, Ruby Camping, Tools | no comments