Removing deadweight - cleaning up our Rails app

Every man has their breaking point when it comes to deadweight code. Andre and I hit ours recently and decided to spend all of last week focusing soley on cleaning up Scout (a Rails app). Our goals:

Here’s how we went about it:

Tests

I needed visibility into our test performance – enter the test_benchmark gem. test_benchmark prints timing information for each test and test suite.

Two patterns emerged:

Slow External HTTP Requests

A number of our tests hit our payment provider’s gateway. All of these tests were slow. I mocked these requests with FakeWeb.

Time Savings: 3.76 minutes

ActiveRecord Generation

Some of our tests setup a decent number of database records. This can be significantly slower than using fixtures. Where appropriate, I switched to fixtures.

There’s a balance though between fixtures and creating fresh database records via ActiveRecord#create. When creating fresh records, you’re guaranteed to see records that match the application code (validations + callbacks run, default values are applied, etc). This isn’t the case with fixtures. It’s also a pain to update fixtures when logic / columns / etc. change. I looked for isolated cases – like recreating basic, seldom-changed records many times when running a test suite.

Time Savings: 35.7 seconds

In total, our test suite now takes 3 minutes to run – a huge improvement from 8 minutes.

Cleanup Unused Assets

Over the years, unused stylesheets, images, and javascript files have accumulated in our /public directory. It’s time to clear out what isn’t being used.

First step: look at our production logs for assets that have actually been requested recently.

We looked over the last week of log files—long enough to feel confident that every asset in use will have been requested. Here’s the grep command we used on Apache’s access.log:

grep -Eo ‘GET /(.*(.gif|.jpg|.jpeg|.png|.css|.js))[ \?]’ access.log > ~/assets.txt

The output of this is a long list of GET commands in assets.txt with lots of duplicate lines.

Second step: Delete assets that aren’t on that list

Andre created a quick ruby program to process assets.txt. Making this was an iterative process:

When all looked good, we uncommented the line in find_unused_assets.rb to actually delete the unused assets.

Third step: sanity check, checkin, sanity check on staging, deploy!

Unused CSS Rules, Views, database tables and columns

These involved mostly manual work:

I did make one mistake removing database columns: I forgot to run our Sinatra test suite (Sinatra handles checkins from our agents) after removing the database columns. For performance, we have some queries that select specific columns vs SELECT *. One of the columns I removed was referenced by a query in Sinatra. Double-check the entry points to your app!

Automated Tools

There are some automated tools that can help with asset cleanup:

CSS – The deadweight gem

From the README: “Deadweight is a CSS coverage tool. Given a set of stylesheets and a set of URLs, it determines which selectors are actually used and reports which can be “safely” deleted.”

Unused Partials – discover-unused-partials

From the README: “To use this script, simply run it in your RAILS_ROOT. It will return a list of unmentioned partials. It supports detection of Haml and Erb (both .erb and .rhtml) templates.”

I decided to go the manual route – we had a lot of low hanging fruit that was easy to discover w/o a separate tool.

The result: a more streamlined Scout

Our week of work resulted in some awesome diff stats:

Sometimes the first step to adding new functionality is clearing out what’s not being used – working on Scout is certainly more fun after a week of cleanup.