In Blink, Malcom Gladwell’s book on split-second decisions, Gladwell tells the story of how the Emergency Department at Chicago’s Cook County Hospital changed their process for diagnosing chest pain.
Dr. Brendan Reilly instituted a simple test to determine whether a patient was suffering from a heart attack. It combined just 4 questions with the results of an ECG. This simple test was 70% better than the barrage of questions previously asked by hospital staff to identify patients that weren’t having a heart attack and was nearly 20% better at identifying patients that were having heart attacks.
More information on the patient’s symptoms often led to an incorrect diagnosis. It distracted doctors from the real issues.
I’ve seen this many times with developers trying to debug performance issues in Rails applications. They look at outliers instead of the obvious culprits. It’s part of the reason I’ve never felt a need for a deep, detailed Rails monitoring application (i.e. – benchmarks from the controller to the database level on every request).
The majority of the time, our performance problems have nothing to do with the Rails framework (and we’ve worked through a lot of issues since we started building Rails apps in 2005). Why benchmark the entire request cycle when the vast majority of issues are isolated at the database layer? After I’ve ruled out the database, I can see benchmarking a single request (there’s a great free tool below), but I simply don’t want the other, often irrelevant information clouding my mind.
The root symptom we want to avoid in our apps is slow requests. Our Scout plugin for analyzing slow Rails requests has been installed nearly 250 times, so we’re not alone there.