There's nothing quite like having a "tool-belt" full of tricks for getting the most performance out of your Rails app. This week, Rails Postgres Guru Greg Navis shares another powerful tip.
A few months ago, I was working on a project that had about 100,000 users. Each user could have multiple names, emails, phone numbers and addresses associated with his account. All these objects totaled to about 500,000 database rows. Customer support staff would search this dataset multiple times a day. One day, they started seeing timeouts.
In this article, I'll show you how I sped things up with the PostgreSQL trigram extension.
Today we're excited to announce the release of our latest Ruby on Rails monitoring agent and a major UI update. This release continues our focus on quickly revealing your path to a faster, more reliable Rails app.
Determining how to make your Rails app faster often involves investigating the opposite: inspecting your app's worst performing controller-actions and their requests. We're working to make this investigation effortless. Just follow your intuition when you see a problem area on a chart:
Click-and-drag on the chart to reveal a list of endpoints we've collected detailed transactions traces for during that period. For context, you'll see a sparkline of each endpoint's response time. Sort by slowest response time or largest memory increase.
Most Rails engineers know the basics of database performance. They know that if a query is slow, an index may be the solution. Some know the trade-offs between having and not having an index. Or why an index on a low-cardinality column might not help. But everyone is surprised when I show them a few more advanced indexing techniques. The only response I get is Wow, I didn't know that's possible! In this article, I'll show you 3 techniques that render this kind of response.
Donald Knuth wrote an often quotedpaper in the 70s which is still referenced when talking about performance in web apps today.
Premature optimization is the root of all evil.
In my line of work, it is sometimes invoked as a sort of apology; an excuse for why more time wasn't spent on performance: "This sucks, but at least we didn't.....prematurely optimize!"
My job is to fix performance issues and help other developers write well-performing code, so I'm not a huge fan of this quote. We shouldn't let an out of context quote guide our development strategy, no matter the source!
When things bother me, I like to stop to take a closer look. Here's the full quote:
We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil.
I translate this to: "When optimizing, don't get distracted by things that don't matter."
I would never translate this to: "Hold off on optimizing things until it really hurts."
Let's get the confusing part out of the way: we're going to get a little "meta". This post is about using one performance tool (Stackprof) to improve another performance tool (Scout's Rails Agent) so Scout can diagnose a common performance pitfall (N+1 queries).
Did I just put "performance" in a sentence three times? You can see I'm a little obsessed.
There's plenty written about why N+1 queries are bad. Is there anything good about an N+1?
Yes! Finding an N+1 is like finding a $20 bill in your couch. It's a performance-enhancing gift and often an easy fix - assuming you can find the spot in your code that is emitting the query.
...and there-in lies the problem. It is difficult to trace problematic N+1 queries on complex apps running in production back to a line-of-code. In fact, I don't know of a tool today you can run in production that does this. Why is it difficult? Grabbing backtraces is not cheap - you can't call Kernel#caller all willy-nilly in your instrumentation. It means you are left with deadends like this when trying to track down an N+1:
In the above screenshot, each ArDescriptor#find query executes quickly (22 ms), but there are 23(!) queries with the same SQL, adding up to nearly 500 ms.
Now, I don't know about you, but I find this incredibly frustrating. Armed with Stackprof, we set out to see: could we make it performant to find the source of N+1s with Scout?