Most Rails engineers know the basics of database performance. They know that if a query is slow, an index may be the solution. Some know the trade-offs between having and not having an index. Or why an index on a low-cardinality column might not help. But everyone is surprised when I show them a few more advanced indexing techniques. The only response I get is Wow, I didn't know that's possible! In this article, I'll show you 3 techniques that render this kind of response.
Premature optimization is the root of all evil.
In my line of work, it is sometimes invoked as a sort of apology; an excuse for why more time wasn't spent on performance: "This sucks, but at least we didn't.....prematurely optimize!"
My job is to fix performance issues and help other developers write well-performing code, so I'm not a huge fan of this quote. We shouldn't let an out of context quote guide our development strategy, no matter the source!
When things bother me, I like to stop to take a closer look. Here's the full quote:
We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil.
I translate this to: "When optimizing, don't get distracted by things that don't matter."
I would never translate this to: "Hold off on optimizing things until it really hurts."
Let's get the confusing part out of the way: we're going to get a little "meta". This post is about using one performance tool (Stackprof) to improve another performance tool (Scout's Rails Agent) so Scout can diagnose a common performance pitfall (N+1 queries).
Did I just put "performance" in a sentence three times? You can see I'm a little obsessed.
There's plenty written about why N+1 queries are bad. Is there anything good about an N+1?
Yes! Finding an N+1 is like finding a $20 bill in your couch. It's a performance-enhancing gift and often an easy fix - assuming you can find the spot in your code that is emitting the query.
...and there-in lies the problem. It is difficult to trace problematic N+1 queries on complex apps running in production back to a line-of-code. In fact, I don't know of a tool today you can run in production that does this. Why is it difficult? Grabbing backtraces is not cheap - you can't call
Kernel#caller all willy-nilly in your instrumentation. It means you are left with deadends like this when trying to track down an N+1:
In the above screenshot, each
ArDescriptor#find query executes quickly (22 ms), but there are 23(!) queries with the same SQL, adding up to nearly 500 ms.
Now, I don't know about you, but I find this incredibly frustrating. Armed with Stackprof, we set out to see: could we make it performant to find the source of N+1s with Scout?
Benjamin Knofe, CTO of Gigmit, sat down with us to share how they use Scout to improve their performance.
What is Gigmit?
Gigmit is a platform to connect music professionals to one another. Artists get opportunities to play gigs, shows, and festivals all over the world by showcasing their music and applying for gig listings created by promoters.
How big is your userbase?
We support 30,000 artists and 2,500 promoters world-wide. We also offer a suite of tools for all things involved in live event management: booking, merchandise, ticketing, and promotions.
Teaser for a soon-to-be released capability:
Overview metrics for all your background jobs
Chart throughput, latency, error rate, and more.
Detailed drill-down on slow jobs
See what's making the job slow, identify N+1 queries, and more.
Tech preview - want in?
Background job monitoring currently supports sidekiq, and is tech preview. Contact your account manager to enable it for your account.
Most Ruby developers use Git for their version control system of choice. Git is a wonderful tool that can save you countless hours of lost productivity and makes collaborating with others a cinch. Git's distributed nature also allows devs to work anywhere with or without an internet connection without fear of losing work.
With all of the power that Git provides also comes some complexity. Newcomers to Git may find it's syntax somewhat unintuitive and clunky at first. Even Git power users often have to lookup lesser known Git subcommands and options from time to time. Git is similar to a Swiss Army knife in that it contains many useful tools, but can take some time to master or even find a use for them all.
The goal of this post it to describe some lesser known features of Git and how you can leverage these to improve your Git workflow.
Let's get started.