Introducing GitHub-enhanced Deploy Tracking

By Derek Bullet_white Comments Comments

We're happy to introduce a lightweight addition to Scout: GitHub-enhanced deploy tracking.


​Deploy tracking makes it easy to correlate deploys to your app's performance. Scout's deploy tracking goes beyond chart markers: enable our GitHub integration to see which branch or tag was deployed, the team members that contributed, and a diff summary of what changed.

Enable deploy tracking

The latest version of the scout_apm gem tracks your deploys without additional configuration if you are running Capistrano. If you aren’t using Capistrano or deploying your app to Heroku, see our deploy tracking configuration docs.

Also

  • DelayedJob - we've added DelayedJob instrumentation, which provides the same level of detail on DelayedJob as you've come to expect on your web requests. Update to the latest agent version to enable.
  • ScoutProf - looking for more insights on your traces? Try our ScoutProf beta.
  • Heroku Addon - featured in last month's Heroku newsletter, the Scout Addon makes provisioning Scout on Heroku even easier.
 

DevTrace and the Art of Staying the F*** Out of the Way

By Andre Bullet_white Comments Comments

DevTrace is a performance widget for your Rails applications in development. It sits unobtrusively in the corner of your page, just waiting to drop insight on your application:

See stack traces, SQL timings, and more with just a click!

This kind of insight is powerful. You can see how your app executes, and head off performance issues before they get to production.

But what I want to cover here is how DevTraces stays out of the way -- how it's simultaineously powerful and unassuming.

1) Installed via Gem only

There's only one thing to install, which is a gem in your application's Gemfile. Earlier iterations needed a Chrome extension to instrument background Ajax requests. But installing the chrome extension was annoying, and it limited who could use DevTrace. So we figured out a way to instrument Ajax calls in Javascript only, by adding hooks to the browser's XMLHttpRequest prototype.

2) Visually Lightweight

DevTrace shows up as a small, semi-transparent badge displaying your page load time. It's placed in the lower left corner of your page, to stay out of the way of navigational elements. Since your eyes natually move from the upper-left to the lower-right of a page, DevTrace is only there if you look for it. Finally, we added subtle animations for Ajax requests -- enough to notice, but not enough to be distracting.

3) HTML Hygene

HTML hygene (AKA staying the f* out of your markup as much as possible) is important. We know that — as a developer — you'll be inspecting your source, and don't want to see extra cruft from your performance tools.

DevTraces injects as little as possible into your page (we do have to add something though):

  • a <script> tag to load the needed JavaScript
  • a <style> tag, with classes carefully scoped to our namespace
  • finally, for HTML pages, a JSON data structure with the trace information for the page

4) Keeping Ajax / JSON Payloads Clean

Modern apps do lots of work via post-pageload Ajax requests, and visibility into those requests is important. DevTrace instruments those calls as well. However, we can't add profiling information to the payload itself - it would stomp on the expected payload structure. So we have a sneaky side-channel to move traces down to your browser in this case: we add a custom header to Ajax responses containing the trace information, then intercept that header & grab the payload in Javascript. It's all transparent to you as a user, and it provides critical visibility.

6) Staying out of your Javascript Namespace

Nobody wants to include a tool which litters the Javascript namespace with variables and functions. DevTrace puts exactly one variable into your addressible namespace, which is a reference to a closure containing the entire DevTrace implementation.

The DevTrace widget does a lot visually: rendering its UI, animating the badge on incoming AJAX requests, etc. To enable this UI goodness without polluting your namespace, we place a copy of MinifiedJS (a very lightweight jQuery-type library) inside the closure. MinifiedJS helps make the rest of the widget code concise and maintainable. Completely encapsulating it inside our closure means it can't interfere with other libraries or JS code on your page. Another key benefit is DevTrace makes no assumptions about which Javascript library your app uses.

6) Development Environment Only

Profiling in development is different from profiling in production. Always-on tracing works in development, because traffic is limited to just you (the developer), and overhead is less of a concern. In contrast, production environments require a smart sampling approach (which we're perfecting in Scout APM). No need to worry about leaving DevTrace on in the wrong environment: even if you accidently configure it for production, it won't even insert its rack middleware into your application in a non-development environment.

Staying out of the Way While Giving you What you Need

A Javascript widget needs to be a good houseguest. It doesn't put its feet up on the furniture or spread its stuff out all over the kitchen. DevTrace goes to great lengths to be a welcome guest in the host application, while also giving you what you need: performance traces at your fingertips, line-of-code tracing, memory usage metrics, and insights on database queries.

Learn more about DevTrace here.

 

How we're turbo-charging our traces with ScoutProf

By Derek Bullet_white Comments Comments

The transaction trace is application monitoring. If you can't track a performance problem to a line-of-code, you aren't app monitoring.

However, there's a gaping hole in the quality of transaction traces today.

We've been instrumenting the quality of our traces since our launch, and noticed that 46% of the time spent in an average Rails request is in custom application code. This falls outside the instrumentation that we, and every other app monitoring service, like New Relic, provides.

Let's fix that.

It started with Stackprof

The release of Ruby 2.1 introduced a C-api for fetching ruby backtraces. What's great about this API? Here's Aman Gupta of GitHub:

The (rb_profile_frames()) api performs no allocations and adds minimal cpu overhead making it ideal for profiling, even in production environments.

Amon wrote Stackprof, a terrific call-stack profiler that leverages rb_profile_frame(). We used it extensively to improve the performance of our agent.

However, with all of that visibility comes a problem: Stackprof output is difficult to decipher.

Enter ScoutProf

ScoutProf builds on StackProf: we're aiming to make all of that data easily decipherable plus work in threaded environments (like Puma). The key part to making it decipherable and actionable? The aggregation. From our docs:

ScoutProf attempts to sample your application every millisecond, capturing a snapshot backtrace of what is running in each thread. For each successful backtrace captured, that is a sample. Later, when we process the raw backtraces, identical traces get combined and the sample count is how many of each unique backtrace were seen.

Screenshot please?

Let's take a look at a trace from New Relic. Notice 2/3rds of the time is spent in "Controller":

nr trace

If you inspect the detailed trace, you'll see the following:

nr trace

Sad face :). That's a big black box of time.

Now, here's a trace to that same controller-action with ScoutProf:

screenshot

The mystery of "Controller" time revealed: we're seeing a breakdown of time spent in our custom code with zero configuration.

The big consumer above is ChartPartData#descriptors. What's going on there?

screen

We're seeing the overhead of ActiveRecord: all that ORM doesn't come for free. Now, who knows about this code? Let's find out with Scout's GitHub integration:

screenshot

Looks like I should talk to Andre :).

Improving the performance of complex Ruby code is time-consuming: it frequently involves the coordination of several people on your team. We're hoping to make it easier to identify these bottlenecks and the humans that can solve them.

Using ScoutProf

Our instructions for enabling ScoutProf are on our help site. Note that ScoutProf is in BETA.

 

Know of an open-source Rails app in need of some performance love?

By Derek Bullet_white Comments Comments

We've just released the BETA version of our new Ruby Profiler (Stackprof-inspired, but hopefully more readable). To really give it a go, we're looking for some open-source Rails apps that are need some performance love.

If you know of - or our a contributor to - an open-source Rails app that needs some help with slow Ruby, ActiveRecord query optimizations, etc, reach out!

 

New Release: Memory Bloat Detection

By Derek Bullet_white Posted in Features Bullet_white Comments Comments

When your app is experiencing memory bloat - a sharp increase in memory usage due to the allocation of many objects - it's a particularly stressful firefight.

Slow performance is one thing. Exhausting all of the memory on your host? That can bring your application down.

Memory bloat is a scary problem: our 2.0 release of the Scout agent aims to track down memory hotspots as easy as performance bottlenecks.

Read on: I'll share what's hard about memory bloat detection, Scout's approach, and some further reading.

Read More →

 

Faster PostgreSQL Searches with Trigrams

By Greg Bullet_white Comments Comments

There's nothing quite like having a "tool-belt" full of tricks for getting the most performance out of your Rails app. This week, Rails Postgres Guru Greg Navis shares another powerful tip.

A few months ago, I was working on a project that had about 100,000 users. Each user could have multiple names, emails, phone numbers and addresses associated with his account. All these objects totaled to about 500,000 database rows. Customer support staff would search this dataset multiple times a day. One day, they started seeing timeouts.

In this article, I'll show you how I sped things up with the PostgreSQL trigram extension.

Read More →

 

Older posts: 1 2 3 ... 58