Overhead Benchmarks: New Relic vs. Scout

By Derek Bullet_white Posted in App Monitoring Bullet_white Comments Comments

High monitoring overhead is a silent killer: your app's requests take longer, throughput capacity shrinks, end users requests start stacking up in a request queue, you react by provisioning more servers, and finally, more servers == more $$$.

So how does Scout's overhead compare with the competition? To find out, we set up a suite of benchmarks comparing Scout's overhead to New Relic.

To ensure fair results, every part of these tests is open-source - from the Rails app we're benchmarking to the Rails log files generated by the benchmarks. We encourage you to analyze the raw data, try these benchmarks on your own, and let us know if you come to a different conclusion.

Benchmarking Scenarios

App monitoring overhead varies based on (1) instruments used and (2) available resources on the application server.

That in mind, we benchmarked agent overhead in the following scenarios:

  1. Representative endpoint - this test hits 100 endpoints in a Rails app, with each controller-action conducting 21 database queries and rendering 20 view partials.
  2. Expensive Endpoints - we simulate an app that does a lot of work to deliver a request (1k database queries and 1k view partials).
  3. Fast Endpoints - we simulate an API-like controller-action that does very little work.

In these benchmarking tests, our metric of comparison will be response time. We're benchmarking a Rails 4.2.5 application running Ruby 2.2.3.

I've put the results below. Beneath this summary, you'll find details and analysis on each benchmark. The percentages below represent the increase in response time when each agent is installed. Lower is better:

Response Time Overhead Benchmarks

Benchmark APM Agent Response Time Overhead
Representative Endpoint

21 database queries and 20 view partials per controller-action.

None 55.6 ms
New Relic 80.4 ms 44.5%
Scout 56.8 ms 2.2%
Expensive Endpoint

1k database queries and 1k view partials per controller-action.

None 2,811.1 ms
New Relic 3,871.0 ms 37.7%
Scout 2,922.8 ms 4.0%
Fast Endpoint

1 database query and no view partials per controller-action.

None 2.82 ms
New Relic 3.71 ms 32.0%
Scout 3.06 ms 8.8%

Read More →


State of the 2016 Rails Stack

By Derek Bullet_white Comments Comments

What's a Rails stacks look like in the wild these days? Most popular Rails version? Most popular Ruby? Is Delayed Job still hanging on?

If you're curious about the above, you've come to the right place. We collect gems used on the apps we monitor at Scout to assist with debugging issues and to prioritize libraries we want to instrument. This data set is across many hundreds of apps - while a small chunk of the Rails ecosystem, it's enough to shed light on usage.

Without delay, lets dig in.

Read More →


NEW! Request queuing and middleware instrumentation

By Derek Bullet_white Posted in App Monitoring Bullet_white Comments Comments

request queuing and middleware

Metrics for Christmas everyone! Get your metrics! We've added more metrics to Scout App Monitoring. These metrics will give you even greater visibility into the full request cycle of your Rails app:

  • Request Queuing - judging if you have enough capacity to serve all your traffic is one of the more difficult (and important) things to track. Measuring the time spent from when a request first hits your load balancer till when your Rails app starts handling the request is a big help. Large request queuing times means traffic is piling up waiting to be served.
  • Middleware instrumentation - we now instrument the middleware used in your app as well.

Additionally, we addressed a couple of quirks:

  • Slow requests were timestamped with the time the request started. Now, they are timestamped with the time the request COMPLETES. This aligns with how other tools (like MySQL slow query logs) handle logging slow method calls.
  • Addresses some skewing issues where metrics may be associated with a time slot a minute-or-two off the actual time.
  • The old agent required a config/scout_apm.yml file even if environment variables were used. The file is no longer needed. Issues with this mostly sprang up in CI systems.
  • Solidifying support for threaded app servers (such as Puma or Thin).

Read More →


AcademicWorks' switch from New Relic to Scout

By Derek Bullet_white Posted in App Monitoring Bullet_white Comments Comments


The engineering team at AcademicWorks, the leading provider of scholarship management solutions for public and private educational institutions, was frustrated. With over two million users and a datastore-heavy Rails app with hundreds of database instances, their ten-person development was feeling bogged down with the bloat of their existing application monitoring tool, New Relic.

Read More →


4 ways to get the most out of your Rails logs

By Derek Bullet_white Comments Comments

Logging is the lowest common denominator of monitoring - it's low effort and low overhead. You can put ANYTHING into a log file (sans animated GIF memes - save that for Slack).

The downside of being a logging addict: logs can quickly become a noisy mess. Digging through logs while debugging a critical issue can become a hair-pulling experience.

Don't fret - there are some terrific tools available today to turn your log streams into beautiful, useful data sources.

Read More →


Pillars of the Rails Monitoring Stack: 2018 Edition

By Derek Bullet_white Posted in HowTo Bullet_white Comments Comments

Here's a behind-the-scenes rundown of how we ensure our apps are in peak condition in 2016* 2018.

*Ed. Note- this is largely unchanged from 2016, so I've updated this post with our 2018 stack.

Focus Service Approx. Price
App MonitoringScout$150/mo
Log MonitoringELK Stack$150/mo
Server & Service Monitoring
Custom Metrics
Pingdom Server Monitor$200/mo
Exception MonitoringSentry$30/mo
Scheduled Job MonitoringDeadman's Snitch$15/mo

Stop searching for a single tool

There's no single, do-everything tool that completely monitors a modern-day Rails stack. If there was, it'd be the software equivalent of the Homer Simpson-designed car. There's simply too many specialized things to put into a single monitoring app.

However, there's good news: a number of best-of-breed services play well together to give you great monitoring coverage of your Rails apps and infrastructure.

When picking a monitoring solution, you can typically choose between two options:

  1. Open Source
  2. SaaS

The upsides of open source: free to install and more customizable. The downsides: generally more difficult to use and fairly complex to maintain.

Most of the monitoring services we use are SaaS. We typically only use open source options when the paid, hosted option is significantly cost prohibitive. Monitoring software is complicated and keeping your own stack running can be a time-sink. The last thing we want is unreliable software monitoring our apps.

Read More →


Older posts: 1 ... 13 14 15 16 17 ... 70