"App Monitoring" Posts


Overhead Benchmarks: New Relic vs. Scout

By Derek Bullet_white Posted in App Monitoring Bullet_white Comments Comments

High monitoring overhead is a silent killer: your app's requests take longer, throughput capacity shrinks, end users requests start stacking up in a request queue, you react by provisioning more servers, and finally, more servers == more $$$.

So how does Scout's overhead compare with the competition? To find out, we set up a suite of benchmarks comparing Scout's overhead to New Relic.

To ensure fair results, every part of these tests is open-source - from the Rails app we're benchmarking to the Rails log files generated by the benchmarks. We encourage you to analyze the raw data, try these benchmarks on your own, and let us know if you come to a different conclusion.

Benchmarking Scenarios

App monitoring overhead varies based on (1) instruments used and (2) available resources on the application server.

That in mind, we benchmarked agent overhead in the following scenarios:

  1. Representative endpoint - this test hits 100 endpoints in a Rails app, with each controller-action conducting 21 database queries and rendering 20 view partials.
  2. Expensive Endpoints - we simulate an app that does a lot of work to deliver a request (1k database queries and 1k view partials).
  3. Fast Endpoints - we simulate an API-like controller-action that does very little work.

In these benchmarking tests, our metric of comparison will be response time. We're benchmarking a Rails 4.2.5 application running Ruby 2.2.3.

I've put the results below. Beneath this summary, you'll find details and analysis on each benchmark. The percentages below represent the increase in response time when each agent is installed. Lower is better:

Response Time Overhead Benchmarks

Benchmark APM Agent Response Time Overhead
Representative Endpoint

21 database queries and 20 view partials per controller-action.

None 55.6 ms
New Relic 80.4 ms 44.5%
Scout 56.8 ms 2.2%
Expensive Endpoint

1k database queries and 1k view partials per controller-action.

None 2,811.1 ms
New Relic 3,871.0 ms 37.7%
Scout 2,922.8 ms 4.0%
Fast Endpoint

1 database query and no view partials per controller-action.

None 2.82 ms
New Relic 3.71 ms 32.0%
Scout 3.06 ms 8.8%

Read More →

 

NEW! Request queuing and middleware instrumentation

By Derek Bullet_white Posted in App Monitoring Bullet_white Comments Comments

request queuing and middleware

Metrics for Christmas everyone! Get your metrics! We've added more metrics to Scout App Monitoring. These metrics will give you even greater visibility into the full request cycle of your Rails app:

  • Request Queuing - judging if you have enough capacity to serve all your traffic is one of the more difficult (and important) things to track. Measuring the time spent from when a request first hits your load balancer till when your Rails app starts handling the request is a big help. Large request queuing times means traffic is piling up waiting to be served.
  • Middleware instrumentation - we now instrument the middleware used in your app as well.

Additionally, we addressed a couple of quirks:

  • Slow requests were timestamped with the time the request started. Now, they are timestamped with the time the request COMPLETES. This aligns with how other tools (like MySQL slow query logs) handle logging slow method calls.
  • Addresses some skewing issues where metrics may be associated with a time slot a minute-or-two off the actual time.
  • The old agent required a config/scout_apm.yml file even if environment variables were used. The file is no longer needed. Issues with this mostly sprang up in CI systems.
  • Solidifying support for threaded app servers (such as Puma or Thin).

Read More →

 

AcademicWorks' switch from New Relic to Scout

By Derek Bullet_white Posted in App Monitoring Bullet_white Comments Comments

academicworks

The engineering team at AcademicWorks, the leading provider of scholarship management solutions for public and private educational institutions, was frustrated. With over two million users and a datastore-heavy Rails app with hundreds of database instances, their ten-person development was feeling bogged down with the bloat of their existing application monitoring tool, New Relic.

Read More →

 

App Monitoring for the Modern Dev Team

By Derek Bullet_white Posted in App Monitoring, Updates Bullet_white Comments Comments

How we're building apps as developers is changing. Fast. The breadth of our responsibility is decreasing (yeah), yet the complexity of our code is increasing (meh).

We've just launched our app monitoring service for the modern development era.

We think it will dramatically change how you make your web apps faster - but don't take our word for it:

Scout gives us unprecedented performance insights into our web apps: down to the slow line-of-code, associated database calls, the developer that wrote it, and when the code was changed. Putting the pieces together without Scout would take hours - we're able to do it in minutes with Scout.

- Aaron Scruggs, Director of Engineering at AcademicWorks, the leading provider of scholarship management solutions for universities, community colleges, and community foundations.

How did we get here and what problems are we solving?

The full stack/full responsibility era

Just a few years ago, if you were lauching an app, you were responsible for EVERYTHING. You and your dev team deployed and maintained the full stack:

full stack

Scaling a full stack isn't trivial. It takes valuable developer/devops time to do this. We started to look at ways to cut out pieces of our stack...

The birth of services

In 2009, Amazon introduced Elastic Load Balancing and Relational Database Service (BETA). This decreased the hardware your dev team was responsible for maintaining.

full stack

We started to like these services. What other parts of our stack could we offload?

App Services Era

It didn't stop with building services for the basic infrastructure building blocks. Today there's everything from file upload to user authentication services.

app services stack

The upshot? We're responsible for less plumbing code than ever before. I'll never complain about having to write an ImageMagick or user auth system again.

As the number of app services have grown, we've been able to off-load specialized infrastructure to vendors. This means we're focused more on just our custom app code...

The acceptance of PaaS

If our infrastructure is simpler, our deploys are simpler too. It's just git push.

If you are running on a PaaS like Heroku or Cloud Foundry, your stack has just become:

paas

With our time freed up for writing custom app code, we're now writing more specialized apps. We're adding more awesome sauce.

This custom code is frequently slow.

Scout is intentionally NOT full-stack app monitoring

If we're not responsible for the uptime of the services beyond our app code, it makes a lot less sense to monitor those services. That's why Scout is the first app monitoring service that's intentionally not full-stack monitoring: the most time-consuming perfomance problem today is fixing bottlenecks in our own custom code.

If our ELB/RDS/File Upload/Web Socket service isn't performing, we'll submit a support ticket with those vendors. We won't debug them ourselves.

In short, Scout is a New Relic Alternative for modern dev teams.

Fixing slow custom code before Scout

Before Scout, fixing slow app code looked something like this:

  • Look at an overview chart of my app's performance
  • Oh there's a spike - I hope my app monitoring captured the slow request then (since they just sample)
  • If my APM tool happened to sample the right slow request, inspect it.
  • Ok - I don't think I wrote this code. Who on my dev team might know about it?
  • How long has this slow code been around?

Tracking people down is hard. I'll wait on this for now.

Fixing slow code with Scout

We've gone beyond code metrics: Scout adds critical pieces to make it easier to track down problems and find the person on your team best equipped to fix slow code.

What you used to suck up hours of dev debugging time now takes just minutes:

apm overview

3 Key Features

  • Don't miss a slow request - most app monitoring services just sample slow requests. It's almost 2016: we capture every one. It's the slow requests that help us learn the most about our app's performance.
  • Git Integration - we git blame your slow code so you can quickly find out who wrote it and when.
  • Context - when you have a slow request, it's super helpful to know the email address of the user that triggered it, which database shard it hit, their monthly spend, etc. Easily add this with our API.

No-Haggle Pricing. No Contracts.

We're confident Scout stands on its own merits: that means no pricing tricks or contracts. Try app monitoring free for 14 days.

App Monitoring for the Modern Dev

As Scout customer Aaron Scruggs of AcademicWorks says:

"We liked New Relic, but we love Scout."

We're committed to building the first APM service that's built for you, the Heroku-deploying, AWS service-using, fast-moving developer.

Questions? Just email us at apm.support@scoutapp.com.

 

App Monitoring: Stability, Stability, Stability ... and pricing.

By Derek Bullet_white Posted in App Monitoring Bullet_white Comments Comments

We're officially in the Stability Phase™ of app monitoring: slamming our site with traffic, trying to break things, and adding resiliency for the spots that need it.

Read on for an updated General Availability date, our planned stability improvements, and pricing information.

Read More →

 

Git-integrated app monitoring

By Derek Bullet_white Posted in App Monitoring Bullet_white Comments Comments

Tracking down slow code is about to get a lot faster. We're excited to debut our Github Integration for Scout Application Monitoring:

github integration

With the Github Integration enabled, instead of seeing backtraces for slow method calls (>= 500 ms), you'll see the relevant code in our UI. This goes beyond the code: you'll also see who wrote it and when it was committed. Fix performance problems faster by assigning them to the developer most familiar with the code.

Read More →

 

Older posts: 1 2