High monitoring overhead is a silent killer: your app's requests take longer, throughput capacity shrinks, end users requests start stacking up in a request queue, you react by provisioning more servers, and finally, more servers == more $$$.
So how does Scout's overhead compare with the competition? To find out, we set up a suite of benchmarks comparing Scout's overhead to New Relic.
To ensure fair results, every part of these tests is open-source - from the Rails app we're benchmarking to the Rails log files generated by the benchmarks. We encourage you to analyze the raw data, try these benchmarks on your own, and let us know if you come to a different conclusion.
App monitoring overhead varies based on (1) instruments used and (2) available resources on the application server.
That in mind, we benchmarked agent overhead in the following scenarios:
- Representative endpoint - this test hits 100 endpoints in a Rails app, with each controller-action conducting 21 database queries and rendering 20 view partials.
- Expensive Endpoints - we simulate an app that does a lot of work to deliver a request (1k database queries and 1k view partials).
- Fast Endpoints - we simulate an API-like controller-action that does very little work.
In these benchmarking tests, our metric of comparison will be response time. We're benchmarking a Rails 4.2.5 application running Ruby 2.2.3.
I've put the results below. Beneath this summary, you'll find details and analysis on each benchmark. The percentages below represent the increase in response time when each agent is installed. Lower is better:
Response Time Overhead Benchmarks
|Benchmark||APM Agent||Response Time||Overhead|
21 database queries and 20 view partials per controller-action.
|New Relic||80.4 ms||44.5%|
1k database queries and 1k view partials per controller-action.
|New Relic||3,871.0 ms||37.7%|
1 database query and no view partials per controller-action.
|New Relic||3.71 ms||32.0%|