Bigger Hardware = Downtime today, 1PM PST

By Derek Bullet_white Comments Comments

We’re upgrading our infrastructure and Scout will be down from approximately 1PM – 1:30 PM PST.

Sorry for any inconvenience this may cause.

 

Solving the random Rails performance problem

By Derek Bullet_white Comments Comments

Last week I uncovered a tricky performance problem on Scout I wanted to share.

In seemingly random fashion, when accessing my account home page, the render time was 5-6 seconds – way too slow. Other times, the render time would be far less than a second.

Here’s what I did to debug:

  1. Tailed my production log
  2. Opened up the mysql console
  3. Generated a slow request
  4. Ran the mysql SHOW PROCESSLIST command

Viola!

Plain as day, I saw a query taking 30+ seconds. A simple mysql EXPLAIN on the query revealed a missing index. A new index fixed the issue. This query was outside the Rails application – a background job – so I never saw the query when optimizing the application.

Checkout the impact on the DB server load:

It’s a great lesson that you really need the complete picture of your infrastructure when something goes wrong – just looking at the Rails application often doesn’t cut it.

P.S. You’ll love what’s coming up with Scout – finding these problems gets a lot easier.

Related:

 

What's in store for 2009 - beyond the numbers

By Andre Bullet_white Comments Comments

2008 was a year full of numbers – 34% (fall of the the Dow Jones Industrial Average), $1.84B (Apple iPhone revenue), and $100M (cost of the Beijing Olympics Opening Ceremony).

But raw numbers are just the start – in 2009, our goal is to tell you what they mean:

  • Your Rails application is slow. Now what? We’re bringing deeper, more actionable Rails instrumentation .
  • You’ve got one hell of a memory leak. What’s going on? We’ll help you get a complete picture of your system right when the leak happens.
  • You’re on the ski slopes but want to stay abreast of trends. We’ll help you get beyond raw numbers and alert you on trends.

And besides the functionality, there’ll be a new name for Scout in Q1!

Stay tuned for details in the coming weeks (or just subscribe to our RSS feed).

 

Say hey at Merb Day

By kevin Bullet_white Comments Comments

Merb Day Atlanta – a “one-day, one-track conference for Ruby and Rails developers looking to learn Merb” – is this Saturday and Scout’s extremely proud to be one of its sponsors.

Our own Charles Brian Quinn and Matt Todd will be there, taking in the sights, and Matt will also be talking some code – he’s teaching a three-part tutorial with Michael Ivey on learning, using, and hacking Merb.

Be sure to say hey if you get the chance!

 

A simple Rails performance tuning workflow

By Derek Bullet_white Comments Comments

I really like Dan Mange’s workflow for Rails performance tuning:

  1. Use curl for benchmarking. It’s good enough (and damn simple).
  2. Use the ruby-prof gem along with an optional request parameter so you can profile any action, any time.

Dan also does a solid job explaining CPU time and what if means for your Rails app. A snippet:

At CarePages we have an Apache log that logs the controller, action, and Rails runtime. We run a script that groups the runtimes by controller and action, and then calculates the total runtime for a controller/action divided by the total runtime for all requests. What we found the first time we did this was that one of our pages was taking nearly 40% of the overall CPU time. It wasn’t the slowest page, but it was the most frequently hit page. We had two other pages that were taking about 19% and 13% of the overall CPU time. This gave us a good place to start – if we could make the 40% page twice as fast, we would lower our overall CPU usage by 20%.
 

The 3 pillars of our Rails Monitoring Stack

By Derek Bullet_white Posted in HowTo Bullet_white Comments Comments

Life as a cross-dresser must be unnerving. I could buy a wig, put some makeup on my adam’s apple. Buy a skirt. Does a sock-filled bra work for my bosom?

With a bit of mood lighting, I might actually look like a woman, but it would be difficult to relax. Knowing my manhood could be exposed in a multitude of ways – some stubble on my chin, hair on my legs, forgetting to make my voice less deep, etc. would leave me on-edge.

In some ways, I feel like I identified with cross-dressers before we solidified our Rails monitoring stack over the past year. I just felt vulnerable – that one wrong move could send our Rails stack tumbling.

We’ve standardized our setup, and I thought I’d share the 3 tools we’re using that make me feel much less like a cross-dresser these days.

The 3 pillars of our Rails monitoring stack

We break Rails monitoring into the 3 parts below (along with the tools we use):

Process Monitoring

System Performance

Exception Notifications

skitched-20080822-140434.jpg

“The Diaper”

A safeguard for emergencies
skitched-20080822-140537.jpg

“The Nerves”

Preventing future problems
skitched-20080822-140602.jpg

“The Megaphone”

A loud voice when your app is breaking
Monit Scout Exception Notification plugin
Ensure Mongrels are running & restart leaking processes Catch disturbing trends before they become problems – disk space usage, server load, slow requests, etc. Organizing and collecting application exceptions


So, to be clear, even though we built Scout, we use a combination of tools to make sure our Rails apps are running.

Read More →

 

Older posts: 1 ... 45 46 47 48 49 ... 68