One change that lowered our DB I/O by 80%

By Derek Bullet_white Posted in Development Bullet_white Comments Comments

When a server reports data to Scout, a small bit of data is updated in a MySQL MyISAM table. Each of these updates doesn’t change much data:

UPDATE `plugins` SET `last_reported_at` = '2010-11-17 19:53:00' WHERE `id` = 999999

I assumed that only the updated data was written to disk. I was wrong: Jesse Newland of Rails Machine pointed out that updating a row in a MyISAM table rewrites the entire row to disk (not just the updated columns).

These updates were expensive: we were updating small bits of data, but our table contained a large text column that accounted for 75% of the data stored in the table. Every update to a row in our plugins table would rewrite the large text column (even though the text column wasn’t changing).

Read More →


Now this is a GUI

By Andre Bullet_white Comments Comments

Scout customer RailsWare has implemented one of the best front-ends we’ve seen for Scout notifications—a real, physical stoplight on the wall of their offices:

Railsware constructed this awesome setup for their client, RatePoint. Yaroslav Lazor outlined the nuts and bolts:

  • Notifications from several monitoring services (Scout, Pingdom, New Relic) are collected in an IMAP folder.
  • A daemon monitors the inbox, and turns on the stoplight if necessary. The daemon also writes status to a database, where web-based and desktop-based notifications can also read and display status.
  • When an issue is marked resolved, the daemon turns off the stoplight—everything is back to normal.

We love the idea of seeing a big, bold, physical presentation of system status up on the wall. Great job RailsWare and RatePoint for putting this together!


A redesigned dashboard & server groups

By Derek Bullet_white Posted in Features Bullet_white Comments Comments

Answering “how is my cluster performing?” just got a lot easier. We’ve rolled out a redesigned account dashboard that makes filtering the noise in your server cluster a lot more efficient.

The dashboard

The updated account homepage lists each server and 3 key metrics:

  • Server load over the last minute
  • Memory usage
  • Disk usage

The sidebar shows any active alerts.

Server Groups

Create groups (ex: web servers) and place servers in those groups. These grouped servers are shown together on the account dashboard.

Timeframe selection

By default, the dashboard shows the most recent data across servers. You can also choose to display aggregated metrics across a longer period (up to the previous 7 days). We’ve worked hard to make toggling between different timeframes speedy so comparing data should be painless.


Our previous dashboard had too much noise. It was difficult to tell where the emergencies were. We’ve found this update makes checking the status of our own cluster at Scout far more efficient.

Past Feature Updates


A lesson from my favorite TV movie

By Derek Bullet_white Posted in Business Bullet_white Comments Comments

Duel, Steven Spielberg first film, had an estimated budget of $450K. For comparison, the opening scene in Spielberg’s Saving Private Ryan had a budget of $12 million. However, I’m just as happy watching Duel. It’s the only television movie I’m proud to talk about in public.

Spielberg on his early work[1]:

I could never go back and make those early films as well as I made them, when I was of the appropriate age and naivety to be working on subjects like that. But also, l couldn’t have made Schindler’s List or Private Ryan. It’s a fair trade-off.

There’s a lot to learn from Spielberg. Instead of trying to imitate the engineering feats of giants like Google, Apple, and Facebook, smaller companies like us are better off looking for inspiration in things that do a lot with a little.

We can deliver as good of an experience as the blockbuster apps – but it’s going to be in a different way.

[1] – From an interview on the Duel (Collector’s Edition) DVD


Where are the Rails infrastructure support firms?

By Derek Bullet_white Posted in Development, Business Bullet_white Comments Comments

There are no formal documents to sign if you and your spouse decide to have children. You don’t have to sit through an accreditation class. There is no credit check. You don’t need a high school diploma. Procreation can even happen accidentally.

A baby is a lot like a Rails application: the problem is caring for it, not creating it.

Five years ago, the typical Rails stack was just a couple of pieces: Apache/Mongrel, Rails, and MySQL. While Rails is remarkably similar to its original form even today, the stack around it is dramatically more diverse. We’re deploying to automated infrastructures, using NoSQL databases, messaging systems, queuing systems, and more. With the increased complexity of web applications, I’m surprised we’re not seeing companies dedicated to 24/7 infrastructure support: it doesn’t matter where your app is hosted, they manage it.

Read More →


Apple's Hidden Genius

By Derek Bullet_white Comments Comments

James Duncan Davidson on support at the Apple Store:

…one of the secrets of Apple’s product design in the last decade has been the use of the data that these stores generate…it’s the data that’s generated at the Genius bar that fascinates me. This data, in aggregate, can tell Apple a lot about what machines break, how they break, and after how long in a much more direct way than what would come out of a third party service center.

While it’s speculation, it’s reassuring to think that much of Apple’s product polish comes from a close connection to customers and not just the vision of Steve Jobs/Jonathan Ive.

At Scout, we care a lot about this connection. However, most customers don’t provide feedback. To combat this, I’ll review the activity of our new customers each week. I’ll email them if I see a problem and make a note of any issues. Over time, taking care of these unreported issues adds up to a polished Scout experience.


Older posts: 1 ... 33 34 35 36 37 ... 68