Data processing slowing you down? Try a teaser check

By Derek Bullet_white Posted in Development Bullet_white Comments Comments

Your cluster of servers ping Scout every minute, but you may only login to our website once a day. There’s a large gap between the amount of work on the data-processing side and the number of times it results in a visit to Scout. It’s a typical profile of a data processing application.

For a write-heavy application like Scout, hardware costs can increase linearly with user growth. It doesn’t scale like a read-heavy web application that can leverage a fast, in-memory caching layer for frequent reads. You need to make writing data more efficient.

For us, teaser checks have dramatically decreased the time spent processing data.

Read More →


Announcing Redwood: Spotlight-like search for your web apps

By Andre Bullet_white Comments Comments

Our knowledge at Scout is spread across web apps: Gmail, Basecamp, Delicious, Dropbox, and more. It’s become more cumbersome to find information as our pile of content has grown.

We saw two options: devote more time to organizing our information or make it easier to search through it.

Spending more time organizing our stuff didn’t sound appealing. When we’re documenting something we don’t know if it’s going to grow into a frequenty referenced source or never get viewed again. We don’t want content creation to become a burden.

Today we’re releasing Redwood, a tool to quickly search across web apps, right from your Mac OSX Desktop. It works a lot like Spotlight.

You can try Redwood for free. For now you can search your Basecamp account. We’ll be adding additional web apps in a paid version.

Just like with Scout, we’ll be iterating on Redwood. Don’t hesitate to share your feedback with us.


Monitoring mod_pagespeed

By Derek Bullet_white Posted in Plugins Bullet_white Comments Comments

Josh Nichols of Rails Machine has developed a Scout plugin for monitoring mod_pagespeed, Google’s Apache 2.x module for performing on-the-fly optimization in the Apache 2 HTTP Server.

mod_pagespeed has several filters that optimize a web page’s resources. These filters combine CSS files, optimize images, and more. mod_pagespeed is still under heavy development (a couple of the filters aren’t working for Josh, including image optimization). These issues shouldn’t impact the plugin’s metric reporting.

To install the plugin, just click the button when viewing a server on Scout. mod_pagespeed is listed in our plugin directory.


"I expected it to …" feature requests

By Derek Bullet_white Posted in Development Bullet_white Comments Comments

When I bike into work, I coast down a decent hill. With gentle curves and smooth pavement for 99% of the decent, it’s a refreshing way to start the day. My one complaint: there’s a 2×2 ft. section in the final turn with bad pavement.

This tiny imperfection breaks up the natural flow: I need to remind myself of it before I descend. It’s a lot like my favorite feature requests: those that begin with “I expected it to …”.

These requests often hint at tiny imperfections our users experience, but live with. When fixed, it makes an entire workflow seem natural.


One change that lowered our DB I/O by 80%

By Derek Bullet_white Posted in Development Bullet_white Comments Comments

When a server reports data to Scout, a small bit of data is updated in a MySQL MyISAM table. Each of these updates doesn’t change much data:

UPDATE `plugins` SET `last_reported_at` = '2010-11-17 19:53:00' WHERE `id` = 999999

I assumed that only the updated data was written to disk. I was wrong: Jesse Newland of Rails Machine pointed out that updating a row in a MyISAM table rewrites the entire row to disk (not just the updated columns).

These updates were expensive: we were updating small bits of data, but our table contained a large text column that accounted for 75% of the data stored in the table. Every update to a row in our plugins table would rewrite the large text column (even though the text column wasn’t changing).

Read More →


Now this is a GUI

By Andre Bullet_white Comments Comments

Scout customer RailsWare has implemented one of the best front-ends we’ve seen for Scout notifications—a real, physical stoplight on the wall of their offices:

Railsware constructed this awesome setup for their client, RatePoint. Yaroslav Lazor outlined the nuts and bolts:

  • Notifications from several monitoring services (Scout, Pingdom, New Relic) are collected in an IMAP folder.
  • A daemon monitors the inbox, and turns on the stoplight if necessary. The daemon also writes status to a database, where web-based and desktop-based notifications can also read and display status.
  • When an issue is marked resolved, the daemon turns off the stoplight—everything is back to normal.

We love the idea of seeing a big, bold, physical presentation of system status up on the wall. Great job RailsWare and RatePoint for putting this together!


Older posts: 1 ... 39 40 41 42 43 ... 68