James Gray, lead developer of the Scout Agent, takes a stab at the eternal question “What makes a great developer?” in the latest RailsCoach Podcast Episode.
Besides talking about Scout’s blended open-source approach to monitoring, James also talks about The Pomodoro Technique (a time management process), his time at the helm of the RubyQuiz, NoSQL, and other topics.
You'll see the "buffers" and "cached" columns, which tell you about the amount of memory that the kernel is using for filesystem buffers, etc.
This sort of cached data will be freed by the kernel when an application tries to allocate more than what is "free", which is why the "-/+ buffers/cache" line is really the important line to pay attention to when you're checking out the free memory on a system.
So in this example, 708MB is how much memory is technically available for allocation should an application need it. The "buffers" (86MB) and "cached" (288MB) will be released by the kernal if they are needed.
Just provide the paths to your Sphinx log files and Sam’s plugin reports the following data:
Average query time
Average time per rebuild
Average results returned
Like with any Scout plugin, you can plot metrics from the Sphinx Monitoring Plugin against other metrics. For example, the chart below shows the relationship between the Sphinx query rate and the server load:
12/23 - UPDATE - Scout Agent v5 has exited the BETA phase. If you're already using Scout, upgrade instructions appear when viewing a server at scoutapp.com.
We’ve prepared a BETA release of the Scout Agent that supports SMS notifications and makes creating your own plugins easier. We and a group of beta accounts been running it internally for some time, and now we invite you to try it out.
Get the New Agent
1) Install the Scout agent gem version 5.0.2:
sudo gem install scout
2) Change your crontab entry for Scout so it runs every minute, which is just * * * * * on Linux. Your current setup is */3, */10, or similar.
Dual-tier SMS Alerts (Ultimate Accounts only)
When things go really wrong it’s important you find out quickly. Using Scout’s triggers, you can now specify a separate threshold for SMS alerts. For example, send an email alert if disk capacity is over 70% and an SMS alert if disk capacity is over 90%). Here’s how:
1) Enter your SMS info in the new "notifications" section of Scout:
2) Edit/create a trigger. You’ll see a new option for the SMS threshold:
When the entered threshold is crossed, you’ll receive an SMS alert. SMS alerts are currently available only on Ultimate accounts.
Easier Plugin Creation and Editing
Because the new Agent pings us every minute, it is much more responsive to changes you make in configurations on Scoutapp.com. Any changes you make will be picked up within 60 seconds by the new Agent, regardless of your plan level.
Explicit Checkin Times
When you upgrade to the new Agent, you'll see the "Server Contact" box on your server homepage, providing visual confirmation of most recent and next checkin times:
Get your new Agent here:
sudo gem install
We’d love your feedback on the new Agent and functionality. Leave your comment here or send an email with with any comments.
Our co-author today is Jesse Newland, CTO of RailsMachine. Jesse keeps RailsMachine customers up and running and troubleshoots their toughest problems. We’re pleased to have him share some of his expertise on Phusion Passenger tuning.
Say your Rails application is running in production and it’s getting good traffic. Performance isn’t as good you would like. You’ve already determined that your database is not the bottleneck. What’s your next move?
There is a good chance that Passenger’s PassengerMaxPoolSize needs to be adjusted. PassengerMaxPoolSize specifies how many instances of your application Passenger will spin up to service incoming requests. If you were running Mongrels back in the day, PassengerMaxPoolSize is equivalent to the number of mongrels you configured for your app. The value of PassengerMaxPoolSize has a major bearing on your application’s performance.
When you get started with Scout, we try to setup sensible default triggers for events you may need to investigate on your servers.
For example, Scout generates an alert if your disk usage exceeds 85% or CPU Load over the last minute exceeds 3.
However, every server and application is different. For example, a CPU load of 3 is fine on some servers but a problem on others. We’ve made it easier to tune Scout for your setup if Scout is sending too many alerts.
Scout customer Eric Lindvall wrote up a nice piece on finding and fixing memory leaks in god -- specifically, when issuing "god load" on long-running god processes. Give it a read, it provides good insight into the troubleshooting process and the tools he used. Eric points to the Scout graphs showing both the symptoms:
And the solution:
Who wouldn't love to see memory usage go down and stabilize like that? Eric also provides patches to god in case you're having similar issues. Check out his full writeup for details.
If you're having trouble with memory consumption of a specific process, check out Scout's Process Memory Usage plugin.
Before Scout, my experience developing software was primarily consulting. Success was measured by delivering software on time and on budget.
With Scout, a subscription-based service, my focus isn’t on scheduling. We are self-funded and we didn’t have the luxury of a venture-backed startup. We’re focused on figuring out which pieces of development work can increase revenue the most. What follows is how we’re approaching it.