Scout is no longer a puppy: in dog years, he’s old enough to drink, get drafted, and rent a car. During that time, cruft gathered around the edges of our server infrastructure.
We’ve been using a hodgepodge of server hardware, some performing multiple roles, some not, all individually configured and tuned. Small changes to our stack seemed to involve a lengthy checklist. Our staging environment didn’t mirror production: what happened on staging didn’t always happen on production. Finally, database changes were painful.
We wanted to get lean in the right places: could we make the young adult Scout as easy to manipulate as the baby Scout?
7.weeks.ago we followed a four-part process to get there.
We don’t have a product roadmap. Why?
We become that with which we busy our minds.
I’m not preoccupied with hitting every deadline on a one-year pipeline of features. I’m focused on making it easier to monitor servers every week. Rather than thinking about deadlines, I wake up thinking: “How can we make monitoring server clusters easier? Is what we’re doing now the least-effort way to do that?”
Over an entire year, that’s a lot of days for small improvements to really add up.
A bike commute leaves me more refreshed than a run in the park, a round of video games, or reading a book. David Byrne wrote about this in Bicycle Diaries:
It (biking) facilitates a state of mind that allows some but not much of the unconscious to bubble up. As someone who believes that much of the source of his work and creativity is to be gleaned from those bubbles, it’s a reliable place to find that connection.
There are lots of things I’m looking for when I’m riding a bike: cars turning right, traffic lights, pot holes, babies in strollers, tourists on Segways, etc. On the surface, these look like annoyances, but they are a secret gift: for my own safety, I’m forced to live in the moment. It’s a needed escape from a day spent coding and planning.
One of the things I was most curious about before starting Scout was what my day-to-day life would look like as a business owner versus life as a developer. How much time would I spend writing code? Answering emails? Handling the business bits? Would I work more?
Three years after launching Scout, I’d like to help answer that question. I used RescueTime to track activity on my computer. Since almost all of my work is from my laptop, it’s an easy way to get a breakdown.
A couple of years ago I went to Argentina. I don’t speak Spanish. This meant a lot of the basic conversations I’d have with locals were frequently interrupted by me paging through a translation guide. I felt very disconnected: basic exchanges were cumbersome.
When I work with an API for the first time, it often feels the same. A lot of my time is spent referencing documentation just to get basic things working. That’s asking for a lot of effort at this stage of the relationship. I just want to know: is there chemistry between your API and me?
- Does it even work?
- How fast is the response time?
- Is the data in sync with the web interface?
- What can I access?
Spending a lot of time trying to get my first method call working is frustrating. When we built the Scout API, I wanted to make that initial experience easier.
Got a site you need to serve up via SSL? Here are your Cliffs notes. This assumes 1) your site already runs without SSL; 2) you’re using Apache and Ubuntu; 3) you don’t want any browser warnings, so no self-signed certificates.
Before a recent plane trip, I checked out the Autobiography of Benjamin Franklin from the library. Franklin might have the greatest list of accomplishments in American history. I was curious to learn how he did it. Three developer-centric takeaways:
The Socratic Method
Like developers, Franklin loved a spirited debate. In his early teens, he found another “Bookish Lad” who also enjoyed an argument. They exchanged several letters over the “Propriety of educating the Female sex in Learning” and Franklin’s father happened to find them. His father noted that while he had the upper hand in spelling and punctuation, he fell woefully short in style. He dropped his previous abrupt approach and picked up the “modest diffidence” of the Socratic method. Words like “certainly” or “undoubtedly” were replaced by “I conceive” or “it appears to me”. Franklin quotes a line from Alexander Pope:
Men should be taught as if you taught them not,
And things unknown proposed as things forgot,
Franklin spent his life persuading people. He credits the modest approach as a big reason for his successes.
For us developers: yes, that blog post on “Why SOMETHING sucks” may get attention, but it’s probably not going to build anything constructive or result in relationships with people you’d actually enjoy working with.
The place for open source
If Github (and the Internet, electricity, Linux, etc) were around in Ben’s time, he’d have some prolific public projects. One of the many amazing things about Franklin is that he didn’t patent his inventions:
This doesn’t mean he didn’t like building wealth. In fact, he wrote a book about that. He knew when to give it away and when to charge for it.
There’s more than typesetting
For Franklin, all takeaways seem to lead here: he showed the kind of impact you can make with a keen sense of balance. Out of any job I’ve held, being a developer lends itself the most to an unbalanced life. You can always make your application better looking, more featured, faster, leaner, etc. You can code and deploy from almost anywhere at any time. Before long, you can begin to measure your worth solely by code.
Franklin made his living as a printer, but his focus on improving himself outside of his typesetting skills is what made him so influential.
It’s always satisfying to find tools that make your workflow smoother. Here are three that I started using recently:
iTerm2 is a Terminal replacement with a ton of features. The three that make the most difference in my workflow are search, mouseless copy, and tabs which you can tear away into their own windows and easily recombine. I haven’t gotten into some of the other features yet, like a hotkey-driven HUD-like terminal, Expose for your tabs, and autocomplete. The few that I am using make the switch worth it. And, the price is right (free).
Kaleidoscope is a very well-implemented diff tool. Kaleidoscope isn’t free, but it’s worth paying for: it starts up quickly, looks great, and integrates seamlessly with git.
Mouse Locator highlights your mouse pointer after it’s been still for a period of time. The use case for me: when I have multiple command line windows open (especially on a multi-monitor setup), I frequently lose the pointer in the dark windows. Mouse Locator solves that. I set the trigger delay time to about 1 minute (so it activates the first time you move the mouse after a minute of inactivity), and a very short display time (it fades out immediately). Once you set it up, you never have to think about it again. Mouse Locator is also free.
Word-of-mouth from like-minded users is a great way to discover new Mac tools. Got one you really like? Let us know in the comments!
To inspire hard work, some young men hang a poster on their wall that includes: (1) an exotic sports car (2) a scantly clad lady and (3) a beach house. My inspirational poster would be much less attractive: a friendly butler who offers time-honored wisdom (with an accent because people with accents are smarter) and absolutely loves running errands for me.
I don’t like running errands because I don’t like waiting in lines. My nightmare: having to pickup groceries during a busy weekend afternoon. There are 3 queues at the grocery store that can cause a delay:
- Finding a parking spot
- Getting a shopping cart
- Checking out
Modern web apps face the same queuing issues serving web requests under heavy traffic. For example, a web request served by Scout passes through several queues:
That’s Apache (for SSL processing) to HAProxy on the load balancer, then Apache to Passenger to the Rails app on a web server.
A request can get stuck in any of those five spots. The worst part about queues? Time in queue is easy to miss. Most of the time, people look at the application log when they suspect a slowdown. However, a slowdown in any of the four earlier queues won’t show up in your application log. Just looking at your application and database activity for slowdowns is like recording the time it takes to get your groceries from the time you grab the first item on the shelf till you start waiting to checkout: you’re leaving out the time it takes to find a parking spot, get a cart, and checkout.
Now, before you start worrying about queues, take a deep breath. First, each of these systems are super reliable. For the most part, they just work. Second, it’s much more likely your application logic is the cause of a performance issue than a queuing problem. Look there first.
Third (and most importantly), each of these systems handles queues in remarkably similar ways. Understanding some basic queuing concepts will go a long way. Let’s take a look at some basics and then specific examples for Apache, HAProxy, and Passenger.
First, Tina Fey needs to write more. Both of her New Yorker articles have been terrific. Second, her latest New Yorker piece had a great nugget on how producing is about discouraging creativity:
You may have an occasion where the script calls for a bran muffin on a white plate, and people from the props department show up with a bran cake in the shape of Santa Claus sitting on a silver platter that says “Welcome to Denmark” on it. “We just thought it would be funny”, they say. And you have to find a polite way to explain that the character is Jewish, so her eating Santa’s face might have negative connotations, and the silver tray, while beautiful, is creating a weird glare on camera, and maybe let’s just go with the brain muffin on the white plate.
It’s easy to get absorbed in code and build something that’s technically awesome yet far more than you need. In other words, that Redis-backed, OpenID-compatible, audible capatcha, two-factor identification login system idea of mine is creative, but wouldn’t a session cookie and that old-school relational database my grandpa talked about work for now?