John P. Wood of Signal, which offers a mobile customer engagement platform used by many top brands, recently created a couple of Scout Plugins for monitoring CouchDB. I’ve always been impressed by the team at Signal, so I was curious how they were using CouchDB in production. It turns out CouchDB is a huge part of their infrastructure – for example, one of their CouchDB databases is over 130GB in size.
John was kind enough to share his experiences with CouchDB below.
You use a number of different storage engines (MySQL, CouchDB, MongoDB, and Memcached) at Signal. Where does CouchDB fit in?
A couple of years ago we were running into performance issues with some very large MySQL tables. Queries against these tables were taking very long to run, and were causing page timeouts in our web application. At the advice of a friend who was helping us out as a consultant, we started looking at CouchDB. CouchDB views turned out to be a great fit for our problem.
A key component of our application is SMS messaging. The problematic MySQL queries we were running were collecting aggregate stats on these messages (how many messages did account A send in January of 2009, all of 2009, how many for all accounts, etc). Most of the queries were executing on past data, meaning the results of those queries would not change over time after that time period had past. So, it was simply a waste to re-calculate these numbers over and over. We considered using summary tables in MySQL to avoid this costly re-calculation, but saw them as being inflexible and difficult to maintain.