With over 50 million plays, OMGPOP – the free multiplayer game site – is logging a lot of data. Tracking stats like app downloads and launches paint a picture of how their games are performing.
This logging data is collected via Flume, a system for collecting streaming data, and delivered to a Hadoop Distributed File System (HDFS). So, how do you keep your Flume nodes configured in a consistent manner?
Enter Apache ZooKeeper, “a centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services” (from the ZooKeeper homepage). Michael Fielder (Blog | Twitter), an NYC-based freelance systems operations engineering consultant, recently created a Scout Plugin for monitoring ZooKeeper at OMGPOP. The plugin parses the output of the
srvr command on the installed server, reporting key ZooKeeper metrics. Additionally, an error is generated if ZooKeeper is not running.