Behind the scenes at Charm, part 1: Ruby libraries

So we’ve launched our new product Charm (right now, it’s invite-only as we scale)—and we’ve learned a lot developing it.

Here’s the first part of a series of blog posts about the underlying technology we use.

This time around, let’s have a look on some of the Ruby libraries in the back-end. To give you an idea about the general architecture of Charm—it’s a Rails 2.3 app (we’re not using 3.x because we ran into memory leaks and performance issues along the way), plus we’re using node.js for real-time push updates, and several storage/database/indexing systems, like PostgreSQL, Redis and Solr; among other things.

Production

As an email support application, Charm obviously reads email. We’re using a custom fetcher (called Fido) for fetching email from IMAP servers, combined with the Mail gem to parse email. The state of email parsing on Ruby is sadly not as well advanced as on some other platforms, and there are bugs and kinks, mostly having to do with parsing broken email and working around encoding problems. We hope to contribute back some fixes for the Mail gem at a future date—we aim to parse any and all email without errors and have a pretty good collection of tests by now (thanks to our awesome beta testers!).

It should be noted that Ruby really lends itself to monkey-patching, and that’s a good thing to quickly fix problems that cause issues in production—and at the same time it’s easy to contribute back and solve the problem for everyone. Yay, open source!

Of course, we need to send email, too. We don’t want to mess with our own infrastructure for this, and leave it to the specialists at Postmark to deliver our mail. It’s an awesome service, and it comes with a snazzy Ruby gem.

Of really great help is Whenever, which helps automating cron jobs—we have a lot of background processing, and it goes a long way to have a nice DSL at your disposal, combined with integration into Capistrano for deploying.

Here’s an example of using Whenever:

every 1.day, :at => '12 pm' do
  rake "charm:subscriptions:process"
end

every 1.day, :at => '12 am' do
  rake "charm:notifications:generate"
end

every :hour do
  rake "charm:push_updates:purge"
end

# a lot more stuff

As for optimizing performance, among others, the date performance gem and of course Erubis (ERB implementation in C, now comes default with Rails 3) makes a big difference. Plus, the Dalli gem is a good choice when you use memcached.

Development

There are a lot of new and shiny libraries for an easier time during development, too. Of note are Foreman, a really incredibly cool way to start all the required services that your app needs (via a Procfile), and Timecop (makes it easy to mess with the current date/time, which is great when writing tests for the receiving email/replying to it cycle; not to mention billing systems).

Here’s our Procfile for Charm:

redis:        redis-server config/redis-dev.conf
guard:        bundle exec guard
sunspot:      bundle exec rake sunspot:solr:run
# plus other secret stuff

We also use Guard and Guard::CoffeeScript to automatically compile CoffeeScript files to JavaScript during development.

Fancy IRB!

I like hacking around in the Rails console, so Wirb comes in handy—I can highly recommend it. To use it, you’ll need a ~/.irbrc file like this (also increases the history buffer):

require 'irb/completion'
require 'irb/ext/save-history'
ARGV.concat [ "--readline", "--prompt-mode", "simple" ]
IRB.conf[:SAVE_HISTORY] = 1000
IRB.conf[:HISTORY_FILE] = "#{ENV['HOME']}/.irb-save-history" 
require 'rubygems' unless defined? Gem
require 'wirb'
Wirb.start

Plus, in your Gemfile, in the development section add:

group :development do
  gem 'term-ansicolor'
  gem 'wirb'
end

Then, enjoy the new syntax highlighting goodness when using script/console.

Next time…

In the next installment of the “what makes Charm Support tick” series I’ll take a look at the client-side and dive into the various JavaScript libraries we use and how it’s all connected! Stay tuned!

P.S. Do you want to break out of working by the hour? Do you want to create your own products and really be your own boss? How about crafting an independent income you can live on — and never have to freelance again? Want to learn how to make a living from developing your own software (or other) products? Take the 30×500 product class.