Wednesday, June 11, 2008

I'm in ur Erlangz, upgrading ur applicationz

Hot code loading is one of the sexiest features of Erlang; after all, who wants downtime? If you're running mnesia, it's also very convenient, because as we've learned mnesia does not like to be shut down (more precisely, it makes no attempt to incrementally replicate changes across brief outages, instead doing full table (re)copying; something probably nobody cares about because they never turn it off).

Erlang's hot code loading support consists of low level language and runtime features, and a high level strategy embodied by release handler. The latter is a very Java-esque strategy of bundling up the complete new state of the system and upgrading everything. Without being normative let's just say that we enjoy using our OS package manager to manage the software on our boxes, and so we were interested in integrating Erlang hot code loading with the OS package manager.

The result is erlrc, which when operating looks something like this:
pmineiro@ub32srvvmw-199% sudo apt-get install egerl
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages will be upgraded:
egerl
1 upgraded, 0 newly installed, 0 to remove and 34 not upgraded.
Need to get 0B/113kB of archives.
After unpacking 0B of additional disk space will be used.
WARNING: The following packages cannot be authenticated! egerl
Install these packages without verification [y/N]? y
(Reading database ... 48221 files and directories currently installed.)
Preparing to replace egerl 4.0.1 (using .../archives/egerl_4.1.0_all.deb) ...
Unpacking replacement egerl ...
erlrc-upgrade: Upgrading 'egerl': (cb8eec1a1b85ec017517d3e51c5aee7b) upgraded
Setting up egerl (4.1.0) ...
erlrc-start: Starting 'egerl': (cb8eec1a1b85ec017517d3e51c5aee7b) already_running
Basically, I issued an apt command, the result was to hot upgrade the corresponding Erlang application on the running Erlang node on the machine.

In addition to integration with the OS package manager, erlrc also features an extensible boot procedure (basically, using a boot directory instead of a boot file to determine what to start, which makes packaging easier); and in addition erlrc will automatically generate appup files which correspond to most of the common cases outlined in the appup cookbook. Of course, Erlang packages made with fw-template-erlang are automatically instrumented for erlrc (but the instrumentation is inert if erlrc is not installed).

If you're interested, there's more extensive documentation on google code.

3 comments:

Tobbe said...

Nice!

What kind of system/product are you guys building?

Paul Mineiro said...

Hey Tobbe.

We're in stealth mode right now, but the application domain involves data mining on large datasets with near-time response requirements.

We really want the "basic infrastructure" pieces to be solid so we don't have to worry about them, so we hope people like yourself will use our stuff and give us feedback!

boorad said...

Um, this is hot. No, it's hawt! Nice work.

Also, ping me if you need help as I have extensive BI/DW experience. Made a SaaS BI app that's going on 8 yrs old =-o

I'm currently working on a Mochi/CouchDB app that has some data mining requirements.