Part 1. Erlang releases using Rebar

At toSeemo.org we are widely using Erlang/OTP. Basically, all our major functionality is developed using Erlang.  And when our application was ready, there was a question on how to deliver and deploy it. Surely,  we wanted to do it in OTP way.Since by that time we had already relied on rebar, it was decided to expand its usage to include making releases.

Using rebar and Erlang releases was the idea we liked the most. Probably, it's the only correct approach to do application releases in OTP way. I say "probably", as there is really awesome tool called sync.  We used it as a development tool and we plan trying it in production.

This is the first article from the series of three or four. Later I will explain how we are doing hot code swapping and describe how it is incorporeted into our Jenkins software. And now please meet

Erlang releases using Rebar.

In this post I will explain how to create Erlang releases. I don't think that it's really necessary to explain how to setup rebar and start new project using it. I will simply assume that you just have rebar installed and an application started. As a result, your folders structure should look like this:

deps - folder for dependencies
ebin - folder for compiled sources 
include - folder for include files 
src - your application sources 
rebar.config - rebar configuration file 

So lets proceed to our releases:

Step 0. Change a bit folders structure.

As OTP release is set of OTP applications, it makes sense to reflect it in our folders structure. There are two type of applications: apps and deps. The apps folder will contain our applications (in my case it's just a single toseemo application) and deps folder will contain 3rd party dependencies:

apps - folder for our custom applications
    toseemo
        ebin - folder for compiled sources 
        include - folder for include files 
        src - your application sources 
deps - folder for dependencies
rebar.config - rebar configuration file 

Step 1. Create folder for releases.  

Go to your project folder root and create one more folder. Name of this folder should be rel.

$ mkdir rel 

Step 2. Change a bit rebar configuration file.

Just open your rebar.config file and add sub_dirs tuple there like in example below (highlighted with bold italic). I'm using real toSeemo.org configuration file in this example.

%>>> rebar.config begin
{sub_dirs, ["apps/toseemo", "rel"]}. 
{deps, [
    {cowboy, ".*", {git, "https://github.com/extend/cowboy.git", {tag, "1.0.0"}}},
    {mongodb, ".*", {git, "https://github.com/kshamko/mongodb-erlang.git", {branch, "cryptofix"}}},
    {cutkey, ".*", {git, "https://github.com/kshamko/cutkey.git", {branch, "r16-compatible"}}},
    {mochijson2, ".*", {git, " https://github.com/tel/mochijson2.git", {tag, "2.3.2"}}}
]}. 
%>>> rebar.config end 

Step 3. Create default application configuration.

On this step we will generate a set of files which are required to build OTP release (some default configs, binaries etc). Please note that I'm using toseemo as node id and it will be different for you application.

$ cd rel
$ rebar create-node nodeid=toseemo 
==> rel (create-node)
Writing reltool.config
Writing files/erl
Writing files/nodetool
Writing files/toseemo
Writing files/sys.config
Writing files/vm.args
Writing files/toseemo.cmd
Writing files/start_erl.cmd
Writing files/install_upgrade.escript

Actually it's not a big problem if you use toseemo as node id. This value can be changed any time in your rel/files/vm.args file

NOTE!!! To learn more about rebar commands you can use $ rebar -c in console. Or just follow this link (perhaps, even better way) to find more about available rebar options

Step 4. Edit reltool.config file.

We need to define paths to out application and it's dependencies dir, our application version, set the list of applications which should be started as part of the release (in case of toseemo one of these apps is cowboy webserver). Please note that rebar already added toseemo to applications list. In case you have no any other applications to start just skip this section.

Here is our initial reltool.config file created on the previous step:

{sys, [
       {lib_dirs, []}, % <=== here we will set pathes.
       {erts, [{mod_cond, derived}, {app_file, strip}]},
       {app_file, strip},
       {rel, "toseemo", "1", <=== here we will set the version
        [
         kernel,
         stdlib,
         sasl,
         toseemo <=== here we will add applications to be started
        ]},
       {rel, "start_clean", "",
        [
         kernel,
         stdlib
        ]},
       {boot_rel, "toseemo"},
       {profile, embedded},
       {incl_cond, derived},
       {excl_archive_filters, [".*"]}, %% Do not archive built libs
       {excl_sys_filters, ["^bin/(?!start_clean.boot)",
                           "^erts.*/bin/(dialyzer|typer)",
                           "^erts.*/(doc|info|include|lib|man|src)"]},
       {excl_app_filters, ["\.gitignore"]},
       {app, toseemo, [{mod_cond, app}, {incl_cond, include}]}
      ]}.

{target_dir, "toseemo"}.

{overlay, [
           {mkdir, "log/sasl"},
           {copy, "files/erl", "\{\{erts_vsn\}\}/bin/erl"},
           {copy, "files/nodetool", "\{\{erts_vsn\}\}/bin/nodetool"},
           {copy, "toseemo/bin/start_clean.boot",
                  "\{\{erts_vsn\}\}/bin/start_clean.boot"},
           {copy, "files/toseemo", "bin/toseemo"},
           {copy, "files/toseemo.cmd", "bin/toseemo.cmd"},
           {copy, "files/start_erl.cmd", "bin/start_erl.cmd"},
           {copy, "files/install_upgrade.escript", "bin/install_upgrade.escript"},
           {copy, "files/sys.config", "releases/\{\{rel_vsn\}\}/sys.config"},
           {copy, "files/vm.args", "releases/\{\{rel_vsn\}\}/vm.args"}
          ]}.

And the updated version of our config file is:

{sys, [
       {lib_dirs, ["../apps", "../deps"]}, 
       {erts, [{mod_cond, derived}, {app_file, strip}]},
       {app_file, strip},
       {rel, "toseemo", "0.1.1", <=== you can use any convention you want for version numbers
        [
         kernel,
         stdlib,
         sasl,
         inets,
         cowboy,
         cutkey,
         mongodb,
         toseemo
        ]},
       {rel, "start_clean", "",
        [
         kernel,
         stdlib
        ]},
       {boot_rel, "toseemo"},
       {profile, embedded},
       {incl_cond, derived},
       {excl_archive_filters, [".*"]}, %% Do not archive built libs
       {excl_sys_filters, ["^bin/(?!start_clean.boot)",
                           "^erts.*/bin/(dialyzer|typer)",
                           "^erts.*/(doc|info|include|lib|man|src)"]},
       {excl_app_filters, ["\.gitignore"]},
       {app, toseemo, [{mod_cond, app}, {incl_cond, include}]}
      ]}.

% target_dir and overlay sections are omitted here but they should exist.

Now let's generate a release.

Step 5. Generate release.

$ rebar get-deps
$ rebar compile
$ rebar generate
==> rel (generate)
$ ls rel
files  reltool.config  toseemo 

In this example toseemo is a folder with the release. It contains bin folder with executable file for your app.

Step 6. Run application.

$ ./rel/toseemo/bin/toseemo
Usage: toseemo {start|start_boot |foreground|stop|restart|reboot
|ping|console|getpid|console_clean|console_boot |attach|remote_console|upgrade}

As you can see it's possible to start app in different modes: daemon, foreground or with console attached, attach console to running node, ping node etc. Rebar has generated this usefull executable file for us.

But sometimes just runnig your erlang application is not enough. Sometimes you need to run erlang VM with some specific parameters. These parameters can be set in vm.args file. It can be changed just for single release so the file can be found in rel/toseemo/releases/RELEASE_VERSION/vm.args. Or it can be changed for all future releases in rel/files/vm.args - it's kind of default vm.args file. A bit more details about  that in my next post