Michael Dyrynda | An Envoyer-like deployment script using Envoy

Introduction

Taylor Otwell announced Envoyer last week, I set about actually looking into Laravel's Envoy for a project I was working on.

This might seem a bit moot now that Envoyer has launched, but bare with me.

The basic service from Envoyer covers 10 different projects for $10 a month. This seems like a very good price point for what it offers, but with Homestead it's really easy for me to do development on my local machine, without affecting what my clients are reviewing on staging. It doesn't, however, make sense to take up two of those ten project slots for effectively a single project in multiple deployments - in my instance this is staging and production.

The plan

Zero-downtime isn't really a huge issue in a staging environment, but you never know when someone might be reviewing the site while you're pushing changes to it, so my plan was to replicate as closely as possible the Envoyer offering.

Grab the latest release
Install dependences
Update permissions
Update releases
Migrate the database
Tidy up old releases

Fairly straight forward and something that Capistrano and Rocketeer have been doing for quite a while now, but I wanted to keep within the Laravel ecosystem.

A bit of googling got me set off in the right direction thanks to @fideloper's Servers for Hackers. He'd covered most of these tasks in his two screencasts, so I won't go into too much detail here, but checkout the further reading section at the end of this post.

Attack

The first thing you'll need to do is make sure that you update your web server configuration to point to the current/public directory, rather than public directly, due to our about-to-be updated directory structure.

1server {
2    server_name iatstuti.net;
3    root /path/to/current/public;
4
5    # Rest of your config
6}

Whilst most of the tasks were covered in the Servers for Hackers screencasts already, I still needed to tackle a couple of items and handle a couple of those items in another way.

Grab the latest release using the tarball, rather than a git clone
Migrate the database
Tidy up old releases

Latest release handling

As above, I wanted to grab the tarball of the latest release in the given branch, rather than running git clone. I don't need to track history or anything like that, so it makes sense to get a clean dump of the files.

The first thing we need to do is setup our deploy. We accept --repo (i.e. deringer/iatstuti-dot-net) and --base_dir (i.e. /data/www/www.iatstuti.net) when we run the envoy command, which means we can easily reuse this Envoy config file in other projects.

Note: the --repo argument and indeed this config file assumes you are using GitHub for hosting your repository.

 1@servers([ 'remote' => 'server.example.com', ])
 2 
 3@setup
 4    if ( ! isset($repo) )
 5    {
 6        throw new Exception('--repo must be specified');
 7    }
 8 
 9    if ( ! isset($base_dir) )
10    {
11        throw new Exception('--base_dir must be specified');
12    }
13 
14    $branch      = isset($branch) ? $branch : 'develop';
15    $repo_name   = array_pop(explode('/', $repo));
16    $repo        = 'https://api.github.com/repos/' . $repo . '/tarball/' . $branch;
17    $release_dir = $base_dir . '/releases';
18    $current_dir = $base_dir . '/current';
19    $release     = date('YmdHis');
20    $env         = isset($env) ? $env : 'staging';
21@endsetup

Then we define our fetch_repo task.

 1@task('fetch_repo')
 2    [ -d {{ $release_dir }} ] || mkdir {{ $release_dir }};
 3    cd {{ $release_dir }};
 4 
 5    # Make the release dir
 6    mkdir {{ $release }};
 7 
 8    # Download the tarball
 9    echo 'Fetching project tarball';
10    curl -sLo {{ $release }}.tar.gz {{ $repo }};
11 
12    # Extract the tarball
13    echo 'Extracting tarball';
14    tar --strip-components=1 -zxf {{ $release }}.tar.gz -C {{ $release }};
15 
16    # Purge temporary files
17    echo 'Purging temporary files';
18    rm -rf {{ $release }}.tar.gz;
19@endtask

This is fairly straightforward (I think), but I'll go over it just in case.

Check if the release directory exists and if not, we create it.
Change into the release directory for the next actions
Grab the tarball from GitHub and drop it into {{ $release_dir }}/{{ $release }}.tar.gz
Extract the tarball, stripping the first component (the user-repo-identifier folder), and dumping the contents into our release directory
Purge the temporary files

Migrate the database

One of the missing pieces in both @fideloper's and Envoyer's offerings is the ability to run migrations. With Envoyer you can add migrations as a deployment hook easily enough.

As this can be potentially disruptive to users on the sight, I have chosen to put Laravel into maintenance mode. This goes against the 'zero-downtime' dream. Realistically, this will only be a few seconds of interruption. If it's something you can't have, you can choose to not call the tasks within the stage macro.

 1@task('down')
 2    cd {{ $release_dir }}/{{ $release }};
 3    php artisan down;
 4@endtask
 5 
 6@task('migrate')
 7    echo 'Running migrations';
 8    cd {{ $release_dir}}/{{ $release }};
 9    php artisan migrate --env={{ $env }} --force;
10@endtask
11 
12@task('up')
13    cd {{ $current_dir }};
14    php artisan up;
15@endtask

Tidy up old releases

Lastly, I purge old releases. If you're deploying often, your releases directory will be cluttered with tens or hundreds of copies of your app. This can add up over time, so I only keep the 5 most recent deployments.

 1@task('clean_old_releases')
 2    # This will list our releases by modification time and delete all but the 5 most recent.
 3    purging=$(ls -dt {{ $release_dir }}/* | tail -n +5);
 4 
 5    if [ "$purging" != "" ]; then
 6        echo Purging old releases: $purging;
 7        rm -rf $purging;
 8    else
 9        echo "No releases found for purging at this time";
10    fi
11@endtask

This runs a list of directories, ordering by modified time, grabbing all records from the 6th and after, and doing a recursive forced delete on those directories. They'll be gone, and there is no confirmation.

Full script

I'll include the full script here for reference within the post. If you want to use it in your projects, you can grab the gist. Be sure to update the @servers declaration at the top of the file.

  1@servers([ 'remote' => 'server.example.com', ])
  2 
  3@setup
  4    if ( ! isset($repo) )
  5    {
  6        throw new Exception('--repo must be specified');
  7    }
  8 
  9    if ( ! isset($base_dir) )
 10    {
 11        throw new Exception('--base_dir must be specified');
 12    }
 13 
 14    $branch      = isset($branch) ? $branch : 'develop';
 15    $repo_name   = array_pop(explode('/', $repo));
 16    $repo        = 'https://api.github.com/repos/' . $repo . '/tarball/' . $branch;
 17    $release_dir = $base_dir . '/releases';
 18    $current_dir = $base_dir . '/current';
 19    $release     = date('YmdHis');
 20    $env         = isset($env) ? $env : 'staging';
 21@endsetup
 22 
 23@macro('deploy', [ 'on' => 'remote', ])
 24    fetch_repo
 25    run_composer
 26    update_symlinks
 27    update_permissions
 28    down
 29    migrate
 30    up
 31    clean_old_releases
 32@endmacro
 33 
 34@task('fetch_repo')
 35    [ -d {{ $release_dir }} ] || mkdir {{ $release_dir }};
 36    cd {{ $release_dir }};
 37 
 38    # Make the release dir
 39    mkdir {{ $release }};
 40 
 41    # Download the tarball
 42    echo 'Fetching project tarball';
 43    curl -sLo {{ $release }}.tar.gz {{ $repo }};
 44 
 45    # Extract the tarball
 46    echo 'Extracting tarball';
 47    tar --strip-components=1 -zxf {{ $release }}.tar.gz -C {{ $release }};
 48 
 49    # Purge temporary files
 50    echo 'Purging temporary files';
 51    rm -rf {{ $release }}.tar.gz;
 52@endtask
 53 
 54@task('run_composer')
 55    echo 'Installing composer dependencies';
 56    cd {{ $release_dir }}/{{ $release }};
 57    composer install --prefer-dist --no-scripts -q -o;
 58@endtask
 59 
 60@task('update_symlinks')
 61    echo 'Updating symlinks';
 62 
 63    # Remove the storage directory and replace with persistent data
 64    echo 'Linking storage directory';
 65    rm -rf {{ $release_dir }}/{{ $release }}/storage;
 66    cd {{ $release_dir }}/{{ $release }};
 67    ln -nfs {{ $base_dir }}/storage storage;
 68 
 69    # Optimise installation
 70    echo 'Optimising installation';
 71    php artisan clear-compiled --env={{ $env }};
 72    php artisan optimize --env={{ $env }};
 73 
 74    # Import the environment config
 75    echo 'Linking .env file';
 76    cd {{ $release_dir }}/{{ $release }};
 77    ln -nfs {{ $base_dir }}/.env .env;
 78 
 79    # Symlink the latest release to the current directory
 80    echo 'Linking current release';
 81    ln -nfs {{ $release_dir }}/{{ $release }} {{ $current_dir }};
 82@endtask
 83 
 84@task('update_permissions')
 85    cd {{ $release_dir }}/{{ $release }};
 86    echo 'Updating directory permissions';
 87    find . -type d -exec chmod 775 {} \;
 88    echo 'Updating file permissions';
 89    find . -type f -exec chmod 664 {} \;
 90@endtask
 91 
 92@task('migrate')
 93    echo 'Running migrations';
 94    cd {{ $release_dir }}/{{ $release }};
 95    php artisan migrate --env={{ $env }} --force;
 96@endtask
 97 
 98@task('down')
 99    cd {{ $release_dir }}/{{ $release }};
100    php artisan down;
101@endtask
102 
103@task('up')
104    cd {{ $current_dir }};
105    php artisan up;
106@endtask
107 
108@task('clean_old_releases')
109    # This will list our releases by modification time and delete all but the 5 most recent.
110    purging=$(ls -dt {{ $release_dir }}/* | tail -n +5);
111 
112    if [ "$purging" != "" ]; then
113        echo Purging old releases: $purging;
114        rm -rf $purging;
115    else
116        echo "No releases found for purging at this time";
117    fi
118@endtask

To run, simply execute envoy run deploy --repo=<user>/<repo> --base_dir=/path/to/base_dir. If you want to deploy a different branch, you can pass the --branch argument, if you want to deploy a different environment, pass the --env argument.

/path/to/base_dir should be to one level below your releases directory i.e. /data/www/iatstuti.net.

Ensure that your storage directory exists in your base_dir and that the layout is the same as what you have within your app directory. For your first deployment you can set this up yourself.

Also, you need to create your .env file in your base_dir also, ensuring that you update this prior to any deployments to make sure that you have all the necessary environment variables present and prevent any accidental downtime.

Conclusion

So there you have it, your own Envoyer-like deployment config for Envoy. As I wrote at the top of the post, this is mainly because I don't want to consume two project slots within Envoyer for a single project on two different environments. Hopefully Taylor will provide this option at some point in the future.

One thing this misses that Envoyer provides is an easy way to rollback to a previous release. In the event you do experience issues, you'll need to do that part manually. It'll just be a matter of logging into your server and running the symlink command to update the current directory to a previous release. Also be mindful that if you have run any migrations in a failed release, you'll need to do that from that release's directory, as the migration files won't exist in your previous release.

Edit Mar 21, 2015: I made some tweaks to the Envoy config such that we now grab a tarball from GitHub, rather than a zipball. This gives us the flexibility to extract the files from the archive directly into the destination folder in one command.

Additionally, I made the environment parameter an argument to the config (--env, defaulting to staging).

I also switched the order of the php artisan optimize and php artisan clear-compiled commands to make sure we were making them after symlinking the storage directory, not before.

The naming of some of the options has been tweaked - the server is now called simply remote - a more generic name in the event you want to use this for other environments. I also renamed the macro to deploy from stage to be more generic.

Lastly, I've added a Laravel 4 version of this script in a gist here.