Michael Dyrynda | Uploading files to Amazon S3 from the browser

Introduction

After releasing part one of this series, it was brought to my attention that the version of AWS Signature I was using - Version 2 - was not compatible with some of the newer regions were only compatible with Version 4.

This post serves to show you how to go about implementing browser uploads for these newer regions.

It's worth pointing out that whilst some regions only support Version 4, all regions that support Version 2 also support Version 4, so you can use this method if there is a possibility you'll be using one of the newer regions in future.

How it works

As we're only updating the signature version we're using, a lot of the code from the previous post remains the same, so I'm only going to focus on the specifics of Version 4 to keep this post brief.

Basic upload form

Lets first look at our basic form. We're firstly going to remove the AWSAccessKeyId input entirely, move the policy to the top of our form just to group all of the signature-related items together, and rename signature to the new x-amz-signature value.

 1<form method="post" action="https://{{ config('filesystems.disks.s3.bucket') }}.s3.amazonaws.com" enctype="multipart/form-data">
 2       <input type="hidden" name="AWSAccessKeyId" value="{{ config('filesystems.disks.s3.key') }}"> // [tl! --]
 3       <input type="hidden" name="policy" value="{{ $policy }}"> // [tl! ++]
 4       <input type="hidden" name="x-amz-algorithm" value="AWS4-HMAC-SHA256"> // [tl! ++]
 5       <input type="hidden" name="x-amz-credential" value="{{ $credential }}"> // [tl! ++]
 6       <input type="hidden" name="x-amz-date" value="{{ $date->format('Ymd\THis\Z') }}"> // [tl! ++]
 7       <input type="hidden" name="x-amz-signature" value="{{ $signature }}"> // [tl! ++]
 8 
 9       <input type="hidden" name="acl" value="private">
10       <input type="hidden" name="key" value="${filename}">
11       <input type="hidden" name="policy" value="{{ $policy }}"> // [tl! --]
12       <input type="hidden" name="success_action_redirect" value="{{ url('/s3-upload') }}">
13       <input type="hidden" name="signature" value="{{ $signature }}"> // [tl! --]
14       <input type="file" name="file">
15       <button type="submit">Upload</button>
16</form>

Creating an upload credential

We'll next introduce a new function to generate the Version 4 credential string. With Version 2, this was just the public AWSAccessKeyId, which has already been removed from the form.

1// routes/web.php
2function createCredential($date) {
3    return vsprintf('%s/%s/%s/s3/aws4_request', [
4        config('filesystems.disks.s3.key'),
5        $date->format('Ymd'),
6        config('filesystems.disks.s3.region'),
7    ]);
8}

The createCredential function takes a date object and returns a formatted string with the public key, region, service, and request type in the following format:

PublicKey/YYYYMMDD/aws-region/s3/aws4_request.

Updated policy document creation

Next, the policy document needs to be updated to match the new x-amz-* inputs that have been added.

 1function createPolicy() { 
 2  function createPolicy($credential) { 
 3    return base64_encode(json_encode([
 4        'expiration' => now()->addHour()->format('Y-m-d\TG:i:s\Z'),
 5        'expiration' => now()->addHour()->format('Y-m-d\TH:i:s\Z'), 
 6        'conditions' => [
 7            ['bucket' => config('filesystems.disks.s3.bucket')],
 8            ['acl' => 'private'],
 9            ['starts-with', '$key', ''],
10            ['eq', '$success_action_redirect', url('/s3-upload')],
11            ['x-amz-algorithm' => 'AWS4-HMAC-SHA256'], 
12            ['x-amz-credential' => $credential], 
13            ['x-amz-date' => now()->format('Ymd\THis\Z')], 
14        ],
15    ]));
16}

Signing the Version 4 request

With Version 4, it is not only necessary to now use sha256 as our signing algorithm, but the process is a little more involved. You can read more about the signing process here under the Calculating a Signature heading, but the process is reasonably straight forward: for each part in the signature chain, use the previous hashed value as the key for the next:

 1function signPolicy($policy) 
 2  function signPolicy($date, $policy) 
 3{
 4     return with($policy, function ($policy) { 
 5         return base64_encode(hash_hmac('sha1', $policy, config('filesystems.disks.s3.secret'), true)); 
 6     });  
 7     $dateKey = hash_hmac('sha256', $date->format('Ymd'), 'AWS4'.config('filesystems.disks.s3.secret'), true); 
 8     $dateRegionKey = hash_hmac('sha256', config('filesystems.disks.s3.region'), $dateKey, true); 
 9     $dateRegionServiceKey = hash_hmac('sha256', 's3', $dateRegionKey, true); 
10     $signingKey = hash_hmac('sha256', 'aws4_request', $dateRegionServiceKey, true); 
11 
12     return hash_hmac('sha256', $policy, $signingKey); 
13}

As you can see in the diff, we take the hashed value of the date and use it as the key to sign the region. The hashed value is used as the key to sign the region, the region for the service, until we arrive at the last hashed value - the signing key for the aws4_request itself.

Each step of hashing process uses raw output, as indicated by the fourth true parameter with exception of the final signature, which returns the string representation.

Updated form view

The last step involves updating the route housing our form with the additional $date and $credential fields, as well as updating calls to the functions responsible for creating and signing our policy.

 1Route::get('/', function () {
 2    return view('welcome', [
 3         'policy' => $policy = createPolicy(), // [tl! --]
 4         'signature' => signPolicy($policy), // [tl! --]
 5         'date' => $date = now(), // [tl! ++]
 6         'credential' => $credential = createCredential($date), // [tl! ++]
 7         'policy' => $policy = createPolicy($credential), // [tl! ++]
 8         'signature' => signPolicy($date, $policy), // [tl! ++]
 9    ]);
10});

Conclusion

In this addendum to part one on uploading files to Amazon S3 from the browser, we've clarified when to use Version 2 and Version 4, based on availability in your S3 region.

At this point, I would suggest using AWS Signature Version 4 for all requests as it is available in all regions and the only option in some others, per Amazon's documentation.

We looked at what is involved in generating upload credentials, and what needs to change in your policy document and signing process for a Version 4 request.

In Part Two of this series, we'll look at taking what we have started with here and provide a more seamless user experience by leveraging DropzoneJS to handle the upload and callback with the user never having to leave your site.