Fargate and EFS

One of the awesome features that we’ve been looking out for is native support for EFS in AWS Elastic Container Service (ECS) and specifically the serverless offering of Fargate. This was announced early April 2020 and it opens up a whole lot of possibilities for those containerised applications requiring a shared filesystem.

While this was achievable with a bit more code and some extra work in ECS, having native support means less moving parts, simpler code, easier to manage and just a better overall solution.

Before we move onto a working example lets have a quick overview of these services.

And finally some key points and traps to be aware of:

  • Platform version of 1.4.0 of Fargate is required.
  • LATEST tag defaults to 1.3.0 so you have to make sure you use version 1.4.0
  • LATEST tag will be updated to point to 1.4.0 in May 2020 according to AWS.

I’ve based my example from this AWS tutorial.

We’ll be using the aws cli here as CloudFormation support is still a work in progress. I’ve put together a script to orchestrate all these commands so if you if you just want to quickly get a working version scroll to the bottom and see the details in the repo.

Firstly we need to set up some environment variables as we’ll be using these in our commands. We need variables for VPC_ID, PUBLIC_SUBNET_ID and PRIVATE_SUBNET_ID. We’ll be creating our Fargate container in a public subnet while our EFS will live in a private subnet.

export VPC_ID=<your vpc id>
export PUBLIC_SUBNET_ID=<your public subnet id>
export PRIVATE_SUBNET_ID=<your private subnet id>

Lets start by creating our security group which we’ll use on both the container and EFS.

aws ec2 create-security-group --group-name fargate-efs-sg \
  --description "Security group for fargate efs example" --vpc-id "$VPC_ID"

Add some ingress rules to our security group. Please substitute or set the variable SG_ID to the security group created above. eg. sg-123456. As our container runs in the public subnet I am locking down access to only my public ip with “curl -s checkip.amazonaws.com”. We also need to open up 2049 so that the container can talk to EFS.

# Authorise your ip only.
aws ec2 authorize-security-group-ingress \
  --group-id "$SG_ID" \
  --protocol tcp \
  --port 80 \
  --cidr "$(curl -s checkip.amazonaws.com)/32"

# Ingress to efs via sg only
aws ec2 authorize-security-group-ingress \
  --group-id "$SG_ID" \
  --protocol tcp \
  --port 2049 \
  --source-group "$SG_ID"

Now we create our EFS filesystem.

aws efs create-file-system

Once that is done, it takes a minute or so to be available we need to create mount targets. Grab the filesytem id from the above command and substitute for variable FILE_SYSTEM_ID.

# Create a mount target in a private subnet.
aws efs create-mount-target \
--file-system-id "$FILE_SYSTEM_ID" \
--subnet-id "$PRIVATE_SUBNET_ID" \
--security-group "$SG_ID"

We can now create our ECS Cluster.

aws ecs create-cluster --cluster-name fargate-cluster

Now we need to create our task definition. Copy the following code and paste into a file called fargate-task.json. Substitute the FILE_SYSTEM_ID variable in the file for your actual filesystem id.

The task is a simple Apache container which has an index.html page containing some details of efs with “df -h /mount/efs > /usr/local/apache2/htdocs/index.html”.

{
    "family": "fargate-efs",
    "networkMode": "awsvpc",
    "containerDefinitions": [
        {
            "name": "fargate-app",
            "image": "httpd:2.4",
            "portMappings": [
                {
                    "containerPort": 80,
                    "hostPort": 80,
                    "protocol": "tcp"
                }
            ],
            "essential": true,
            "entryPoint": [
                "sh", "-c"
            ],
            "command": [
                "/bin/sh -c \"df -h /mount/efs > /usr/local/apache2/htdocs/index.html && httpd-foreground\""
            ],
            "mountPoints": [
                {
                    "sourceVolume": "fargate-efs",
                    "containerPath": "/mount/efs",
                    "readOnly": false
                }
            ]
        }
    ],
    "volumes": [{
      "name": "fargate-efs",
      "efsVolumeConfiguration": {
         "fileSystemId": "$FILE_SYSTEM_ID",
         "rootDirectory": "/"
      }
    }],
    "requiresCompatibilities": [
        "FARGATE"
    ],
    "cpu": "256",
    "memory": "512"
}

Once we have the file saved with the correct filesystem id we can now register the task definition with.

# Register the task definition
aws ecs register-task-definition --requires-compatibilities FARGATE \
  --cli-input-json file://fargate-task.json

Now we can create the service. You will need to substitute your TASK_ARN from above along with PUBLIC_SUBNET_ID and SG_ID.

# Create the service.
aws ecs create-service --cluster fargate-cluster \
  --service-name fargate-service --task-definition "$TASK_ARN" \
  --desired-count 1 --launch-type "FARGATE" \
  --network-configuration \
    "awsvpcConfiguration={subnets=[$PUBLIC_SUBNET_ID],securityGroups=[$SG_ID],assignPublicIp=ENABLED}" \
  --platform-version 1.4.0 \
  --tags key=Name,value=fargate-efs \
  --enable-ecs-managed-tags

The service takes a minute or two to startup so after a little wait we need to get the containers public ip. You can log on to the console and get it from there or run the following script.

aws ec2 describe-network-interfaces \
    --filters Name=tag:"aws:ecs:serviceName",Values=fargate-service

Once we have the public ip we can now test to see if it all works. Substitute PUBLIC_IP from the value above. We put in some retries for our curl command as the container might not be completely running.

curl --retry-connrefused --retry 10 http://$PUBLIC_IP

And finally we should see some output like.

Filesystem                                      Size  Used Avail Use% Mounted on
fs-abc123.efs.ap-southeast-2.amazonaws.com:/    8.0E     0  8.0E   0% /mount/efs

You can see most of the code in our Github repo. Its been orchestrated into a run.sh script.

Finally that’s it. EFS inside Fargate. Pretty easy and an exciting feature that will be another great option for applications in AWS.