[SOLVED] How to Mount EFS to EC2 Instance with UserData using Pulumi?

Issue

I've been struggling to be able to mount an EFS volume to an EC2 instance on creation with the UserData field. I'm using Pulumi's Go library and what I have looks like the following:

class="lang-golang prettyprint-override">
// ... EFS with proper security groups and mountTarget created above ...

dir := configuration.Deployment.Efs.MountPoint
availabilityZone := configuration.Deployment.AvailabilityZone
region := configuration.Deployment.Region

userdata := args.Efs.ID().ToStringOutput().ApplyT(func(id string) (string, error) {
    script := `
            #!/bin/bash -xe
            exec > >(tee /var/log/user-data.log|logger -t user-data -s 2>/dev/console) 2>&1

            mkdir -p %s
            echo "%s.%s.%s.amazonaws.com:/ %s nfs4 nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport,_netdev 0 0" | tee -a /etc/fstab
            mount -a
            `
    return fmt.Sprintf(script, dir, availabilityZone, id, region, dir), nil
}).(pulumi.StringOutput)


ec2, err := ec2.NewInstance(ctx, fmt.Sprintf("%s_instance", name), &ec2.InstanceArgs{
    // ... (other fields) ...
    UserData: userdata,
    // ... (other fields) ...
})

But when I create all the resources with Pulumi, the UserData script doesn't run at all. My assumption is that the EFS ID isn't resolved in time by the time the EC2 instance is created, but I thought that Pulumi would handle the dependency ordering automatically since the EC2 instance is now dependent on the EFS volume. I also added an explicit DependsOn() to see if that could be the issue, but it didn't help.

Is there something that I am doing wrong? Any help would be appreciated, thank you!

I've tried several variations of the above example. I looked at this example: Pulumi - EFS Id output to EC2 LaunchConfiguration UserData

But couldn't get that to work either.

Solution

I was able to figure it out, the issue ended up being a couple things:

1. The formatting on the inlined script needed to not have tabs.
2. pulumi.Sprintf() ended up working better than using ApplyT().
3. The EFS volume wasn't ready to mount when it tried to do mount -a.

Put together, it now looks like this:

instanceArgs := &ec2.InstanceArgs{
    // ... arg fields ...
}
script := `#!/bin/bash
exec > >(tee /var/log/user-data.log|logger -t user-data -s 2>/dev/console) 2>&1
mkdir -p %s
echo "%s.efs.%s.amazonaws.com:/ %s nfs4 nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport,_netdev 0 0" >> /etc/fstab
EFS_STATUS="unknown"
WAIT_TIME=10
RETRY_CNT=15
while [[ $EFS_STATUS != "\"available\"" ]]; do
  echo "Waiting for EFS to start..."
  sleep $WAIT_TIME
  EFS_STATUS=$(aws efs describe-file-systems | jq '.FileSystems | map(select(.FileSystemId == "%s")) |  map(.LifeCycleState) | .[0]')
done
while true; do
  mount -a -t nfs4
  if [ $? = 0 ]; then
    echo "Successfully mounted EFS to instance."
    break
  fi;
  if [ $RETRY_CNT -lt 1 ]; then
    echo "EFS could not mount after $RETRY_CNT retries."
  fi;
  echo "EFS could not mount, retrying..."
  ((RETRY_CNT--))
  sleep $WAIT_TIME
done`

userData := pulumi.Sprintf(script, mountDir, Efs.ID(), region, mountDir, Efs.ID())
instanceArgs.UserData = userData

ec2, err := ec2.NewInstance(ctx, fmt.Sprintf("%s_instance", name), instanceArgs)

Answered By - brjmwdhl

Answer Checked By - Willingham (WPSolving Volunteer)