[SOLVED] user_data is not executed in EC2 with error in cloud-init thru terraform

Issue

I have a simple deployment of an EC2 instance thru terraform in a private network giving it internet access via NAt gw, everything is ok just the fact that I using user_data to parse a script to install a simple web server on this ubuntu instance but I see the following error on cloud-init in the instance in question:

/var/log/cloud-init-output.log

 96 Cloud-init v. 21.1-19-gbad84ad4-0ubuntu1~20.04.2 running 'modules:config' at Wed, 08 Sep 2021 18:17:18 +0000. Up 58.79 seconds.
 97 *** Installing apache2
 98 Err:1 http://security.ubuntu.com/ubuntu focal-security InRelease
 99   Cannot initiate the connection to security.ubuntu.com:80 (2001:67c:1562::18). - connect (101: Network is unreachable) Cannot initiate the conn    ection to security.ubuntu.com:80 (2001:67c:1562::15). - connect (101: Network is unreachable) Could not connect to security.ubuntu.com:80 (91.18    9.91.38), connection timed out Could not connect to security.ubuntu.com:80 (91.189.91.39), connection timed out
100 Err:2 http://us-east-2.ec2.archive.ubuntu.com/ubuntu focal InRelease
101   Could not connect to us-east-2.ec2.archive.ubuntu.com:80 (52.15.155.232), connection timed out Could not connect to us-east-2.ec2.archive.ubun    tu.com:80 (52.15.107.13), connection timed out Could not connect to us-east-2.ec2.archive.ubuntu.com:80 (52.15.106.142), connection timed out Co    uld not connect to us-east-2.ec2.archive.ubuntu.com:80 (52.15.102.108), connection timed out Could not connect to us-east-2.ec2.archive.ubuntu.c    om:80 (52.15.159.198), connection timed out Could not connect to us-east-2.ec2.archive.ubuntu.com:80 (52.15.158.54), connection timed out
102 Err:3 http://us-east-2.ec2.archive.ubuntu.com/ubuntu focal-updates InRelease
103   Unable to connect to us-east-2.ec2.archive.ubuntu.com:http:
104 Err:4 http://us-east-2.ec2.archive.ubuntu.com/ubuntu focal-backports InRelease
105   Unable to connect to us-east-2.ec2.archive.ubuntu.com:http:
106 Reading package lists...
107 W: Failed to fetch http://us-east-2.ec2.archive.ubuntu.com/ubuntu/dists/focal/InRelease  Could not connect to us-east-2.ec2.archive.ubuntu.com:8    0 (52.15.155.232), connection timed out Could not connect to us-east-2.ec2.archive.ubuntu.com:80 (52.15.107.13), connection timed out Could not     connect to us-east-2.ec2.archive.ubuntu.com:80 (52.15.106.142), connection timed out Could not connect to us-east-2.ec2.archive.ubuntu.com:80 (5    2.15.102.108), connection timed out Could not connect to us-east-2.ec2.archive.ubuntu.com:80 (52.15.159.198), connection timed out Could not con    nect to us-east-2.ec2.archive.ubuntu.com:80 (52.15.158.54), connection timed out
108 W: Failed to fetch http://us-east-2.ec2.archive.ubuntu.com/ubuntu/dists/focal-updates/InRelease  Unable to connect to us-east-2.ec2.archive.ubun    tu.com:http:

here's is how I'm doing this thru terraform:

resource "aws_instance" "vray_instance" {
  ami                    = "ami-00399ec92321828f5"
  instance_type          = "t2.micro"
  key_name               = aws_key_pair.vray_key_pair.key_name
  vpc_security_group_ids = [aws_security_group.vray_security_group_web.id]
  subnet_id              = aws_subnet.vray_privated_subnet[0].id
  user_data              = file("${path.cwd}/install_el_apache.sh")

  tags = {
    Name = "Instance  Web Server"
  }
}

and here is the content of my shell script:

       │ File: install_el_apache.sh
───────┼──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
   1 ~ │ #!/bin/bash
   2 ~ │ echo "*** Installing apache2"
   3 ~ │ sudo apt-get update
   4 ~ │ sudo apt-get install -y apache2
   5 ~ │ sudo systemctl start apache2
   6 ~ │ sudo systemctl enable apache2
   7 ~ │ echo "<h1>Web server Task2 with Terraform</h1>" | sudo tee /var/www/html/index.html
   8 ~ │ echo "*** Completed Installing apache2"

any clues or chide on this will be very welcome, I was pointed that the NAT is not created by the time the script is executed but I check in other similar issues that there is not a requirement to do a cath error or enter a delay in the script in order to wait for the creation of this to star processing the script steps.

thanks in advance

+vRay

Solution

I was pointed that the NAT is not created by the time the script is executed

You are almost correct. But in your case it is about your vray_vpc_us_east2a_privated_association. Basically your instances get created before the associations are made, thus they timeout. You should be able to fix that with depends_on:

resource "aws_instance" "vray_instance" {
  ami                    = "ami-00399ec92321828f5"
  instance_type          = "t2.micro"
  key_name               = aws_key_pair.vray_key_pair.key_name
  vpc_security_group_ids = [aws_security_group.vray_security_group_web.id]
  subnet_id              = aws_subnet.vray_privated_subnet[0].id
  user_data              = file("${path.cwd}/install_el_apache.sh")

  tags = {
    Name = "Instance vRay Web Server"
  }
  
  depends_on = [aws_route_table_association.vray_vpc_us_east2a_privated_association]
}

#-------------------Creation of the JumpBox in Privated Subnet---------------------------
#not possible to reach Privated subnet within the VPC!!!
resource "aws_instance" "vray_jumpbox" {
  ami                    = "ami-00399ec92321828f5"
  instance_type          = "t2.micro"
  key_name               = aws_key_pair.vray_key_pair.key_name
  vpc_security_group_ids      = [aws_security_group.vray_security_group.id]
  subnet_id                   = aws_subnet.vray_public_subnet[0].id
  associate_public_ip_address = true
  tags = {
    Name = "Jumpbox vRay"
  }
  
  depends_on = [aws_route_table_association.vray_vpc_us_east2a_privated_association]  
  
}

Your app will still be not accessible over the internet as its in private subnet anyway, but at least the instance should launch only after the route to NAT is present.

Answered By - Marcin