Tuesday, April 12, 2022

[SOLVED] ALB results in 504 gateway time out error with ECS

Issue

I have an httpd container with ECS service along with ALB.

Container with ALB are using a dynamic port feature which means host port is set to 0. if i try to ssh in an instance container and try to curl localhost:port number it works.

But when i try to use ALB DNS name it turns out to 504.

ALb security group allows HTTP 80 connections from anywhere and instance sg allows any connection on any port from alb sg.

Interestingly

when I try to check the target group associated with alb all the instances are unhealthy.

Update:- i tried to open a security group of ecs container to public and yet the instance were not healthy


Solution

you need to check the events of the ECS service and see what is the exact error message. If it states something like port 45675 is unhealthy then you need to check your security group configuration, it should get rid of 504 error message. If it states health check failed (this should give 502) then you should ssh into the container and check on which port the application is running and create a new service with the modification.

Assuming, you have configured the health check for traffic port and haven't modified it.

httpd service generally works on port 80. So I'll suggest use the container port as 80.

504 is Gateway Timeout error, if the above information doesn't help you can provide look at the AWS link here - https://aws.amazon.com/premiumsupport/knowledge-center/troubleshoot-http-5xx/

If you can share the error message from the ecs events that will help in narrowing down the issue.

Adding the screenshots of the changes I made to fix the issue, I hope it helps. I am assuming you are using the default httpd image -

Attaching Load balancer to the Service

Modified the security groups to allow the health checks on the ephemeral ports

Target Health

Health Check Configuration

Load Balancer Details



Answered By - Gurpreet Singh
Answer Checked By - David Goodson (WPSolving Volunteer)