[SOLVED] How to build docker image for multiple platforms with cross-compile?

Issue

I'm trying to build a multi platform (linux/amd64,linux/arm64) node image. As far as I understood I can try the corss-compile approach. Building with docker buildx and pushing to the docker registry did not throw any errors. Running the image on the build machine also works fine, but on my raspberry-pi4 (running ubuntu 64bit) following occurs after pulling the image

exec /usr/local/bin/docker-entrypoint.sh: exec format error

My Dockerfile:

FROM --platform=$BUILDPLATFORM node:18-alpine as node
WORKDIR /usr/src/app

RUN echo "console.log('hello world!');" > main.js
EXPOSE 4000

CMD [ "node", "main.js" ]

I builded and pushed my image the following way:

docker buildx build --platform=linux/amd64,linux/arm64 -t buntel/node-multi-arch --push .

I ran my container the following way (no problem on my laptop, but fails on my raspberry-pi):

docker run --rm -it buntel/node-multi-arch

In the docker registry I also see tags for both platforms https://hub.docker.com/r/buntel/node-multi-arch/tags. What's going wrong here?

Additional info:

docker buildx ls:

NAME/NODE DRIVER/ENDPOINT STATUS BUILDKIT PLATFORMS loving_tesla * docker-container
loving_tesla0 unix:///var/run/docker.sock running v0.10.4 linux/amd64, linux/amd64/v2, linux/amd64/v3, linux/386 default
docker default default running 20.10.18 linux/amd64, linux/386

docker buildx inspect:

Name: loving_tesla Driver: docker-container

Nodes: Name: loving_tesla0 Endpoint: unix:///var/run/docker.sock Status: running Buildkit: v0.10.4 Platforms: linux/amd64, linux/amd64/v2, linux/amd64/v3, linux/386

laptop: docker version

Client: Version: 20.10.18 API version: 1.41 Go version: go1.19.1 Git commit: b40c2f6b5d Built:
Sat Sep 10 11:31:10 2022 OS/Arch: linux/amd64 Context:
default Experimental: true

Server: Engine: Version: 20.10.18 API version: 1.41 (minimum version 1.12) Go version: go1.19.1 Git commit:
e42327a6d3 Built: Sat Sep 10 11:30:17 2022 OS/Arch:
linux/amd64 Experimental: false containerd: Version:
v1.6.8 GitCommit: 9cd3357b7fd7218e4aec3eae239db1f68a5a6ec6.m runc: Version: 1.1.4 GitCommit: docker-init:
Version: 0.19.0 GitCommit: de40ad0

raspberry-pi docker version:

Client: Docker Engine - Community Version: 20.10.18 API version: 1.41 Go version: go1.18.6 Git commit:
b40c2f6 Built: Thu Sep 8 23:10:58 2022 OS/Arch:
linux/arm64 Context: default Experimental: true

Server: Docker Engine - Community Engine: Version:
20.10.18 API version: 1.41 (minimum version 1.12) Go version: go1.18.6 Git commit: e42327a Built: Thu Sep 8 23:09:16 2022 OS/Arch: linux/arm64 Experimental:
false containerd: Version: 1.6.8 GitCommit:
9cd3357b7fd7218e4aec3eae239db1f68a5a6ec6 runc: Version:
1.1.4 GitCommit: v1.1.4-0-g5fd4c4d docker-init: Version: 0.19.0 GitCommit: de40ad0

UPDATE:

In my case, I didn't need to emulate anything, because I didn't handle binary builds. So for me, the key was to use --platform=$BUILDPLATFORM on my build stage and --platform=$TARGETPLATFORM on the final stage. This will use the architecture of the machine building the docker image to build my JavaScript/Typescript. And then switch to the target architecture when collecting the build artifacts, which can then be run by those machines.

Example:

FROM node:18-alpine as node
WORKDIR /usr/src/app

RUN echo "console.log('hello world!');" > main.js


FROM --platform=$TARGETPLATFORM node:18-slim
WORKDIR /usr/src/app

COPY --from=node /usr/src/app/main.js .

CMD [ "node", "main.js" ]

EXPOSE 4000

Solution

The issue is you're using BUILDPLATFORM, which will always be the same architecture as the host machine. In this case, it will always be x86_64. What you want is the TARGETPLATFORM, which will be implicit anyway. You can see this in an explanation of multi-platform Docker builds.

You can get the Docker info using docker version | grep 'OS/Arch' (this will give the client first, then the server).

What you actually want is to use the node:18-alpine image for the linux/arm64 when building for that platform. Changing it to the following will work:

FROM node:18-alpine as node
WORKDIR /usr/src/app

RUN echo "console.log('hello world!');" > main.js
EXPOSE 4000

CMD [ "node", "main.js" ]

If you still want to build for linux/amd64 and run it on your ARMv8 machine, you would have to install binfmt to register these x86_64 binaries with the correct Qemu emulator. This would be then emulating x86_64 on your ARM machine, which you almost certainly do not want to do.

Answered By - Alex Huszagh

Answer Checked By - David Goodson (WPSolving Volunteer)