Issue
I am utilizing yocto (dunfell) to cross-compile a project for multiple different architectures. Specifically, the targets I have are a 64-bit RaspberryPi4 (aarch64) and a 32-bit Orange Pi (armhf). My project that I am cross-compiling compiles and runs without issue when building for the raspi target; the runtime linker is properly set and things run without issue. However, whenever I build for the Orange Pi target, the program appears to compile without issue, but when I try to execute it on the platform, I get a "File not found" error.
This appears to be because the interpreter (runtime linker) is set to /usr/lib/ld.so
which is not actually on the system. See below:
$ file my-exec
my-exec: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /usr/lib/ld.so.1, with debug_info, not stripped
In contrast, when I build the same program for the raspi target, the interpreter seems to be set properly for the system:
$ file my-exec
my-exec: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, with debug_info, not stripped
This issue seems to be isolated to just this project on the Orange Pi target. In comparison, other projects on the Orange Pi target look like the following and run without issue:
$ file other-exec
other-exec: ELF 32-bit LSB shared object, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 3.2.0, stripped
What I am trying to understand is the following:
- Which things could possibly be influencing the interpreter which is chosen at compile time? I have done some digging into this and nothing is explicitly set in my Makefile which could be causing this. Additionally, when I build a simple hello world application, the correct interpreter is chosen. It seems to be some set of flags in my compiling/linking which is causing this. Below is the compiling/linking output (I have excluded all the libraries it is also building which use the same arguments):
NOTE: make -j 24 i2c-core VERBOSE=1 CC=arm-poky-linux-gnueabi-gcc -march=armv7ve -mthumb -mfpu=neon -mfloat-abi=hard -fstack-protector-strong -D_FORTIFY_SOURCE=2 -Wformat -Wformat-security -Werror=format-security --sysroot=/home/my-yocto-os/build/tmp/work/armv7vet2hf-neon-poky-linux-gnueabi/my-exec/1.0+git999-r0/recipe-sysroot LD=arm-poky-linux-gnueabi-ld --sysroot=/home/my-yocto-os/build/tmp/work/armv7vet2hf-neon-poky-linux-gnueabi/my-exec/1.0+git999-r0/recipe-sysroot
arm-poky-linux-gnueabi-gcc -march=armv7ve -mthumb -mfpu=neon -mfloat-abi=hard -fstack-protector-strong -D_FORTIFY_SOURCE=2 -Wformat -Wformat-security -Werror=format-security --sysroot=/home/my-yocto-os/build/tmp/work/armv7vet2hf-neon-poky-linux-gnueabi/my-exec/1.0+git999-r0/recipe-sysroot -I.. -I. -g -std=gnu99 -Wall -Wextra -pedantic -Wshadow -Wpointer-arith -Wcast-align -Wwrite-strings -Wmissing-prototypes -Wmissing-declarations -Wredundant-decls -Wnested-externs -Winline -Wno-long-long -Wconversion -Wstrict-prototypes -DBUILD_CORE -c -o build/other.o ../other.c
...
arm-poky-linux-gnueabi-ld --sysroot=/home/my-yocto-os/build/tmp/work/armv7vet2hf-neon-poky-linux-gnueabi/my-exec/1.0+git999-r0/recipe-sysroot build/my-exec.o build/other.o build/other1.o -luv -lsystemd -lc --entry main -o build/exe/my-exec
- The linker which is being set (
/usr/lib/ld.so.1
) seems to be a really common linker to use, so why doesn't it actually end up on my system? - Am I fundamentally misunderstanding something about the ARM architecture which is resulting in this outcome? Are there different flags I could set at linking which would resolve this?
One further piece of information I have, which may or may not be relevant is the following:
An issue I hit while building for the Raspi target (showed up on both targets, actually) was the compiler wasn't able to find the entry symbol. This is quite a mature project and cross-compilers have been used to compile it, historically, but in a way that isn't yocto. This has never been an issue in the past. It wasn't until I added --entry main
to the linker flags that this issue was resolved. I am wondering if there is a more fundamental problem which is resulting in these odd outcomes.
Thank you for any help you can provide on this. Please let me know if you have any additional questions about my environment.
Solution
After a few days of debugging, I figured out there problem. If anyone with more knowledge than I on linking would like to chime in to add things, please do. Ultimately, this was resolved by using gcc
as the linker as opposed to using ld
(the ones provided by yocto's cross compiler; i.e. aarch64-poky-linux-gcc
).
In order to do this, I modified my recipe to pass in LD=${CC} LDFLAGS=${LDFLAGS}
to my Makefile. Now, it builds and executes properly for both the RPi and OrangePi targets.
I believe this is mainly the case because the LDFLAGS
provided by yocto actually can't be parsed by ld
. From my research, it looks like ld
is typically invoked by gcc
. However, the flags still need to get to the complier. So, originally, LDFLAGS
that needed to be passed into linking, weren't being passed in at all because I just assumed there was an error with doing it that way. So, be sure you're passing your LDFLAGS
that yocto gives you into gcc
.
Answered By - Mitch