[SOLVED] Why does mremap on two consecutive shared memory regions result in SIGBUS?

Issue

For context: I am implementing an x86 linux emulator and am trying to wrap my head around understanding what the mremap syscall does. The only case that currently does not make much sense to me is the case where old_len is set to 0.

As far as i understand the docs this can be used to make a new mapping of an already mapped shared memory area, effectively creating a second virtual address mapping to the same physical memory. (I.e. it would be possible to access the same physical memory via the two different virtual address ranges)

So far so good. This works as expected when i try it. However, when there are two consecutive memory regions, e.g., 0xf1000..0xf4000 and 0xf4000..0xf7000 this results in a SIGBUS when writing across the boundary of the regions (after mremapping). See the example below:

#define _GNU_SOURCE 1
#include <stdio.h>
#include <sys/mman.h>

int main() {
    char* mapping = mmap((void *) 0xf1000, 0x3000, PROT_READ | PROT_WRITE, MAP_SHARED | MAP_ANONYMOUS, -1, 0);
    char* mapping2 = mmap((void *) 0xf4000, 0x3000, PROT_READ | PROT_WRITE, MAP_SHARED | MAP_ANONYMOUS, -1, 0);
    char* mapping3 = mremap((void *) 0xf3000, 0x0, 0x3000, MREMAP_MAYMOVE | MREMAP_FIXED, 0xf9000);
    for (int i = 0; i < 0x3000; ++i) mapping3[i] = 'A';
    printf("%c\n", mapping[0x2000]);
    printf("Mapping pointer: %p\n", mapping);
    printf("Mapping2 pointer: %p\n", mapping2);
    printf("Mapping3 pointer: %p\n", mapping3);
    return 0;
}

If the loop only iterates over the first 0x1000 characters, then the changes are reflected in mapping as i would expect.

Why does this result in a SIGBUS and why does the call to mremap not fail, if this is not allowed?

Solution

So after looking into this, it appears you cannot do this mremap across maps thing with MAP_SHARED the way you can with MAP_PRIVATE.

If you change the mmap calls to use MAP_PRIVATE, you hit this error:

    if (!old_len && !(vma->vm_flags & (VM_SHARED | VM_MAYSHARE))) {
        pr_warn_once("%s (%d): attempted to duplicate a private mapping with mremap.  This is not supported.\n", current->comm, current->pid);
        return ERR_PTR(-EINVAL);
    }

... which I can actually verify as that message appeared in dmesg:

[ 3566.135161] a.out (4234): attempted to duplicate a private mapping with mremap.  This is not supported.

If you change, the mremap call to not do the old_len == 0 trick:

char* mapping3 = mremap((void *) 0xf3000, 0x3000, 0x3000, MREMAP_MAYMOVE | MREMAP_FIXED, 0xf9000);

... you will get errno of EFAULT (bad address). This appears to get caught here:

    /* We can't remap across vm area boundaries */
    if (old_len > vma->vm_end - addr)
        return ERR_PTR(-EFAULT);

As you can see, the old_len == 0 will skip this statement (assuming intentionally).

When using MAP_SHARED, this is managed by the shmem system which appears to operate a bit differently in terms of map boundaries. However, I could not pinpoint exactly where in the source this is decided. I think this is related to the fact that MAP_SHARED | MAP_ANONYMOUS mappings cannot grow. So you hit the same error if you were to grow a MAP_SHARED | MAP_ANONYMOUS mapping. It will silently succeed, but then throw a bus error when you read/write into the newly allocated section.

Your bus error occurs when you try to write into the first page of what is originally defined as mapping2. So it would seem, this is just something you cannot do. As it stands, it would be better to think of MAP_SHARED | MAP_ANONYMOUS mappings as distinct units when using mremap.

Answered By - Jason

Answer Checked By - Dawn Plyler (WPSolving Volunteer)