[SOLVED] Why memory mapping to `STDOUT_FILENO` failed?

Issue

I have written some code to test the mmap system call.

Here I want to map the virtual memory address space to the STDOUT, and print a string via the pointer ptr that returned by the mmap.

int main()
{
    void *ptr = mmap(NULL, 1024, PROT_WRITE | PROT_READ, MAP_PRIVATE, STDOUT_FILENO, 0);
    memcpy(ptr, "hello", 6);
}

But this code failed:

$ gcc mmap.c
$ ./a.out
Segmentation fault (core dumped)

And I have tested mmap on STDIN, it is ok.

int main()
{
    // executed by `./a.out < text.txt`
    void *ptr = mmap(NULL, 1024, PROT_WRITE | PROT_READ, MAP_PRIVATE, STDIN_FILENO, 0);
    write(STDOUT_FILENO, ptr, 1024); 
}

Why mmap on STDOUT failed here? And what are the differences between mmap on STDIN and STDOUT ?

Solution

As a general rule, when you use services, check the error codes to narrow down the problem. Re-writing your program as follow:

#include <errno.h>
#include <stdio.h>
#include <sys/mman.h>
#include <string.h>
#include <unistd.h>

int main(void)
{
    void *ptr = mmap(NULL, 1024, PROT_WRITE | PROT_READ, MAP_PRIVATE, STDOUT_FILENO, 0);
    if (MAP_FAILED == ptr) {
      fprintf(stderr, "mmap(): error '%m' (%d)\n", errno);
      return 1;
    } else {
      memcpy(ptr, "hello", 6);
    }

    return 0;
}

The program displays the following:

$ ./a.out
mmap(): error 'No such device' (19)

That is to say that you get the ENODEV error code. Looking at the manual, you get the following explanation:

ENODEV The underlying filesystem of the specified file does not support memory mapping.

Actually, the file descriptor is pointing on the terminal device driver and the latter does not allow the mmap() operation.

Concerning the second program mapping STDIN_FILENO, when you run it like that:

$ ./a.out < text.txt

The preceding makes STDIN_FILENO "point" on the text.txt file, not the input terminal. Hence, mmap() works here...

So, with the same idea, we expect that the first program works with:

$ ./a.out > foo.txt

Since the output is no longer a terminal but a file. But you get the EACCES error:

$ ./a.out > foo.txt
mmap(): error 'Permission denied' (13)

The answer is explained in this post. A file must be opened with the read access to be mapped but the standard output of a program launched by the shell is opened O_WRONLY. Hence, the mmap() fails. Under Linux, it is possible to use a trick. The file descriptors are symbolic links in /proc/pid/fd directory. So, it is possible to reopen the file pointed by the symbolic link /proc/pid/fd/1 with O_RDWR flag and use the famous close()/dup() trick to make the file descriptor number 1 (STDOUT_FILENO) point on this newly opened file. A call to ftruncate() is necessary to reserve space in the file and MAP_SHARED flag is required to make visible the modifications to other processes (e.g. the shell when the program terminates):

#include <errno.h>
#include <stdio.h>
#include <sys/mman.h>
#include <string.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>

int main(void)
{
    char symlink[256];
    int fd;

    snprintf(symlink, sizeof(symlink), "/proc/%d/fd/1", getpid());
    fd = open(symlink, O_RDWR);
    if (fd < 0) {
      fprintf(stderr, "fopen(): error '%m' (%d)\n", errno);
      return 1;
    }
    close(STDOUT_FILENO);
    dup(fd);
    close(fd);
    ftruncate(STDOUT_FILENO, 1024);
    void *ptr = mmap(NULL, 1024, PROT_WRITE | PROT_READ, MAP_SHARED, STDOUT_FILENO, 0);
    if (MAP_FAILED == ptr) {
      fprintf(stderr, "mmap(%d): error '%m' (%d)\n", fd, errno);
      return 1;
    } else {
      memcpy(ptr, "hello", 6);
    }

    return 0;
}

Hence, you get what you expect:

$ ./a.out
mmap(3`): error 'No such device' (19) # The output is the terminal (mmap() forbidden)
$ ./a.out > foo.txt   # The output is a file
$ cat foo.txt
hello$

Answered By - Rachid K.