[SOLVED] How to write the output of an executable binary into memory rather than disk

Issue

I got a binary that works like the below:

> ./my_bin raw.avi output_file.avi

output_file.avi is what I want, some verbose information will print in the terminal when the job is succeeded, like:

Copyright 2022 Company Inc... Success.

I want to run this command inside my code and redirect the output_file.avi into some byte array so that I don't have to read it from disk and delete it. My approach looks like the below Golang snippet:

func wrongOne(stdin []byte) ([]byte, error) {
    inBuf := bytes.NewBuffer(stdin)
    outBuf := bytes.NewBuffer(nil)
    cmd := exec.Command("./my_bin", "/dev/stdin", "/dev/stdout")
    cmd.Stdin = inBuf
    cmd.Stdout = outBuf
    err := cmd.Run()
    if err != nil {
        return nil, err
    }
    return outBuf.Bytes(), nil // wrong
}

However, the return byte array is longer than the below approach, which leads to failure on the MD5 check.

func correctOne(stdin []byte) ([]byte, error) {
    inBuf := bytes.NewBuffer(stdin)
    cmd := exec.Command("./my_bin", "/dev/stdin", "output_file")
    cmd.Stdin = inBuf
    err := cmd.Run()
    if err != nil {
        return nil, err
    }
    return os.ReadFile("output_file")
}

the wrongOne function can be modified to following code to be correct:

func modifiedWrongOne(stdin []byte) ([]byte, error) {
    inBuf := bytes.NewBuffer(stdin)
    outBuf := bytes.NewBuffer(nil)
    cmd := exec.Command("./my_bin", "/dev/stdin", "/dev/stdout")
    cmd.Stdin = inBuf
    cmd.Stdout = outBuf
    err := cmd.Run()
    if err != nil {
        return nil, err
    }
    correct, _ := correctOne(stdin)
    return outBuf.Bytes()[:len(correct)], nil // diff
}

I presume that the output verbose information is included in the /dev/stdout so that the wrongOne function doesn't works. i.e.,

the output of wrongOne = the output of correctOne + []byte{"Copyright 2022 Company Inc... Success."}

Is there any solution that I can get the output_file.avi in the pipe without save it as file and read it from disk? Thanks!

Solution

The command writes the copyright notice to stdout. To avoid commingling the copyright notice with the output file, use a file other than /dev/stdout as the output file.

The function below uses Cmd.ExtraFiles to connect a pipe to fd 3 in the child process. The function copies data from the pipe to a byte buffer and returns those bytes to the caller.

func otherOne(stdin []byte) ([]byte, error) {
    r, w, err := os.Pipe()
    if err != nil {
        return nil, err
    }
    defer r.Close()
    defer w.Close()

    cmd := exec.Command("./my_bin", "/dev/stdin", "/dev/fd/3")
    cmd.Stdin = bytes.NewReader(stdin)
    cmd.ExtraFiles = []*os.File{w} // The first file is fd 3.
    if err := cmd.Start(); err != nil {
        return nil, err
    }
    w.Close()
    var outbuf bytes.Buffer
    if _, err := io.Copy(&outbuf, r); err != nil {
        return nil, err
    }
    if err := cmd.Wait(); err != nil {
        return nil, err
    }
    return outbuf.Bytes(), nil
}

Answered By - Cerise Limón

Answer Checked By - Robin (WPSolving Admin)