Wednesday, February 7, 2024

[SOLVED] .NET (Core) on Unix: In-process modification of environment variables via P/Invoke system calls is unexpectedly not seen by .NET

Issue

A notable limitation of the .NET environment-variable APIs as of .NET 8 is the inability to define environment variables without a value or, to put it differently, with a value that is the empty string.

Both Windows and Unix-like platforms do support this natively, however, via system calls, which is why calling the latter directly is of interest.

While using P/Invoke declarations to access the relevant system calls works as expected on Windows, it does not on Unix-like platforms, as of .NET 8:

  • .NET does not see the P/Invoke-mediated environment modifications and continues to report the old environment, both via Environment and when launching child processes.

Is this a bug, or is the code below missing something?

Sample C# console-application code, tested with version 8.0.101 of the .NET SDK:

  • Run on a Unix-like platform (Linux or MacOS), as the content of the Program.cs file in a project created with dotnet new console

  • Add <AllowUnsafeBlocks>true</AllowUnsafeBlocks> inside the <PropertyGroup> element of the *.csproj file in order for the project to compile.

using System.Diagnostics;
using System.Runtime.InteropServices;

partial class Program
{
  [LibraryImport("libc", StringMarshalling = StringMarshalling.Utf8)]
  private static partial int setenv(string name, string value, int overwrite);

  [LibraryImport("libc", StringMarshalling = StringMarshalling.Utf8)]
  private static partial IntPtr getenv(string name); // Convert to string with Marshal.PtrToStringUTF8()

  [LibraryImport("libc", StringMarshalling = StringMarshalling.Utf8)]
  private static partial int system(string command);

  static void ThrowSysCallError() => throw new System.ComponentModel.Win32Exception(Marshal.GetLastSystemError());

  static void Main(string[] args)
  {
    // Define / set env. var. 'FOO' with / to value 'new' 
    if (-1 == setenv("FOO", "new", 1)) ThrowSysCallError();

    Console.WriteLine(
      $"In-process value of FOO after setting it to 'new', via syscall: [{Marshal.PtrToStringUTF8(getenv("FOO"))}]"
    );

    Console.Write(
      "Value of FOO in a syscall-launched child process: "
    );
    system("echo [$FOO]");

    Console.WriteLine(
      $"In-process value of FOO per Environment.GetEnvironmentVariable(\"FOO\"): [{Environment.GetEnvironmentVariable("FOO")}]"
    );

    Console.Write(
      "Value of FOO in a .NET-launched child process: "
    );
    Process.Start(
      new ProcessStartInfo()
      {
        FileName = "sh",
        ArgumentList = { "-c", @"echo [$FOO]" },
      }
    )?.WaitForExit();

  }

}

The expected output would be:

In-process value of FOO after setting it to 'new', via syscall: [new]
Value of FOO in a syscall-launched child process: [new]
In-process value of FOO per Environment.GetEnvironmentVariable("FOO"): [new]
Value of FOO in a .NET-launched child process: [new]

The actual output is - note how .NET apparently doesn't see the FOO variable that the setenv() syscall set:

In-process value of FOO after setting it to 'new', via syscall: [new]
Value of FOO in a syscall-launched child process: [new]
In-process value of FOO per Environment.GetEnvironmentVariable("FOO"): []
Value of FOO in a .NET-launched child process: []

Solution

This was an interesting rabbit hole, thank you.

If we look at the runtime source, GetEnvironmentVariable gets delegated to the platform-specific GetEnvironmentVariableCore.

On Windows, this results in the appropriate interop call to Kernel32 (also when Set-ing). Nothing to see there.

On Unix, things look a little more interesting. It contains a static dictionary cache s_environment. If we're very lucky and it hasn't been populated yet, we'll get our desired interop call to getenv.

Unfortunately, any call to GetEnvironmentVariables (the plural version) and SetEnvironmentVariable will populate the cache - courtesy of EnsureEnvironmentCached.

Once that happens, all operations will happen on the cache. So if anything in the runtime has called either of them (it does. in so many places), we're out of luck.

This explains our first missing value. Calling GetEnvironmentVariable will likely end up with us being served a cached value, skipping the getenv call.

But what about the child process? Shouldn't that inherit the process's environment - which we *did* set via setenv?

If only we were so lucky. Like before, Process.Start() gets delegated to the platform-specific StartCore. Here it is in all its glory. Now notice the string[] envp = CreateEnvp(startInfo); line. If we keep digging, this envp keeps getting passed deeper, eventually ending up in the execve syscall, after forking.

This is a problem because CreateEnvp gets its values from ProcessStartInfo.Environment, which gets its values from... our good friend Environment.GetEnvironmentVariables() which we know from the last episode will serve us cached values.

The take home message from all this is that, in Unix/Linux, we should be safe if we stick to Environment.[Set,Get]EnvironmentVariable calls, as we'll be confined to operating with cached values.

But as you've established in your question, we're out of luck if we want to use empty variables.



Answered By - NPras
Answer Checked By - Candace Johnson (WPSolving Volunteer)
at