Building code that executes a program and redirects the standard output and standard error seems like an easy task when you have access to .NET.

At first glance it is as simple as:

var startInfo = new ProcessStartInfo("C:\\Windows\\System32\\diskperf.exe","/?")
{
	RedirectStandardOutput = true,
	RedirectStandardError = true,
	UseShellExecute = false
};

using (var process = Process.Start(startInfo)) 
{
	if (process.WaitForExit(100)) 
	{
		Console.WriteLine(process.StandardOutput.ReadToEnd());		
	}
}

Which seems like the appropriate way of doing this, execute process, wait for it to finish and then read the output.

So far so good and its all well documented in the documentation

The documentation explains that to avoid potential deadlocks, you should read from the output/error streams before waiting for the process to exit, but it fails to explain why - except that it can cause deadlocks.

I had a piece of code that was written more or less like the example above and it worked perfectly every single time, except when the output from the process was longer than a certain amount of characters then it just hang.

Digging into the code its quite simple - the problem is the streams being created are dependent on windows named pipes which states:

“When a process uses WriteFile to write to an anonymous pipe, the write operation is not completed until all bytes are written. If the pipe buffer is full before all bytes are written, WriteFile does not return until another process or thread uses ReadFile to make more buffer space available.”

This is what happens when the contents of the process output is too big and you have written your code like the example above

1. Process start - and starts writing to the output stream - and at some point the buffer runs full and then it blocks (according to the documentation for named pipes)
2. WaitForExit - waits for the process to exit, but process cannot exit because its blocked from writing all its output to the output stream because code has not called ReadToEnd yet.

So if output of process could be contained within the default buffer size, then the above code snippet would work perfectly, since the process would not be blocked from writing to the output stream, but if the process output was bigger than the size of the buffer you have effectively created a deadlock, where the process waits for your program to read the output stream so more room comes available and your program waits for the process to exit.

So to make the code robust you should either do what the documentation says and read from the stream before WaitForExit - or use the asynchronous methods for reading from the streams.

var startInfo = new ProcessStartInfo("C:\\Windows\\System32\\diskperf.exe","/?")
{
	RedirectStandardOutput = true,
	RedirectStandardError = true,
	UseShellExecute = false
};

var output = new List<string>();
var error = new List<string>();
using (var process = new Process())
{
	process.StartInfo = startInfo;
    // Set up event handlers that receives the output and add it to the list of strings
    process.OutputDataReceived += (s, e) => { lock (output) { output.Add(e.Data); } };
	process.ErrorDataReceived += (s, e) => { lock (error) { output.Add(e.Data); } };

    // start the process
	process.Start();
    
    // Start the async reads from the streams
	process.BeginErrorReadLine();
	process.BeginOutputReadLine();

	if (!process.WaitForExit(100))
	{
		Console.WriteLine("Timed out waiting");
	}
	Console.WriteLine(string.Join("\r\n", output));
	Console.WriteLine(string.Join("\r\n", error));
}

Of course the example above is not the most optimal way of appending the output and error streams, but its simple and for the purpose of showing how to use the asynchronous methods correctly its good enough.

In a real application where performance and scalability matters you should probably consider re-using either the list of strings, or use a stream writer instead with buffers that are re-used.