title: Process.StandardOutput.Read does not get data reliably and Process.StandardInput.WriteLine does not flush its BaseStream's buffer content
body: <p>I'm trying to make a simple application that communicates with a console program that operates in a similar way to the Windows classical cmd.exe.</p>
  
  <p>The execution sequence is expected to be as follows:</p>
  
  <ol>
  <li>Setup a background worker and run it to read the output.</li>
  <li>Run the process and wait for a short while.</li>
  <li>Feed a command to the Process.StandardInput</li>
  <li>Wait for a few seconds</li>
  <li>Exit the background thread and kill the process</li>
  </ol>
  
  <p>But what's going on is not as expected because:</p>
  
  <ol>
  <li>Not all the output of the cmd.exe is read. Even when excluding the step which writes to the StandardInput</li>
  <li>The string was not passed to the command from the StandardInput even though it has AutoFlush = true.</li>
  </ol>
  
  <p>I also tried Process.StandardInput.Flush() which does nothing
  I also tried to pass it a string padded with spaces to a length larger than 4096 which is the buffer size with no results.
  Nothing happens!! Why?</p>
  
  <p>I tried this on dot net 4.5.2 and 4.7.1</p>
  
  <p>Similar questions exist <a href="https://stackoverflow.com/questions/11292917/why-is-process-standardinput-writeline-not-supplying-input-to-my-process">here</a>, <a href="https://stackoverflow.com/questions/6595698/get-net-process-object-to-flush-input-stream-continously">here</a>, and <a href="https://stackoverflow.com/questions/7612524/parse-output-from-a-process-that-updates-a-single-console-line">here</a> but none of the answers work. Others are implemented in another language. i.e. Java, Delphi etc</p>
  
  <p>This is a simplified version of my code:</p>
  
  <pre><code>BackgroundWorker _outputWorker;
  Process _process;
  StreamWriter _inputWriter;
  TextReader _outputReader;
  
  void Main()
  {
      _outputWorker = new BackgroundWorker { WorkerSupportsCancellation = true };
      _outputWorker.DoWork += OnOutputWorkerDoWork;
      _outputWorker.RunWorkerAsync();
  
      _process = new Process
      {
          EnableRaisingEvents = true,
          StartInfo = new ProcessStartInfo
          {
              FileName = "cmd.exe",
              Arguments = string.Empty,
              UseShellExecute = false,
              CreateNoWindow = true,
              WindowStyle = ProcessWindowStyle.Hidden,
              WorkingDirectory = Directory.GetCurrentDirectory(),
              StandardOutputEncoding = Encoding.UTF8,
              StandardErrorEncoding = Encoding.UTF8,
              RedirectStandardInput = true,
              RedirectStandardOutput = true,
              RedirectStandardError = true
          }
      };
  
      Console.WriteLine("Starting...");
      if (!_process.Start()) return;
      _inputWriter = _process.StandardInput;
      _inputWriter.AutoFlush = true; // does nothing
      _outputReader = TextReader.Synchronized(_process.StandardOutput);
  
      // You can exclude this step too and still not get the expected output
      Thread.Sleep(500);
      _inputWriter.WriteLine("dir");
      _inputWriter.Flush(); // does nothing, private field carpos = 0
      _inputWriter.BaseStream.Flush(); // does nothing, private field carpos = 5 which is equal to length of "dir" command + 2 characters (NewLine \r\n)
      //_inputWriter.WriteLine("dir".PadLeft(4096)); // does nothing
      // also closing the stream does nothing and does something that I can't afford which is closing the exe
      // _inputWriter.Close();
      //
  
      _process.WaitForExit(5000);
      _outputWorker.CancelAsync();
      _process.Kill();
      Console.WriteLine("Done");
  }
  
  void OnOutput(string data)
  {
      // never mind thread safety for now. It's just a single line static call
      Console.WriteLine(data);
  }
  
  void OnOutputWorkerDoWork(object sender, DoWorkEventArgs e)
  {
      const int BUFFER_SIZE = 4096;
  
      StringBuilder builder = new StringBuilder(BUFFER_SIZE);
  
      while (!_outputWorker.CancellationPending)
      {
          /*
          * It'll keep on running untill it's canceled to reduce thread costs
          * because the program will run different executables sequentially which 
          * all are similar to cmd.exe.
          */
          try
          {
              // Simplified version without locking
              if (_outputReader == null) continue;
  
              TextReader reader = _outputReader;
              if (reader.Peek() &lt; 1) continue;
  
              char[] buffer = new char[BUFFER_SIZE];
  
              do
              {
                  int count = reader.Read(buffer, 0, buffer.Length);
                  if (count &gt; 0) builder.Append(buffer, 0, count);
              }
              while (reader.Peek() &gt; 0);
          }
          catch (Exception ex)
          {
              // handle exception in debug mode
              Console.WriteLine(ex.Message); // no exception generated!
              continue;
          }
  
          if (builder.Length == 0) continue;
          OnOutput(builder.ToString());
          builder.Length = 0;
      }
  
      if (!IsWaitable(_process)) return;
  
      try
      {
          if (_outputReader == null) return;
  
          TextReader reader = _outputReader;
          if (reader.Peek() &lt; 1) return;
  
          char[] buffer = new char[BUFFER_SIZE];
  
          do
          {
              int count = reader.Read(buffer, 0, buffer.Length);
              if (count &gt; 0) builder.Append(buffer, 0, count);
          }
          while (reader.Peek() &gt; 0);
      }
      catch (Exception ex)
      {
          // handle exception in debug mode
          Console.WriteLine(ex.Message); // no exception generated!
          return;
      }
  
      if (builder.Length &gt; 0) OnOutput(builder.ToString());
  }
  
  bool IsWaitable(Process thisValue)
  {
      return thisValue != null &amp;&amp; !thisValue.HasExited;
  }
  </code></pre>
  
  <p>I can't use Process.BeginOutputReadLine because it waits for a NewLine to be present in the stream which does not happen for the last line of output. See <a href="https://stackoverflow.com/questions/7612524/parse-output-from-a-process-that-updates-a-single-console-line">this</a> and <a href="https://www.codeproject.com/Articles/170017/Solving-Problems-of-Monitoring-Standard-Output-and" rel="nofollow noreferrer">this</a> for details</p>
  
  <p>I get 
  Microsoft Windows [Version xxxxx]</p>
  
  <p>(c) copyright line</p>
  
  <p>And the program does not display the command-line prompt unless more output comes from the process that contains a NewLine</p>
  
  <p><strong>The points of interest are:</strong></p>
  
  <p><strong>1. Why does this code not read all the output as it should to the end?</strong></p>
  
  <p>After trying many things, it seems as if the buffer does not contain any more text to read and the stream seems to be missing some data from the original output of the executable. What is weird is that happens randomly. Sometimes I get the second line and sometimes not. In any case, I never got the output that should result from feeding internal commands to the cmd process.</p>
  
  <p><strong>2. The StandardInput (StreamWriter) actually does flush the buffer (or it thinks it does) and sets its charpos to zero while the BaseStream still has its charpos field = [length of buffer]. Why is this happening?</strong> </p>
  
  <p>Any insights into what could be the problem or a workaround would be greatly appreciated.</p>