title: Take desktop screenshot with Electron
body: <p>I am using Electron to create a Windows application that creates a fullscreen transparent overlay window. The purpose of this overlay is to:</p>
  <ol>
  <li>take a screenshot of the entire screen (not the overlay itself which is transparent, but the screen 'underneath'),</li>
  <li>process this image by sending the image as a byte stream to my python server, and</li>
  <li>draw some things on the overlay</li>
  </ol>
  <p>I am getting stuck on the first step, which is the screenshot capturing process.</p>
  <p>I tried <strong>option 1</strong>, which is to use <code>capturePage()</code>:</p>
  <pre><code>this.electronService.remote.getCurrentWindow().webContents.capturePage()
      .then((img: Electron.NativeImage) =&gt; { ... }
  </code></pre>
  <p>but this captures my overlay window only (and not the desktop screen). This will be a blank image which is useless to me.</p>
  <p><strong>Option 2</strong> is to use <code>desktopCapturer</code>:</p>
  <pre><code>this.electronService.remote.desktopCapturer.getSources({types: ['screen']}).then(sources =&gt; {
      for (const source of sources) {
          if (source.name === 'Screen 1') {
              try {
                  const mediaDevices = navigator.mediaDevices as any;
                  mediaDevices.getUserMedia({
                      audio: false,
                      video: { // this specification is only available for Chrome -&gt; Electron runs on Chromium browser
                          mandatory: {
                              chromeMediaSource: 'desktop',
                              chromeMediaSourceId: source.id,
                              minWidth: 1280,
                              maxWidth: 1280,
                              minHeight: 720,
                              maxHeight: 720
                          }
                      }
                  }).then((stream: MediaStream) =&gt; { // stream.getVideoTracks()[0] contains the video track I need
                      this.handleStream(stream);
                  });
              } catch (e) {
              }
          }
      }
  });
  </code></pre>
  <p>The next step is where it becomes fuzzy for me.  What do I do with the acquired <code>MediaStream</code> to get a bytestream from the screenshot out of it? I see plenty of examples how to display this stream on a webpage, but I wish to send it to my backend. <a href="https://stackoverflow.com/questions/51543595/get-a-stream-of-bytes-from-navigator-mediadevices-getusermedia">This StackOverflow post</a> mentions how to do it, but I am not getting it to work properly.  This is how I implemented <code>handleStream()</code>:</p>
  <pre><code>import * as MediaStreamRecorder from 'msr';
  
  private handleStream(stream: MediaStream): void {
      recorder.stop()
      const recorder = new MediaStreamRecorder(stream);
      recorder.ondataavailable = (blob: Blob) =&gt; { // I immediately get a blob, while the linked SO page got an event and had to get a blob through event.data
          this.http.post&lt;Result&gt;('http://localhost:5050', blob);
      };
  
      // make data available event fire every one second
      recorder.start(1000);
  }
  </code></pre>
  <p>The <code>blob</code> is not being accepted by the Python server.  Upon inspecting the contents of <code>Blob</code>, it's a video as I suspected.  I verified this with the following code:</p>
  <pre><code>let url = URL.createObjectURL(blob);
  window.open(url, '_blank')
  </code></pre>
  <p>which opens the blob in a new window. It displays a video of maybe half a second, but I want to have a static image. So how do I get a specific snapshot out of it? I'm also not sure if simply sending the Javascript blob format in the POST body will do for Python to be correctly interpret it. In Java it works by simply sending a <code>byte[]</code> of the image so I verified that the Python server implementation works as expected.</p>
  <p>Any suggestions other than using the <code>desktopCapturer</code> are also fine.  This implementation is capturing my mouse as well, which I rather not have.  I must admit that I did not expect this feature to be so difficult to implement.</p>