title: How to enable multi-line matching in custom tokens provider
body: <p>I have a custom Monarch tokenizer for function names in Monaco that works well when the open paren appears on the same line:</p>
  
  <p><img src="https://i.stack.imgur.com/shmAY.png" width="400"></p>
  
  <p>But the tokenizer fails to correctly identify the function name if the open paren appears on a separate line:</p>
  
  <p><img src="https://i.stack.imgur.com/GIOtO.png" width="400"></p>
  
  <p>Note that the <code>foo</code> function on line 1 is correctly highlighted, but the tokenizer gets lost when it comes to the <code>foo</code> function on line 5, so it highlights it as "invalid" (the default). (<em>For color-blind readers, sorry about the green/red coloring... it's the theme default.</em>)</p>
  
  <p>The tokenizer uses regexes, but it seems like nothing I can do will force the regex to span more than the current line. Is there some way to instruct Monaco's tokenizer to use multi-line matching, or is this just a known limitation? I can't find any mention of it in the <a href="https://microsoft.github.io/monaco-editor/api/interfaces/monaco.languages.imonarchlanguage.html" rel="nofollow noreferrer">documentation</a>, one way or the other.</p>
  
  <p>Here's a simple repro you can copy/paste into the <a href="https://microsoft.github.io/monaco-editor/playground.html" rel="nofollow noreferrer">Monaco Editor Playground</a>:</p>
  
  <pre class="lang-js prettyprint-override"><code>// Register a new language
  monaco.languages.register({ id: 'mySpecialLanguage' });
  
  // Register a tokens provider for the language
  monaco.languages.setMonarchTokensProvider('mySpecialLanguage', {
      defaultToken: "invalid",
      tokenizer: {
          root: [
              [
                  /(foo)([\s\n\r]*)([(])/,
                  ["type.function", "", "@brackets"],
              ],
  
              [
                  /"/,
                  {
                      token: "string.quote",
                      bracket: "@open",
                      next: "@string",
                  },
              ],
  
              [
                  /[()\[\]]/,
                  "@brackets",
              ],
          ],
  
          string: [
              [
                  /[^"]+/,
                  "string",
              ],
              [
                  /"/,
                  {
                      token: "string.quote",
                      bracket: "@close",
                      next: "@pop",
                  },
              ],
          ],
      }
  });
  
  monaco.editor.create(document.getElementById("container"), {
      theme: 'vs',
      value: 'foo\t(\n\t\t"bar"\n\t)\n\nfoo\n\t(\n\t\t"bar"\n\t)\n',
      language: 'mySpecialLanguage'
  });
  </code></pre>