title: Haskell/Trifecta: Parsing completely optional semicolons without polluting AST
body: <h2>I have rewritten the question since it was originally posted with a more concise code example:</h2>
  
  <p>Consider a language with fully optional semicolons almost entirely as sugar, i.e.:</p>
  
  <ul>
  <li><code>;; foo; bar;;;;</code> is valid</li>
  <li><code>foo bar foobar</code> is valid</li>
  <li><code>if (+1); foo</code> is different to <code>if (+1) foo</code> in semantics, so <code>;</code> cannot be considered whitespace</li>
  </ul>
  
  <p>Here is an example parser:</p>
  
  <pre><code>{-# LANGUAGE OverloadedStrings #-}
  
  import Text.Trifecta
  import Text.Trifecta.Delta
  import Text.PrettyPrint.ANSI.Leijen (putDoc, (&lt;&gt;), linebreak)
  import Control.Monad.Trans.State.Strict
  import Control.Applicative
  
  type TestParser a = StateT Int Parser a
  
  data AST a = Foo a | Bar a deriving (Show)
  
  pFoo :: TestParser (AST (Delta, Int))
  pFoo = curry Foo &lt;$ string "foo" &lt;*&gt; position &lt;* modify (+1) &lt;*&gt; get
  
  pBar :: TestParser (AST (Delta, Int))
  pBar = curry Bar &lt;$ string "bar" &lt;*&gt; position &lt;*&gt; get
  
  pStmt :: TestParser (AST (Delta, Int))
  pStmt = semi *&gt; pStmt &lt;|&gt; pFoo &lt;|&gt; pBar &lt;?&gt; "statement"
  
  pTest :: TestParser [AST (Delta, Int)]
  pTest = some pStmt
  
  main :: IO ()
  main
   = do   let res = parseByteString (evalStateT pTest 0)
                      (Directed "(test)" 0 0 0 0) ";;foo;bar;\nfoo;; foobarbar;;"
          case res of
              Success ast
               -&gt; print ast
              Failure errdoc
               -&gt; putDoc (errdoc &lt;&gt; linebreak)
  </code></pre>
  
  <p>The problem I am having with such a parser is that I need to be able to skip over semicolons without committing to parse a <code>pStmt</code>. At the moment the following error occurs:</p>
  
  <pre><code>(test):2:18: error: unexpected
      EOF, expected: statement
  foo;; foobarbar;;&lt;EOF&gt;
  </code></pre>
  
  <p>This is because it expects a statement (in <code>semi *&gt; pStmt</code>), however because stacked semicolons can sugar both the beginning and end of expressions I can't be sure I really want to expect/parse one before I already expect one.</p>
  
  <p>One hack I developed was to have <code>Nop</code> as a constructor in my AST, but I really don't want to do that -- it feels like a hack and with the number of semicolons in some documents it would greatly increase memory usage.</p>
  
  <p>I am looking for solutions/suggestions.</p>
  
  <hr>
  
  <p>Attempt at EBNF form of the desired grammar:</p>
  
  <pre><code>expr = "foo" | "bar"
  expr with sugar = expr | ";"
  program = { [white space], expr with sugar, [white space] }
  </code></pre>