title: BNF: input going to wrong nonterminal
body: <p>I'm developing a BNF for chess algebraic notation and ran into an interesting case, input going to the wrong non-terminal.</p>
  
  <p>My start BNF rule is as follows (note that this intentionally doesn't include castling or notes):</p>
  
  <pre><code>algebraic_notation : piece start_position capture end_position promotion
  </code></pre>
  
  <p><code>piece</code>, <code>start_position</code>, <code>capture</code>, and <code>promotion</code> can be empty, thus allowing for a move like 'd4'. The problem is that when such a move is entered, the input ('d4') is taken by <code>start_position</code>, thus resulting in an error b/c there is no more input for <code>end_position</code>, which cannot be empty.</p>
  
  <p>The obvious hack/workaround is to allow <code>end_position</code> to be empty and then check to see if we got any input for it and act accordingly.</p>
  
  <p>This does work, but I would like to know if there is a way to deal with this. Is it possible for input not to go to the first matching symbol if it causes the entire expression not to match?</p>
  
  <p>Another question is whether this is standard behaviour for BNF, or a problem with the yaccer I'm using: PLY v 3.3.</p>
  
  <p>Tried using flex/bison and got same thing. So it appears its not specific to PLY.</p>
  
  <p>Here are all the relevant rules for completeness:</p>
  
  <pre><code>algebraic_notation : piece start_position capture end_position promotion
  
  piece : KING
          | QUEEN
          | BISHOP
          | KNIGHT
          | ROOK
          | pawn
  
  pawn : empty
  
  start_position : FILE
                  | NUMBER
                  | FILE NUMBER
                  | empty
  
  end_position : FILE NUMBER
                  | empty                 // this line is the hack/workaround
  
  capture : CAPTURE
          | empty
  
  promotion : EQUAL QUEEN
              | EQUAL ROOK
              | EQUAL KNIGHT
              | EQUAL BISHOP
              | empty
  
  empty : 
  </code></pre>