title: Writing a Z80 assembler - lexing ASM and building a parse tree using composition?
body: <p>I'm very new to the concept of writing an assembler and even after reading a great deal of material, I'm still having difficulties wrapping my head around a couple of concepts.</p>
  
  <ol>
  <li><p>What is the process to actually break up a source file into tokens? I believe this process is called lexing, and I've searched high and low for a real code examples that make sense, but I can't find a thing so simple code examples very welcome ;)</p></li>
  <li><p>When parsing, does information ever need to be passed up or down the tree? The reason I ask is as follows, take:</p>
  
  <p>LD BC, nn</p></li>
  </ol>
  
  <p>It needs to be turned into the following parse tree once tokenized(???)</p>
  
  <pre><code>  ___ LD ___
    |        |
   BC        nn
  </code></pre>
  
  <p>Now, when this tree is traversed it needs to produce the following machine code:</p>
  
  <pre><code>01 n n
  </code></pre>
  
  <p>If the instruction had been:</p>
  
  <pre><code>LD DE,nn
  </code></pre>
  
  <p>Then the output would need to be:</p>
  
  <pre><code>11 n n
  </code></pre>
  
  <p>Meaning that it raises the question, does the LD node return something different based on the operand or is it the operand that returns something? And how is this achieved? More simple code examples would be excellent if time permits.</p>
  
  <p>I'm most interested in learning some of the raw processes here rather than looking at advanced existing tools so please bear that in mind before sending me to <a href="https://en.wikipedia.org/wiki/Yacc" rel="noreferrer">Yacc</a> or <a href="https://en.wikipedia.org/wiki/Flex_(lexical_analyser_generator)" rel="noreferrer">Flex</a>.</p>