title: Using PLY to parse SQL statements
body: <p>I know there are other tools out there to parse SQL statements, but I am rolling out my own for educational purposes.  I am getting stuck with my grammar right now..  If you can spot an error real quick please let me know.</p>
  
  <pre><code>SELECT = r'SELECT'
  FROM = r'FROM'
  COLUMN = TABLE = r'[a-zA-Z]+'
  COMMA = r','
  STAR = r'\*'
  END = r';'
  t_ignore = ' ' #ignores spaces
  
  statement : SELECT columns FROM TABLE END
  
  columns : STAR
          | rec_columns
  
  rec_columns : COLUMN
              | rec_columns COMMA COLUMN
  </code></pre>
  
  <p>When I try to parse a statement like 'SELECT a FROM b;' I get an syntax error at the FROM token... Any help is greatly appreciated!</p>
  
  <p>(Edit) Code:</p>
  
  <pre><code>#!/usr/bin/python
  import ply.lex as lex
  import ply.yacc as yacc
  
  tokens = (
      'SELECT',
      'FROM',
      'WHERE',
      'TABLE',
      'COLUMN',
      'STAR',
      'COMMA',
      'END',
  )
  
  t_SELECT    = r'select|SELECT'
  t_FROM      = r'from|FROM'
  t_WHERE     = r'where|WHERE'
  t_TABLE     = r'[a-zA-Z]+'
  t_COLUMN    = r'[a-zA-Z]+'
  t_STAR      = r'\*'
  t_COMMA     = r','
  t_END       = r';'
  
  t_ignore    = ' \t'
  
  def t_error(t):
      print 'Illegal character "%s"' % t.value[0]
      t.lexer.skip(1)
  
  lex.lex()
  
  NONE, SELECT, INSERT, DELETE, UPDATE = range(5)
  states = ['NONE', 'SELECT', 'INSERT', 'DELETE', 'UPDATE']
  current_state = NONE
  
  def p_statement_expr(t):
      'statement : expression'
      print states[current_state], t[1]
  
  def p_expr_select(t):
      'expression : SELECT columns FROM TABLE END'
      global current_state
      current_state = SELECT
      print t[3]
  
  
  def p_recursive_columns(t):
      '''recursive_columns : recursive_columns COMMA COLUMN'''
      t[0] = ', '.join([t[1], t[3]])
  
  def p_recursive_columns_base(t):
      '''recursive_columns : COLUMN'''
      t[0] = t[1]
  
  def p_columns(t):
      '''columns : STAR
                 | recursive_columns''' 
      t[0] = t[1]
  
  def p_error(t):
      print 'Syntax error at "%s"' % t.value if t else 'NULL'
      global current_state
      current_state = NONE
  
  yacc.yacc()
  
  
  while True:
      try:
          input = raw_input('sql&gt; ')
      except EOFError:
          break
      yacc.parse(input)
  </code></pre>