title: Beginner in clojure: Tokenizing lists of different characters
body: <p>So I know this isn't the best method of solving this issue, but I'm trying to go through a list of lines from an input file, which end up being expressions. I've got a list of expressions, and each expression has it's own list thanks to the split-the-list function. My next step is to replace characters with <code>id</code>, ints with <code>int</code>, and + or - with <code>addop</code>. I've got the regexes to find whether or not my symbols match any of those, but when I try and replace them, I can only get the last for loop I call to leave any lasting results. I know what it stems down to is the way functional programming works, but I can't wrap my head around the trace of this program, and how to replace each separate type of input and keep the results all in one list.</p>
  
  <pre><code>(def reint #"\d++")
  (def reid #"[a-zA-Z]+")
  (def readdop #"\+|\-")
  
  (def lines (into () (into () (clojure.string/split-lines (slurp "input.txt")) )))
  
  (defn split-the-line [line] (clojure.string/split line #" " ))
  
  (defn split-the-list  [] (for [x (into [] lines)] (split-the-line x)))
  
  (defn tokenize-the-line [line] 
  (for [x line] (clojure.string/replace x reid "id")) 
  (for [x line] (clojure.string/replace x reint "int")) 
  (for [x line] (clojure.string/replace x readdop "addop")))
  
  (defn tokenize-the-list [] (for [x (into [] (split-the-list) )] (tokenize-the-line x)))
  </code></pre>
  
  <p>And as you can probably tell, I'm pretty new to functional programming, so any advice is welcome!</p>