title: How to split Python source code into tokens?
body: <p>I have source code like:</p>
  
  <pre><code>model_stateless.fit(x_train,
                      y_train,
                      batch_size=batch_size,
                      epochs=epochs,
                      verbose=1,
                      validation_data=(x_test, y_test),
                      shuffle=False)
  
  print('Predicting')
  predicted_stateless = model_stateless.predict(x_test, batch_size=batch_size)
  </code></pre>
  
  <p>or</p>
  
  <pre><code>def create_model(stateful):
      model = Sequential()
      model.add(LSTM(20,
                     input_shape=(lahead, 1),
                     batch_size=batch_size,
                     stateful=stateful))
      model.add(Dense(1))
      model.compile(loss='mse', optimizer='adam')
      return model
  </code></pre>
  
  <p>And I want to tokenize this. If I do a straight <code>filecontents.split(' ')</code>, then it groups commas, parentheses, etc.</p>
  
  <p>Ideally, I want to tokenize this to have:
  <code>model_stateless.fit</code>, <code>(</code>, <code>x_train</code>, <code>,</code> <code>y_train</code>, <code>,</code>, <code>batch_size=batch_size</code>, <code>,</code>, etc</p>