BUG: Fix buffer overflows in tokenizer.c that caused python to segfault with certain
closes #9205 

whoosh, quite a number of changes for a 'simple' bug.....
 there was some pruning of routines that didn't seem to be used anymore and some changes to the way pointers were reallocated and freed while investigating the problem.  the heart of the fix is checking lengths against the buffer caps as items are added to the buffers.    i thought the extra checking might slow things down slightly, but vbench claims the tokenizer branch is as fast as upstream/master.  i'll take a look at the other csv issues you pointed me to earlier
 @selasley hahh.. wasn't being critical. I am very glad you refactored. Sometimes just needs to be done! thanks.
 will have to test this on windows.
 :)  I was a bit nervous about changing so many lines of other people's code.  Thanks for checking on windows.  I don't have access to a windows machine at home or work to do any testing.
I fixed the typo in the whatsnew document and pushed the changes
 ok, this looks good on windows.

can you do a vbench run on this? just to confirm nothing has changed? 

ping when done and we'll get this in.
 merged via https://github.com/pydata/pandas/commit/c6c9c0bfc10e24bdd0e4d32f7a8faf7fbc2294b2
 thanks!
