CLN: remove unnecessary Series._convert calls
cc @WillAyd my confidence on the correctness here is only "pretty sure", so could use an extra set of eyes.
@jorisvandenbossche  Brock, can you respond to my comments? can you merge master, this looks ok. I guess @jorisvandenbossche has some comments this is fine