CLN: remove unnecessary get_value_at calls
I found in another branch that these calls didn't play nicely with `nogil`, so went through and removed the ones that are unnecessary.  Turns out we really only need the one in libindex.get_value_at, since that is the only place where we _arent_ assured that `loc` is intlike.
thanks any perf implications? > thanks any perf implications?

Not yet, but I think this will allow us to `nogil` some functions that we couldn't before, so possibly there