This is all just to get around matplotlib/matplotlib#4165. Fortunately this is an obscure case. It sucks but I don't see any other way right now (and fortunately ISTM importing matplotlib doesn't really add much overhead). 