CI: clean Azure Windows
try if 4 workers work now and try cache

This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this. Closing for now - feel free to reopen @fangchenli 