Measurement-device-independent quantum key distribution (MDI-QKD) enables two legitimate users to generate shared information-theoretic secure keys with immunity to all detector side attacks. However, the original proposal using polarization encoding is sensitive to polarization rotations stemming from birefringence in fibers or misalignment. To overcome this problem, here we propose a robust QKD protocol without detector vulnerabilities based on decoherence-free subspaces using polarization-entangled photon pairs. A logical Bell state analyzer is designed specifically for such encoding. The protocol exploits common parametric down-conversion sources, for which we develop a MDI-decoy-state method, and requires neither complex measurements nor a shared reference frame. We have analyzed the practical security in detail and presented a numerical simulation under various parameter regimes, showing the feasibility of the logical Bell state analyzer along with the potential that double communication distance can be achieved without a shared reference frame.