Semi-insulating CdTe and CdZnTe crystals fabricated into pixelated sensors and integrated into radiation detection modules have demonstrated a remarkable ability to operate under rapidly changing X-ray irradiation environments. Such challenging conditions are required by all photon-counting-based applications, including medical computed tomography (CT), airport scanners, and non-destructive testing (NDT). Although, maximum flux rates and operating conditions differ in each case. In this paper, we investigated the possibility of using the detector under high-flux X-ray irradiation with a low electric field satisfactory for maintaining good counting operation. We numerically simulated electric field profiles visualized via Pockels effect measurement in a detector affected by high-flux polarization. Solving coupled drift–diffusion and Poisson’s equations, we defined the defect model, consistently depicting polarization. Subsequently, we simulated the charge transport and evaluated the collected charge, including the construction of an X-ray spectrum on a commercial 2-mm-thick pixelated CdZnTe detector with 330 µm pixel pitch used in spectral CT applications. We analyzed the effect of allied electronics on the quality of the spectrum and suggested setup optimization to improve the shape of the spectrum.