Side chains play an important role in the photo-oxidation process of low band gap (LBG) polymers. For example, it has been shown that their photostability can be increased by the introduction of aromatic-oxy-alkyl links. We studied the photostability of prototypical LBG polymers with alkyl and oxyalkyl side chains during irradiation with white light (AM 1.5 conditions) in dry air using UV/vis and IR spectroscopy. Though its degradation kinetics were distinctly affected by the presence or absence of oxygen in the structure of the side chains, in particular cases, the stability was more affected by the presence of linear or branched side chains. Moreover, we showed that the exact position of the alkyl/oxyalkyl side chain at the polymer backbone could be crucial. Although minor effects of chemical modifications on the electronic parameters (ionization potential and gap) were observed, the molecular orientation, determined by polarization modulation-infrared reflection-absorption spectroscopy (PMIRRAS), could be affected. The aggregation and crystallinity of these polymers may distinctly affect their stability.