Immunosenescence and inflammaging facilitate the insurgence of chronic diseases. The Mediterranean diet is a non-invasive intervention to improve the chronic low-grade inflammatory status associated with aging. Olive oil oleuropein (OLE) and hydroxytyrosol (HT) demonstrated a controversial modulatory action on inflammation in vitro when tested at concentrations exceeding those detectable in human plasma. We studied the potential anti-inflammatory effects of OLE and HT at nutritionally relevant concentrations on peripheral blood mononuclear cells (PBMCs) as regards cell viability, frequency of leukocyte subsets, and cytokine release, performing an age-focused analysis on two groups of subjects: Adult (age 18–64 years) and Senior (age ≥ 65 years). OLE and HT were used alone or as a pre-treatment before challenging PBMCs with lipopolysaccharide (LPS). Both polyphenols had no effect on cell viability irrespective of LPS, but 5 µM HT had an LPS-like effect on monocytes, reducing the intermediate subset in Adult subjects. OLE and HT had no effect on LPS-triggered release of TNF-α, IL-6 and IL-8, but 5 µM HT reduced IL-10 secretion by PBMCs from Adult vs. Senior group. In summary, nutritionally relevant concentrations of OLE and HT elicit no anti-inflammatory effect and influence the frequency of immune cell subsets with age-related different outcomes.