Hydrolyzes dipeptides containing N-terminal aspartate residues. May play a role in allowing the cell to use peptide aspartate to spare carbon otherwise required for the synthesis of the aspartate family of amino acids. Dipeptidase E catalyzes the hydrolysis of dipeptides Asp-|-Xaa. It does not act on peptides with N-terminal Glu, Asn or Gln, nor does it cleave isoaspartyl peptides. Belongs to the peptidase S51 family.