Sense of Agency (SoA) is the feeling of control over one’s actions and their outcomes. A well-established implicit measure of SoA is the temporal interval estimation paradigm, in which participants estimate the time interval between a voluntary action and its sensory consequence. In the present study, we aimed to investigate whether the valence of action outcome modulated implicit SoA. The valence was manipulated through interaction partner’s (i) positive/negative facial expression, or (ii) type of gaze (gaze contact or averted gaze). The interaction partner was the humanoid robot iCub. In Experiment 1, participants estimated the time interval between the onset of their action (head movement towards the robot), and the robot’s facial expression (happy vs. sad face). Experiment 2 was identical, but the outcome of participants’ action was the type of robot’s gaze (gaze contact vs. averted). In Experiment 3, we assessed—in a within-subject design—the combined effect of robot’s type of facial expression and type of gaze. Results showed that, while the robot’s facial expression did not affect participants’ SoA (Experiment 1), the type of gaze affected SoA in both Experiment 2 and Experiment 3. Overall, our findings showed that the robot’s gaze is a more potent factor than facial expression in modulating participants’ implicit SoA.