Color carries gender information (e.g., red-female). This study explored whether background color could influence the gender categorization of human faces. Visual stimuli were generated from faces whose sexually dimorphic content was morphed monotonically from female to male perception. The face stimulus was presented upright (Experiment 1) and inverted (Experiment 2) with three background colors (i.e., red, green, and gray). Participants were instructed to categorize the gender of the face stimulus as male or female by pressing one of two labelled keys. Results showed that a red background could bias the gender of an ambiguous upright face toward a female compared with green and gray background colors (Experiment 1). However, this red effect was diminished when the face stimulus was inverted (Experiment 2). These results suggest that red background color interacting with facial configuration features biases gender perception toward a female face, possibly through top-down processing of learned associations between the color red and femininity.