The hitting position and velocity control for table tennis robots have been investigated widely in the literature. However, most of the studies conducted do not consider the opponent’s hitting behaviors, which may reduce hitting accuracy. This paper proposes a new table tennis robot framework that returns the ball based on the opponent’s hitting behaviors. Specifically, we classify the opponent’s hitting behaviors into four categories: forehand attacking, forehand rubbing, backhand attacking, and backhand rubbing. A tailor-made mechanical structure that consists of a robot arm and a two-dimensional slide rail is developed such that the robot can reach large workspaces. Additionally, a visual module is incorporated to enable the robot to capture opponent motion sequences. Based on the opponent’s hitting behaviors and the predicted ball trajectory, smooth and stable control of the robot’s hitting motion can be obtained by applying quintic polynomial trajectory planning. Moreover, a motion control strategy is devised for the robot to return the ball to the desired location. Extensive experimental results are presented to demonstrate the effectiveness of the proposed strategy.