Functional objects are large and small physical entities installed in urban environments to offer specific functionalities to visitors, such as shops, escalators, and information kiosks. Instances of the novel notion are focal points of human activities and are significant in pedestrian movement. Pedestrian trajectory modelling in an urban scene is a challenging problem because of the complex patterns resulting from social interactions of the crowds and the diverse relation between pedestrians and functional objects. Many data-driven methods have been proposed to explain the complex movements in urban scenes. However, the methods considering functional objects in their formulation are rare. This study aims to reduce the knowledge gap by demonstrating the importance of pedestrian–object relations in the modelling task. The proposed modelling method, called pedestrian–object relation guided trajectory prediction (PORTP), uses a dual-layer architecture that includes a predictor of pedestrian–object relation and a series of relation-specific specialized pedestrian trajectory prediction models. The experiment findings indicate that the inclusion of pedestrian–object relation results in more accurate predictions. This study provides an empirical foundation for the novel notion and a strong baseline for future work on this topic.