A number of recent methods to understand neural networks have focused on quantifying the role of individual features.   One such method, NetDissect identifies interpretable features of a model using the Broden dataset of visual semantic labels (colors, materials, textures, objects and scenes).   Given the recent rise of a number of action recognition datasets, we propose extending the Broden dataset to include actions to better analyze learned action models.   We describe the annotation process, results from interpreting action recognition models on the extended Broden dataset and examine interpretable feature paths to help us understand the conceptual hierarchy used to classify an action. The success of Deep convolutional neural networks (DNNs) is partly due to their ability to learn hidden representations that capture the important factors of variation in the data. Previous works have visualized the units of deep convolutional networks by sampling image patches that maximize the activation of each feature BID6 or by generating images that maximize each feature activation. Such visualizations show that individual features act as visual concept detectors. Features at lower layers detect concrete patterns such as textures or shapes while features at higher layers detect more semantically meaningful concepts such as dog heads or bicycle wheels. One tool for network interpretability (NetDissect) BID0 BID8 uses the Broden dataset (consists of objects, scenes, object parts, textures and materials) to evaluate individual units.Recently, DNNs have shown significant progress in action recognition with the introduction of large-scale video datasets. However, while NetDissect with the Broden dataset is appropriate for networks trained on object or scene recognition, it does not include the ability to detect learned action concepts.In this paper, we propose extending the Broden dataset to include actions so that we can more appropriately interpret action recognition networks. We describe our annotation process to collect images across action classes and select Sample videos Example frames from a few videos to show intraclass action variation Action Regions Spatial localization of actions in single frames for network interpretation Action Interpretation Identifying interpretable action features regions of importance for identifying each action. We then show results using our Action Region dataset together with the existing Broden set to identify interpretable action features in deep networks trained for action recognition. The Action Region dataset presented, and the code for integrating with NetDissect, will be made available online.
We expand Network Dissection to include action interpretation and examine interpretable feature paths to understand the conceptual hierarchy used to classify an action.