<p>This would come under &quot;Action Recognition&quot;. I think it should be able to handle your requirement. You need not find the key frames.</p>
<p>Torch-vision has some pre-trained models which you can directly use in pytorch or you can fine-tune them with very less data. Look for &quot;Video classification&quot; models in this <a href="https://pytorch.org/docs/stable/torchvision/models.html" rel="nofollow noreferrer">Link</a>.</p>
<p>I would suggest to go for R(2+1)D (<a href="https://arxiv.org/abs/1711.11248" rel="nofollow noreferrer">paper Link</a>)</p>
<p>It is able to identify sports actions, gesture actions and sign-language.</p>
Finding a frame with the most 'weight' in a video classification problem
<p>I am starting a project with the task to recognize micro expressions in a human face. However, the first task that I formulated is to get the key frames in a 10 second video that has the most relation with the predicted expression. For eg, raising your eyebrows may represent surprise, but the raising activity may occur in only in say 10 frames and that 10 frames represents the micro expression for surprise. Any guides or research papers you can direct me to would be much helpful. I was planning to use some form of 3D-CNN, but I also welcome more efficient ways to do this as 3D CNN's are quite computationally expensive.</p>