<p>It's hard to tell without looking into the dataset and experimenting. But hopefully, the following research materials will guide you in the right direction.</p>

<ul>
<li><p>Machine learning-based approach:
<a href="https://www.researchgate.net/publication/266672947_Estimating_smile_intensity_A_better_way" rel="nofollow noreferrer">https://www.researchgate.net/publication/266672947_Estimating_smile_intensity_A_better_way</a></p></li>
<li><p>Deep learning (CNN): <a href="https://arxiv.org/pdf/1602.00172.pdf" rel="nofollow noreferrer">https://arxiv.org/pdf/1602.00172.pdf</a></p></li>
<li><p>A list of awesome papers for smile and smile intensity detection:
<a href="https://github.com/EvelynFan/AWESOME-FER/blob/master/README.md" rel="nofollow noreferrer">https://github.com/EvelynFan/AWESOME-FER/blob/master/README.md</a></p></li>
<li><p>SmileNet project: <a href="https://sites.google.com/view/sensingfeeling/" rel="nofollow noreferrer">https://sites.google.com/view/sensingfeeling/</a></p></li>
</ul>

<p>Now, I'm assuming you don't have any label for actual smile intensity.</p>

<p>In such a scenario, the existing smile detection methods can be used directly, you'll use the last activation output (sigmoid) as a confidence score for smiling. If the confidence is higher, the intensity should be higher.</p>

<p>Now, you can use the facial landmark points as separate features (pass them through an LSTM block) and concatenate to the CNN at an early stage/ or later to improve the performance of your model.</p>

<p>If you have the label for smiling intensity, you can just solve it as a regression problem, the CNN will have one output, will try to regress the smile intensity (the normalized smile intensity with sigmoid in this case).</p>
CNN vs SVM for smile intensity detection training?
<p>I have a dataset made up of images of faces, with the corresponding landmarks that make up the mouth.
These landmarks are sets of 2D points (x,y pixel position).
Each image-landmark set data pair is tagged as either a smile, or neutral.</p>

<p>What i would like to do is train a deep learning model to return a smile intensity for a new image-landmark data pair.</p>

<p>What should I be searching for to help me with the next step?
Is it a CNN that i need? In my limited understanding, the usual training input is just an image, where I would be passing the landmark sets to train with. Or would an SVM approach be more accurate?</p>

<p>I am looking for maximum accuracy, as much as is possible.</p>

<p>What is the approach that I need called?</p>

<p>I am happy to use PyTorch, Dlib or any framework, I am just a little stuck on the search terms to help me move forward.</p>

<p>Thank you.</p>