<p>I got a few recent works for the problem of <strong>"Image-Text Matching"</strong> which is slightly different from my problem statement but I can adjust the code for my project. </p>

<ol>
<li>Transformer Reasoning Network for Image-Text Matching and Retrieval <a href="https://arxiv.org/pdf/2004.09144v1.pdf" rel="nofollow noreferrer">enter link description here</a></li>
<li>Stacked Cross Attention for Image-Text Matching <a href="https://kuanghuei.github.io/SCANProject/" rel="nofollow noreferrer">enter link description here</a></li>
</ol>
Text Image Combined Classification Model
<p>I am working on a problem statement where I have to match (text, image) pair. Given a furniture description and furniture image, I have to say they are same or not. This is a binary classification problem but I have to combine both text and image data. 
One possible solution I am trying as follows
 <a href="https://i.stack.imgur.com/E1EIu.jpg" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/E1EIu.jpg" alt="enter image description here"></a>
In the above diagram, I am combining the feature from the pre-trained text and image model and training the linear layer end to end. </p>

<p>Is there any other way to handle this type of problem. any leads are most welcome. thanks a lot in advance for your help. </p>