<p>For your first question, you are talking about image style transfer. In that case, <a href="https://www.cv-foundation.org/openaccess/content_cvpr_2016/papers/Gatys_Image_Style_Transfer_CVPR_2016_paper.pdf" rel="nofollow noreferrer">CNNs</a> may help you.</p>

<p>For the second, if I understand correctly, by growing you mean introducing variations in the image patch while keeping it realistic. If that's the goal, you may use <a href="https://papers.nips.cc/paper/5423-generative-adversarial-nets.pdf" rel="nofollow noreferrer">GANs</a> for generating images, provided you have a reasonable sized dataset to train with:</p>

<p><a href="https://arxiv.org/pdf/1803.04469.pdf" rel="nofollow noreferrer">Image Synthesis with GANs</a></p>

<p>Intuitively, conditional GANs model the joint distribution of the input dataset (which in your case, are images you want to imitate) and can draw new samples (images) from the learned distribution, thereby allowing you to create more images having similar content.</p>

<p><a href="https://github.com/phillipi/pix2pix" rel="nofollow noreferrer">Pix2Pix</a> is the open-source code of a <a href="https://arxiv.org/abs/1611.07004" rel="nofollow noreferrer">well-known paper</a> that you can play around to generate images. Specifically, let X be your input image and Y be a target image. You can train the network and feed X to observe the output O of the generator. Thereafter, by tweaking the architecture a bit or by changing the skip connections (read the paper) train again and you can generate variety in the output images O.</p>

<p><a href="https://bair.berkeley.edu/blog/2018/03/13/mcgan/" rel="nofollow noreferrer">Font Style Transfer</a> is an interesting experiment with text on images (rather than image on image, as in your case).</p>
How do we transfer a particular structure(defined by mask) in an image to another image?
<p><a href="https://i.stack.imgur.com/NwVZx.png" rel="nofollow noreferrer">Image showing required content transfer between source and target images</a></p>

<p>Essentially is there a way to grow an image patch defined by mask and keep it realistic?</p>