<p>Yes, you could do it based on <a href="https://arxiv.org/pdf/1703.06870.pdf" rel="nofollow noreferrer">this paper</a></p>

<p>There is an implementation <a href="https://github.com/CharlesShang/FastMaskRCNN" rel="nofollow noreferrer">here</a> but note that this is not related to the object detection API.</p>
TensorFlow object detection other shape of classification
<p>in TensorFlow object detection API, is there any possibility to get other shape of recognized object than rectangle? </p>

<p>For example when I'm teaching my computer to recognize a forest, mostly it should be more useful to get some kind of polygon instead of rectangle. </p>

<p>Other thing is for example river, but polygon seems much easier than line. </p>