<p>You can use <code>U-Net</code> or <code>SegNet</code> for image segmentation. In fact you add residual layers to your CNN to get this result:</p>

<p><a href="https://i.stack.imgur.com/7kOeB.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/7kOeB.png" alt="Image Segmentation"></a></p>

<p>About <strong>U-Net</strong>:</p>

<p>Arxiv: <a href="https://arxiv.org/abs/1505.04597" rel="nofollow noreferrer">U-Net: Convolutional Networks for Biomedical Image Segmentation</a></p>

<p><a href="https://i.stack.imgur.com/EwXH5.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/EwXH5.png" alt="U-Net Architecture"></a></p>

<p><strong>Seg-Net</strong>:</p>

<p>Arxiv: <a href="https://arxiv.org/pdf/1511.00561.pdf" rel="nofollow noreferrer">SegNet: A Deep Convolutional
Encoder-Decoder Architecture for Image
Segmentation</a></p>

<p><a href="https://i.stack.imgur.com/ydVfU.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/ydVfU.png" alt="SegNet Architecture"></a></p>

<p><strong>Here are Simple Examples of Codes:</strong> <code>keras==1.1.0</code></p>

<p><strong>U-Net:</strong></p>

<pre><code>shape=60
batch_size = 30
nb_classes = 10
img_rows, img_cols = shape, shape
nb_filters = 32
pool_size = (2, 2)
kernel_size = (3, 3)
input_shape=(shape,shape,1)

reg=0.001
learning_rate = 0.013
decay_rate = 5e-5
momentum = 0.9

sgd = SGD(lr=learning_rate,momentum=momentum, decay=decay_rate, nesterov=True)
shape2

recog0 = Sequential()
recog0.add(Convolution2D(20, 3,3,
                        border_mode='valid',
                        input_shape=input_shape))
recog0.add(BatchNormalization(mode=2))

recog=recog0
recog.add(Activation('relu'))
recog.add(MaxPooling2D(pool_size=(2,2)))
recog.add(UpSampling2D(size=(2, 2)))
recog.add(Convolution2D(20, 3, 3,init='glorot_uniform'))
recog.add(BatchNormalization(mode=2))
recog.add(Activation('relu'))

for i in range(0,2):
    print(i,recog0.layers[i].name)

recog_res=recog0
part=1
recog0.layers[part].name
get_0_layer_output = K.function([recog0.layers[0].input, K.learning_phase()],[recog0.layers[part].output])

get_0_layer_output([x_train, 0])[0][0]

pred=[np.argmax(get_0_layer_output([x_train, 0])[0][i]) for i in range(0,len(x_train))]

loss=x_train-pred
loss=loss.astype('float32')

recog_res.add(Lambda(lambda x: x,input_shape=(56,56,20),output_shape=(56,56,20)))
recog2=Sequential()
recog2.add(Merge([recog,recog_res],mode='ave'))
recog2.add(Activation('relu'))
recog2.add(Convolution2D(20, 3, 3,init='glorot_uniform'))
recog2.add(BatchNormalization(mode=2))
recog2.add(Activation('relu'))
recog2.add(Convolution2D(1, 1, 1,init='glorot_uniform'))
recog2.add(Reshape((shape2,shape2,1)))
recog2.add(Activation('relu'))

recog2.compile(loss='mean_squared_error', optimizer=sgd,metrics = ['mae'])
recog2.summary()

x_train3=x_train2.reshape((1,shape2,shape2,1))

recog2.fit(x_train,x_train3,
                nb_epoch=25,
                batch_size=30,verbose=1)
</code></pre>

<p><strong>SegNet:</strong></p>

<pre><code>shape=60
batch_size = 30
nb_classes = 10
img_rows, img_cols = shape, shape
nb_filters = 32
pool_size = (2, 2)
kernel_size = (3, 3)
input_shape=(shape,shape,1)

reg=0.001
learning_rate = 0.012
decay_rate = 5e-5
momentum = 0.9

sgd = SGD(lr=learning_rate,momentum=momentum, decay=decay_rate, nesterov=True)

recog0 = Sequential()
recog0.add(Convolution2D(20, 4,4,
                        border_mode='valid',
                        input_shape=input_shape))
recog0.add(BatchNormalization(mode=2))
recog0.add(MaxPooling2D(pool_size=(2,2)))

recog=recog0
recog.add(Activation('relu'))
recog.add(MaxPooling2D(pool_size=(2,2)))
recog.add(UpSampling2D(size=(2, 2)))
recog.add(Convolution2D(20, 1, 1,init='glorot_uniform'))
recog.add(BatchNormalization(mode=2))
recog.add(Activation('relu'))

for i in range(0,8):
    print(i,recog0.layers[i].name)

recog_res=recog0
part=8
recog0.layers[part].name
get_0_layer_output = K.function([recog0.layers[0].input, K.learning_phase()],[recog0.layers[part].output])
get_0_layer_output([x_train, 0])[0][0]
pred=[np.argmax(get_0_layer_output([x_train, 0])[0][i]) for i in range(0,len(x_train))]

loss=x_train-pred
loss=loss.astype('float32')

recog_res.add(Lambda(lambda x: x-np.mean(loss),input_shape=(28,28,20),output_shape=(28,28,20)))

recog2=Sequential()
recog2.add(Merge([recog,recog_res],mode='sum'))
recog2.add(UpSampling2D(size=(2, 2)))
recog2.add(Convolution2D(1, 3, 3,init='glorot_uniform'))
recog2.add(BatchNormalization(mode=2))
recog2.add(Reshape((shape2*shape2,)))
recog2.add(Reshape((shape2,shape2,1)))
recog2.add(Activation('relu'))
recog2.compile(loss='mean_squared_error', optimizer=sgd,metrics = ['mae'])
recog2.summary()

x_train3=x_train2.reshape((1,shape2,shape2,1))

recog2.fit(x_train,x_train3,
                nb_epoch=400,
                batch_size=30,verbose=1)
</code></pre>

<p>Then add a threshold for the colors of segmentation.</p>
Image semantic segmentation of repeating patterns without CNNs
<p>Suppose I have one or multiple tiles consisting of a single pattern (e.g. materials like: wood, concrete, gravel...) that I would like to train my classifier on, and then I'll use the trained classifier to determine to which class each pixel in another image belong.</p>

<p>Below are example of two tiles I would like to train the classifier on:</p>

<p><a href="https://i.stack.imgur.com/8l97k.jpg" rel="noreferrer"><img src="https://i.stack.imgur.com/8l97k.jpg" alt="concrete"></a>
<a href="https://i.stack.imgur.com/fS0sh.jpg" rel="noreferrer"><img src="https://i.stack.imgur.com/fS0sh.jpg" alt="wood"></a></p>

<p>And let's say I want to segment the image below to identify the pixels belonging to the door and those belonging to the wall. It's just an example, I know this image isn't made of exactly the same patterns as the tiles above:</p>

<p><a href="https://i.stack.imgur.com/eas1R.jpg" rel="noreferrer"><img src="https://i.stack.imgur.com/eas1R.jpg" alt="door"></a></p>

<p>For this specific problem, is it necessary to use convolutional neural networks? Or is there a way to achieve my goal with a shallow neural network or any other classifier, combined with texture features for example?</p>

<p>I've already implemented a classifier with Scikit-learn which works on tile pixels individually (see code below where <code>training_data</code> is a vector of singletons), but I want instead to train the classifier on texture patterns.</p>

<pre class="lang-py prettyprint-override"><code># train classifier
classifier = SGDClassifier()
classifier.fit(training_data, training_target)

# classify given image
test_data = image_gray.flatten().reshape((-1, 1))
predictions = classifier.predict(test_data)
image_classified = predictions.reshape(image_gray.shape)
</code></pre>

<p>I was reading <a href="https://medium.com/@arthur_ouaknine/review-of-deep-learning-algorithms-for-image-semantic-segmentation-509a600f7b57" rel="noreferrer">this review</a> of recent deep learning methods used for image segmentation and the results seem accurate, but since I've never used any CNN before I feel intimidated by it.</p>