<p>U-Net is used for semantic segmentation whereas VGG-16 is used for classification.</p>

<p><a href="https://i.stack.imgur.com/8lc3x.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/8lc3x.png" alt="enter image description here"></a></p>

<p>usually, with U-Net we predict a single-channel mask. The input can have 3 channels but for generating binary masks we almost always use signle channel output.</p>

<p>VGG on the other hand just gives a softmax probability for the input image which we use to decide which class an image belongs to.</p>

<p><a href="https://i.stack.imgur.com/UVrBO.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/UVrBO.png" alt="enter image description here"></a></p>

<p>Finally, in keras/tensorflow, <code>kernel_size  = 3</code> and <code>kernel_size = (3,3)</code> is equivalent. For Conv2D, you always need 2-d kernels, when we pass an integer like <code>3</code>, keras uses same dimensional kernels meaning the width and height of the kernel is 3.</p>

<p><a href="https://i.stack.imgur.com/LFh5f.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/LFh5f.png" alt="enter image description here"></a></p>

<p>So, <code>kernel_size</code> has no relations with channels, kernels are related with the spatial dimension (width and height of images/feature maps), number of filters is related with channels.</p>

<p>Relevant papers and code:</p>

<p><a href="https://arxiv.org/pdf/1411.4038.pdf" rel="nofollow noreferrer">https://arxiv.org/pdf/1411.4038.pdf</a> (vgg16 segmentation)</p>

<p><a href="https://courses.cs.washington.edu/courses/cse576/17sp/notes/Sachin_Talk.pdf" rel="nofollow noreferrer">https://courses.cs.washington.edu/courses/cse576/17sp/notes/Sachin_Talk.pdf</a> (encode-decoder architecture)</p>

<p><a href="https://arxiv.org/pdf/1511.00561.pdf" rel="nofollow noreferrer">https://arxiv.org/pdf/1511.00561.pdf</a> (vgg16 encode-decoder (+skip connections) for segmentation)</p>

<p><a href="https://github.com/divamgupta/image-segmentation-keras" rel="nofollow noreferrer">https://github.com/divamgupta/image-segmentation-keras</a></p>

<p><a href="https://github.com/upul/Semantic_Segmentation" rel="nofollow noreferrer">https://github.com/upul/Semantic_Segmentation</a></p>
What kernel size do I have to use when there are 3 channels on Conv2D Keras Layer?
<p>I've just started to learn CNN with Tensorflow and Keras.</p>

<p>I have found these two implementations, the first is for U-NET and the second one is for VGG-16:</p>

<pre><code>def unet(pretrained_weights = None,input_size = (256,256,1)):
    inputs = Input(input_size)
    conv1 = Conv2D(64, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(inputs)
</code></pre>

<p>VGG-16:</p>

<pre><code>def vgg16(input_size = (224,224,3)):
    model = Sequential()

    model.add(Conv2D(input_shape=input_size,filters=64,kernel_size=(3,3),padding="same", activation="relu"))
    model.add(Conv2D(filters=64,kernel_size=(3,3),padding="same", activation="relu"))
</code></pre>

<p>I have notices that on U-NET, they are using one channel images, and VGG-16, they are using three channels images. And also, U-NET Conv2D layer use <code>kernel_size</code> equals to 3, and on VGG-16, equals to (3, 3).</p>

<p>Is there any relationship between that with one channel image, they use 1D kernel size, and with three channels image, they use 2D kernel size?</p>