<p>Your problem is that when you get to the convolution, your time dimension (2) is smaller than the filter that you have specified (5).</p>
<pre class="lang-py prettyprint-override"><code>import tensorflow as tf

from tensorflow.keras.layers import Input
from tensorflow.keras.layers import Permute
from tensorflow.keras.layers import Concatenate
from tensorflow.keras.layers import BatchNormalization
from tensorflow.keras.layers import Conv1D


# dummy data w/batch 32
X_train = tf.random.normal([32, 6249, 1])
X_train_phase = tf.random.normal([32, 6249, 1])
    
signal1 = Input(shape=(X_train.shape[1:]))
signal2 = Input(shape=(X_train_phase.shape[1:]))

concat_signal = Concatenate()([signal1, signal2])

x = Permute(dims=(2, 1))(concat_signal)
x = BatchNormalization()(x)

print(x.shape)
# (None, 2, 6249)
</code></pre>
<p>If you see the docs for <a href="https://www.tensorflow.org/api_docs/python/tf/keras/layers/Conv1D" rel="nofollow noreferrer"><code>tf.keras.layers.Conv1D</code></a>, you'll notice that <code>&quot;valid&quot;</code> is the default padding, which means there is no padding. There is a great reference, <a href="https://arxiv.org/pdf/1603.07285.pdf" rel="nofollow noreferrer">&quot;A guide to convolution arithmetic for deep
learning&quot;</a>, which does a good job of illustrating the relationship between input size, kernel size, strides, and padding.</p>
<p>While I am not sure what you're trying to accomplish with this network, adding the argument <code>padding=&quot;same&quot;</code> to your convolution layers will send the input through without issue.</p>
<pre class="lang-py prettyprint-override"><code>x = Conv1D(
    filters=64,
    kernel_size=5,
    activation=&quot;relu&quot;,
    padding=&quot;same&quot;,  # &lt;= add this.
    kernel_initializer=&quot;glorot_normal&quot;)(x)
</code></pre>
Permute Layer: Negative dimension size caused by subtracting 3 from 2
<p>I have two sensor inputs for which I have applied the Concatenate layer previously for fusion. Both of them are time series data for which I'm now trying to apply a permutation layer. However, when I do so, I get the error:</p>
<blockquote>
<p>Negative dimension size caused by subtracting 3 from 2 for '{{node conv1d_334/conv1d}} = Conv2D[T=DT_FLOAT, data_format=&quot;NHWC&quot;, dilations=[1, 1, 1, 1], explicit_paddings=[], padding=&quot;VALID&quot;, strides=[1, 1, 1, 1], use_cudnn_on_gpu=true](conv1d_334/conv1d/ExpandDims, conv1d_334/conv1d/ExpandDims_1)' with input shapes: [?,1,2,6249], [1,3,6249,128].</p>
</blockquote>
<p>My inputs are both time series data with input dimension <code>(1176, 6249, 1)</code>. Can anybody tell me what I'm doing wrong? Here is a sample code:</p>
<pre><code>lr = 0.0005
n_timesteps = 3750
n_features = 1
n_outputs = 3

def small_model(optimizer='rmsprop', init='glorot_uniform'):
    signal1 = Input(shape=(X_train.shape[1:]))
    signal2 = Input(shape=(X_train_phase.shape[1:]))
    
    concat_signal = Concatenate()([signal1, signal2])
    
    # x = InputLayer(input_shape=(None, X_train.shape[1:][0],1))(inputA)
    
    x = Permute(dims=(2, 1))(concat_signal)
    x = BatchNormalization()(x)
    x = Conv1D(64, 5, activation='relu', kernel_initializer='glorot_normal')(x) #, input_shape=(None, 3750, n_features)
    x = Conv1D(64, 5, activation='relu', kernel_initializer='glorot_normal')(x)
    x = MaxPooling1D(5)(x)
    x = Dropout(0.3)(x)
</code></pre>