<p>The classic work in this area is <a href="http://arxiv.org/abs/1312.6082" rel="nofollow noreferrer">'Multi-digit Number Recognition from Street View Imagery using Deep Convolutional Neural Networks'</a> </p>

<p>Keras model (functional, not sequential):</p>

<pre><code>inputs = Input(shape=(28, 140, 1), name="input")
x = inputs
x = Conv2D(32, kernel_size=(3, 3), activation='relu', input_shape=(28, 140, 1))(x)
x = Conv2D(64, (3, 3), activation='relu')(x)
x = MaxPooling2D(pool_size=(2, 2))(x)
x = Dropout(0.25)(x)
x = Flatten()(x)
x = Dropout(0.5)(x)
digit1 = Dense(10, activation='softmax', name='digit1')(x)
digit2 = Dense(10, activation='softmax', name='digit2')(x)
digit3 = Dense(10, activation='softmax', name='digit3')(x)
digit4 = Dense(10, activation='softmax', name='digit4')(x)
digit5 = Dense(10, activation='softmax', name='digit5')(x)
predictions = [digit1,digit2,digit3,digit4,digit5]
model = Model(inputs=inputs, outputs=predictions)
model.compile(optimizer=Adam(), metrics=['accuracy'], oss='categorical_crossentropy')
</code></pre>

<p>PS
You may use 11 classes for 10 digits and empty space.</p>
Mnist recognition using keras
<p>How can I train the model to recognize five numbers in one picture.
The code is as follows:</p>

<pre><code>from keras.layers import Conv2D
from keras.layers import MaxPooling2D
from keras.layers import Flatten
from keras.layers import Dropout, Dense, Input
from keras.models import Model, Sequential

model = Sequential()
model.add(Conv2D(32, kernel_size=(3, 3),
             activation='relu',
             input_shape=(28, 140, 1)))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dropout(0.5))
</code></pre>

<p>Here should be a loop for recognizing each number in the picture, but I don't know how to realize it.</p>

<pre><code>model.add(Dense(11, activation='softmax'))

model.compile(loss=keras.losses.categorical_crossentropy,
          optimizer=keras.optimizers.Adadelta(),
          metrics=['accuracy'])

model.fit(X_train, y_train,
      batch_size=1000,
      epochs=8,
      verbose=1,
      validation_data=(X_valid, y_valid))
</code></pre>

<p>The picture of combined mnist number is as follows:</p>

<p><img src="https://i.stack.imgur.com/uOFFU.png" alt="combined numbers in one picture"></p>