In this task, you are given a part of an article. Your task is to generate headline (title) for this text. Preferred headlines are under fifteen words.

[Q]: In this paper, we introduce a new deep convolutional neural network (ConvNet) module that promotes competition among a set of multi-scale convolutional filters. This new module is inspired by the inception module, where we replace the original collaborative pooling stage (consisting of a concatenation of the multi-scale filter outputs) by a competitive pooling represented by a maxout activation unit. This extension has the following two objectives: 1) the selection of the maximum response among the multi-scale filters prevents filter co-adaptation and allows the formation of multiple sub-networks within the same model, which has been shown to facilitate the training of complex learning problems; and 2) the maxout unit reduces the dimensionality of the outputs from the multi-scale filters. We show that the use of our proposed module in typical deep ConvNets produces classification results that are either better than or comparable to the state of the art on the following benchmark datasets: MNIST, CIFAR-10, CIFAR-100 and SVHN.
[A]: Competitive Multi-scale Convolution


[Q]: A famous biologically inspired hierarchical model firstly proposed by Riesenhuber and Poggio has been successfully applied to multiple visual recognition tasks. The model is able to achieve a set of positionand scale-tolerant recognition, which is a central problem in pattern recognition. In this paper, based on some other biological experimental results, we introduce the Memory and Association Mechanisms into the above biologically inspired model. The main motivations of the work are (a) to mimic the active memory and association mechanism and add the ’top down’ adjustment to the above biologically inspired hierarchical model and (b) to build up an algorithm which can save the space and keep a good recognition performance. More details of the work could be explained as follows: (1) In objects memorizing process: Our proposed model mimics some characteristics of human’s memory mechanism as follows: (a) In our model, one object is memorized by semantic attributes and special image patches (corresponding to episodic memory). The semantic attributes describe each part of the object with clear physical meaning, for example, if eyes and mouths of faces are ’big’ or ’small’ and so on. One special patch is selected if the value of the corresponding semantic feature is far from average one. The patch should be the most prominent part of the object. (b) In our model, different features (semantic attributes and special patches) of one object are stored in distributed places and the common feature of different objects is saved aggregately, which can learn to classify the difference of similar features of different objects. The similarity thresholds to each object can be learnt when new objects are learnt. (2) In object recognition process: In biological process, the associated recognition including familiarity discrimination and recollective matching. In our proposed model, firstly mimicking familiarity discrimination (’knowing’ though the episode), we compare the special patches of candidates with that of saved objects using above mentioned biologically inspired hierarchical model, where the candidates and saved objects have the same prominent semantic features. Then mimicking recollective matching, the comparison results of special patches are combined with semantic feature comparison. The new model is also applied to object recognition processes. The primary experimental results show that our method is efficient with much less memory requirement.
[A]: Introducing Memory and Association Mechanism Into a Biologically Inspired Visual Model


[Q]: Exponential Linear Units (ELUs) are a useful rectifier for constructing deep learning architectures, as they may speed up and otherwise improve learning by virtue of not have vanishing gradients and by having mean activations near zero [1]. However, the ELU activation as parametrized in [1] is not continuously differentiable with respect to its input when the shape parameter α is not equal to 1. We present an alternative parametrization which is C continuous for all values of α, making the rectifier easier to reason about and making α easier to tune. This alternative parametrization has several other useful properties that the original parametrization of ELU does not: 1) its derivative with respect to x is bounded, 2) it contains both the linear transfer function and ReLU as special cases, and 3) it is scale-similar with respect to α. The Exponential Linear Unit as described in [1] is as follows: ELU(x, α) = { x if x ≥ 0 α(exp(x)− 1) otherwise (1) Where x is the input to the function, and α is a shape parameter. The derivative of this function with respect to x is: d dx ELU(x, α) = { 1 if x ≥ 0 α exp(x) otherwise (2) In Figures 1a and 1b we plot this activation and its derivative with respect to x for different values of α. We see that when α 6= 1, the activation’s derivative is discontinuous at x = 0. Additionally we see that large values of α can cause a large (“exploding”) gradient for small negative values of x, which may make training difficult. Our alternative parametrization of the ELU, which we dub “CELU”, is simply the ELU where the activation for negative values has been modified to ensure that the derivative at x = 0 for all values of α is 1: CELU(x, α) = { x if x ≥ 0 α ( exp ( x α ) − 1 ) otherwise (3) Note that ELU and CELU are identical when α = 1: ∀x ELU(x, 1) = CELU(x, 1) (4) The derivative of the activation with respect to x and α are as follows: d dx CELU(x, α) = { 1 if x ≥ 0 exp ( x α ) otherwise (5) d dα CELU(x, α) = { 0 if x ≥ 0 exp ( x α ) ( 1− x α ) − 1 otherwise Like in ELU, derivatives for CELU can be computed efficiently by precomputing exp ( x α ) and using it for the activation and its derivatives. Unlike ELU, CELU is scale-similar as a function of x and α: CELU(x, α) = 1 c CELU(cx, cα) (6) The CELU also converges to ReLU as α approaches 0 from the right and converges to a linear “no-op” activation as α approaches∞: lim α→0+ CELU(x, α) = max(0, x) (7) lim<lb>α→∞<lb>CELU(x, α) = x<lb>(8) This gives the CELU a nice interpretation as a way to in-<lb>terpolate between a ReLU and a linear function using α.<lb>Naturally, CELU can be slightly shifted in x and y such that<lb>it converges to any arbitrary shifted ReLU, in case negative<lb>activations are desirable even for small values of α.<lb>References<lb>[1] D. Clevert, T. Unterthiner, and S. Hochreiter. Fast and accu-<lb>rate deep network learning by exponential linear units (elus).<lb>CoRR, abs/1511.07289, 2015. 1<lb>ar<lb>X<lb>iv<lb>:1<lb>70<lb>4.<lb>07<lb>48<lb>3v<lb>1<lb>[<lb>cs<lb>.L<lb>G<lb>]<lb>2<lb>4<lb>A<lb>pr<lb>2<lb>01<lb>7
[A]:
Continuously Differentiable Exponential Linear Units


