This disclosure describes techniques and architectures to morph well-trained networks to other related applications or modified networks with relatively little retraining. For example, a well-trained neural network (e.g., parent network) may be morphed to a new neural network (e.g., child network) so that the new neural network function may be preserved. After morphing a parent network, the child network may inherit the knowledge from its parent network and also may have a potential to continue growing into a more powerful network. Such morphing and growing may occur with a relatively short training time.
Please help me write a proper abstract based on the patent claims. CLAIM: 1. A system comprising: one or more processors; and memory storing instructions that, when executed by the one or more processors, configure the system to perform operations comprising: receiving a first neural network having a first level of knowledge; and morphing the first neural network to form a second neural network so that the second neural network inherits the first level of knowledge from the first neural network. 2. The system of claim 1, wherein the first neural network is a class neural network (multiple layer perceptrons). 3. The system of claim 1, wherein the first neural network is a deep convolutional neural network (DCNN). 4. The system of claim 1, wherein the operations further comprise training the second neural network to increase the first level of knowledge to a second level of knowledge. 5. A method for morphing a neural network, the method comprising: receiving the neural network that includes a first existing layer and a second existing layer; inserting a third layer between the first and the second existing layers; generating two or more new layers based, at least in part, on the first existing layer or the second existing layer; and extending channel size or kernel size of at least one convolutional filter of the neural network. 6. The method of claim 5, further comprising: splitting the third layer of the neural network to two or more stacked layers. 7. The method of claim 5, wherein at least one of the first existing layer or the second existing layer is a fully connected layer. 8. The method of claim 5, wherein the third layer is a fully connected layer. 9. The method of claim 5, wherein the layer is a convolutional layer. 10. The method of claim 5, wherein at least one of the first existing layer or the second existing layer is a convolutional layer. 11. The method of claim 5, further comprising padding the weight matrix or convolution filter with zeroes. 12. The method of claim 5, wherein morphing the neural network includes width morphing. 13. The method of claim 5, wherein morphing the neural network includes kernel size morphing. 14. The method of claim 5, further comprising forming a second neural network by morphing the first neural network by subnet. 15. The method of claim 14, wherein at least a portion of the neural network is nonlinear, and wherein forming the second neural network is based, at least in part, on a parametric-activation function. 16. The method of claim 15, wherein the parametric-activation function includes one or more parameters, the method further comprising training the one or more parameters over a time span. 17. A method comprising: receiving a parent neural network at least partially defined by a network function and outputs, wherein the parent neural network comprises a nonlinear portion of nodes and segments; morphing the depth of at least a portion of the nodes; after morphing the depth, morphing the width and kernel size of at least another portion of the nodes to generate a child neural network such that the child neural network preserves the network function and the outputs of the parent neural network. 18. The method of claim 17, further comprising: after morphing the width and the kernel size, morphing at least a portion of the segments to generate a subnet morphing of the parent neural network. 19. The method of claim 17, further comprising applying a parametric-activation function to the nonlinear portion of nodes and segments. 20. The method of claim 19, wherein the parametric-activation function includes one or more parameters, the method further comprising training the one or more parameters over a time span.