Based on the approximate formulation , efficient algorithms for solving least squares problems can be applied to scale the formulation to very large data sets . In addition , existing regularization techniques for least squares can be incorporated into the model for improved generalization performance . We have conducted experiments using large-scale benchmark data sets , and experimental results show that the proposed hypergraph spectral learning formulation is effective in capturing the high-order relations in multi-label problems . We propose a new regularization framework for sparse online learning . We focus on regularization terms , and we enhance the state-of-the-art regularization approach by integrating information on all previous subgradients of the loss function into a regularization term . The resulting algorithms enable online learning to adjust the intensity of each feature ’s truncations without pre-processing and eventually eliminate the bias of L1-regularization . This paper presents an Artificial Neural Networks ( ANN ) as a machine learning approach to intrusion detection system for Denial of Service Attacks ( DoS ) and Distributed Denial of Service Attacks ( DDoS ) . The proposed method used the Bayesian Regularization ( BR ) backpropagation and scaled conjugate gradient ( SCG ) descent backpropagation algorithm . The network was trained and tested with subset of the CICID2017 dataset that meets the real world criteria . SRRS seeks low-rank representations from the noisy data , and learns a discriminative subspace from the recovered clean data jointly . A supervised regularization function is designed to make use of the class label information and therefore to enhance the discriminability of subspace . Our approach is formulated as a constrained rank minimization problem . SRRS seeks low-rank representations from the noisy data , and learns a discriminative subspace from the recovered clean data jointly . A supervised regularization function is designed to make use of the class label information , and therefore to enhance the discriminability of subspace . Our approach is formulated as a constrained rank-minimization problem . Prox-regularization algorithms for solving generalized fractional programs ( GFP ) were already considered by several authors . Since the standard dual of a generalized fractional program has not generally the form of GFP , these approaches can not apply directly to the dual problem . In this paper , we propose a primal-dual algorithm for solving convex generalized fractional programs . That is , we use a prox-regularization method to the dual problem that generates a sequence of auxiliary dual problems with unique solutions . So we can avoid the numerical difficulties that can occur if the fractional program does not have a unique solution . Second , the authors implement the regularization method with the variational iteration method for the same purpose . The effectiveness of the regularization-Homotopy method and the regularization-variational method is shown by using them for several illustrative examples , which have been solved by other authors using the so-called regularization-Adomian method . Findings The large-scale focusing inversion of gravity and magnetic potential field data using $ L_1$-norm regularization is considered . The use of the randomized singular value decomposition methodology facilitates tackling the computational challenge that arises in the solution of these large-scale inverse problems . From the typical anisotropic diffusion equations , the PDEs for image super-resolution are deduced and , then a robust regularization-type reconstruction algorithm is devised . The proposed approach can suppress the system noise and preserve more details of the estimated HR image by orientation-selective regularization . The experimental results show that the PDE-based approach makes a promising progress in image super-resolution . In this paper , we propose to solve the partial multi-modal sparse coding problem via multi-modal similarity structure regularization . Specifically , we propose a partial multi-modal sparse coding framework termed Adaptive Partial Multi-Modal Similarity Structure Regularization for Sparse Coding ( AdaPM2SC ) , which preserves the similarity structure within the same modality and between different modalities . Experimental results conducted on two real-world datasets demonstrate that AdaPM2SC significantly outperforms the state-of-the-art methods under partial multi-modality scenario . In Sanger data analysis , we introduce a strategy to identify expression modules indicating gene regulatory mechanisms via the principal component analysis ( PCA ) , and perform logistic regression modeling based on not a single gene but gene expression modules . We can see through Monte Carlo experiments and real world example that the proposed adaptive penalized logistic regression outperforms feature selection and classification compared with existing L < inline-formula><tex-math notation="LaTeX " > $ _ {1}$</tex-math><alternatives><inline-graphic xlink : href="park-ieq2 - 2561937.gif"/></alternatives></inline-formula > -type regularization . The discriminately imposed penalty based on WRST effectively performs crucial gene selection , and thus our method can improve classification accuracy without interruption of noise genes . Adaptative cubic regularization ( ARC ) methods for unconstrained optimization compute steps from linear systems with a shifted Hessian in the spirit of the modified Newton method . In the simplest case , the shift is a multiple of the identity , which is typically identified by trial and error . Key statistics for these datasets is presented in Table [ reference ] . As mentioned in the introduction , some of the best existing results on these datasets are obtained by using extensive regularization techniques on relatively large LSTMs ( ) . We apply our regularization technique to these models , the so called AWD - LSTM . Consequently , more insights on CNN architecture design can be obtained . Due to the ill - posed nature of SISR , regularization needs to be imposed to constrain the solution . Mathematically , the HR counterpart of an LR image y can be estimated by solving the following MAP problem where is the data fidelity term , is the regularization term ( or prior term ) and is the trade - off parameter . | to approximately match the magnitudes of the derivatives of J and J aug ( see ( 4.3 ) ) . Since we aim to achieve the minimum training loss possible , and the goal is to show a particular result rather than to achieve good generalization , we do not use any kind of regularization in the neural network ( e.g. weight decay , dropout ) . For this set of experiments , we also constrain each row of the input embedding matrix to have a norm of 1 because training becomes difficult without this constraint when only augmented loss is used . Then , acting as a “ student , ” it is retrained on the new training set . Many recent approaches ( including the consistentency regularization methods discussed below and our own method ) train the student with soft targets from the teacher ’s output distribution rather than a hard label , making the procedure more akin to knowledge distillation hinton2015distilling . It is also possible to use multiple models or prediction modules for the teacher , such as in tri - training zhou2005tri , ruder2018strong . Consistency Regularization . Recent works add noise ( e.g. , drawn from a Gaussian distribution ) or apply stochastic transformations ( e.g. , horizontally flipping an image ) to the student ’s inputs . This trains the model to give consistent predictions to nearby data points , encouraging distributional smoothness in the model . Consistency regularization has been very successful for computer vision applications bachman2014learning , laine2016temporal , tarvainen2017weight . However , stochastic input alterations are more difficult to apply to discrete data like text , making consistency regularization less used for natural language processing . One solution is to add noise to the model Multi - view learning on data where features can be separated into distinct subsets has been well studied Xu2013ASO . Particularly relevant are co - training blum1998combining and co - regularization Sindhwani2005ACA , which trains two models with disjoint views of the input . On unlabeled data , each one acts as a “ teacher ” for the other model . – based dropout Gal2015 and zoneout Krueger2016 . These schemes extend dropout to the recurrent setting by taking advantage of the repeating structure of recurrent networks , providing more powerful and less destructive regularization . Variational inference – based dropout locks the dropout mask used for the recurrent connections across timesteps , so a single RNN pass uses a single stochastic subset of the recurrent weights . It is also important to note that the cuDNN library ’s RNN primitives do not natively support any form of recurrent dropout . That is , running an LSTM that uses a state - of - the - art regularization scheme at cuDNN - like speeds would likely require an entirely custom kernel . Batch size First , for categorical image generation , we apply the proposed method on DCGAN using the CIFAR - 10 dataset . Second , for image - to - image translation , we embed the proposed regularization scheme in Pix2Pix and DRIT using the facades , maps , Yosemite , and cat dog datasets . Third , for text - to - image synthesis , we incorporate StackGAN ++ with the proposed regularization term using the CUB - 200 - 2011 dataset . We make the following three contributions towards this goal : We demonstrate that GANs benefit dramatically from scaling , and train models with two to four times as many parameters and eight times the batch size compared to prior art . We introduce two simple , general architectural changes that improve scalability , and modify a regularization scheme to improve conditioning , demonstrably boosting performance . As a side effect of our modifications , our models become amenable to the “ truncation trick , ” a simple sampling technique that allows explicit , fine - grained control of the trade - off between sample variety and fidelity . document : Loss - Sensitive Generative Adversarial Networks on Lipschitz Densities In this paper , we present the Lipschitz regularization theory and algorithms for a novel Loss - Sensitive Generative Adversarial Network ( LS - GAN ) . Specifically , it trains a loss function to distinguish between real and fake samples by designated margins , while learning a generator alternately to produce realistic samples by minimizing their losses . ( 2 ) We present a novel flow regularization layer to ameliorate the issue of outliers and vague flow boundaries by using a feature - driven local convolution . ( 3 ) Our network owns an effective structure for pyramidal feature extraction and embraces feature warping rather than image warping as practiced in FlowNet2 . In particular , if we are willing to sacrifice the ability to perform ancestral sampling , can we improve our performance ? The regularization view of the elbo ( Eq . 6 ) introduces a trade - off between how well we can fit the data and how close the approximate posterior stays to the prior during learning . We propose using β 1 . For models such as NNs for which we can evaluate the gradient of the output with respect to the input , however , virtual adversarial perturbation admits an approximation that can be computed efficiently with the power method [ reference ] . This property enables us to implement VAT for NNs with no more than three times the computational cost of the standard , regularization - free training . This approximation step is an important part of the VAT algorithm that makes it readily applicable for various settings and model architectures . A new label smoothing method that makes use of prior knowledge of a language at human level , homophone , is proposed in this paper for automatic speech recognition ( ASR ) . Compared with its forerunners , the proposed method uses pronunciation knowledge of homophones in a more complex way . Most recent computer vision approaches against label noise are relatively complex , requiring complex networks or extra data resources . In this work , we evaluate simple and efficient model-agnostic approaches to handling noisy labels when training sound event classifiers , namely label smoothing regularization , mixup and noise-robust loss functions . The main advantage of these methods is that they can be easily incorporated to existing deep learning pipelines without need for network modifications or extra resources . Second , a novel domain adaptive network is designed to decompose the representations to person-related and domain-related part . The former aims at learning domain-invariant and discriminative representation by a adversarial loss and a Re-ID loss with the label smoothing regularization . And the latter further improve a model ’s ability of extracting domain-invariant features by separating the domain unique features . However , the fabric smoothness labels contain sequence information , and the problem shall be defined as an ordinal classification problem . This article presents an effective method including an image preprocessing algorithm , a compact convolutional neural network(CNN ) model , and a label smoothing process . Compared with the commonly used CNN frameworks , the proposed compact CNN model is more suitable for this small-sample and low-abstraction problem . The image processing algorithm can improve the model ’s illumination adaptability , and the label smoothing process can modify the model to satisfy the ordinal classification problems better . In the experiments , the method is tested on a fabric image set including 385 graded fabric specimens . We can adjust the hyperparameter cardinality instead of the network depth to extract more discriminative features of HSIs and improve the classification accuracy . On the other hand , in order to improve the classification accuracies of classes with small numbers of samples , we enrich the input of the 3D-ResNeXt spectral-spatial feature learning network by additional spectral feature learning , and finally use a loss function modified by label smoothing strategy to solve the imbalance of classes . The experimental results on three popular HSI datasets demonstrate the superiority of our proposed network and an effective improvement in the accuracies especially for the classes with small numbers of training samples . However , while conceptually similar , theoretical relation between LPA and GCN has not yet been investigated . Here we study the relationship between LPA and GCN in terms of two aspects : ( 1 ) feature/label smoothing where we analyze how the feature/label of one node is spread over its neighbors ; And , ( 2 ) feature/label influence of how much the initial feature/label of one node influences the final feature/label of another node . Based on our theoretical analysis , we propose an end-to-end model that unifies GCN and LPA for node classification . Under a Bayesian parameter estimation framework , we employ different probability distributions depending on the correspondence labels . We incorporate models for image similarity , an image intensity prior , label map smoothing , and a transformation prior that encourages deformation near the estimated tumor location . Our proposed algorithm increases registration accuracy compared to a traditional voxel-based registration method as shown using both synthetic and real patient images . For occlusion invariance , a feature-level GAN is incorporated to generate more challenging feature-level GAN data by creating random occlusion masks in deep feature space . To facilitate the online learning process , a label smoothing loss regularization is introduced to achieve model regularization and over-fitting reduction by integrating the unlabeled GAN-generated training data with the realistically labeled ones . In addition , a re-detection correlation filter conservatively trained with reliable training data is employed to integrate a classification score metric to perform reliable model updates and avoid heavy degradation . In this paper , we propose a new label smoothing regularization scheme for person re-identification . We first use an unsupervised method for discriminative learning representation . However , stabilizing and training cGAN systems is difficult and they still fall short of the performance achieved by the spectral enhancement approaches . This paper investigates whether different normalization strategies and one-sided label smoothing can further stabilize the cGAN-based speech enhancement model . In addition , we propose incorporating a Gammatone-based auditory filtering layer and a trainable pre-emphasis layer to further improve the performance of the cGAN framework . On the other hand , our training criterion can also help the existing model adapt to new speech characteristics from unlabeled speech . In our experiments of phonetic classification , there is a consistent reduction of error rate from MLE to MMIE with I-smoothing , and then to MMIE with unlabeled-smoothing . Error rates can be further reduced by transductive-MMIE . For the base model , we use a rate of . paragraph : Label Smoothing During training , we employed label smoothing of value DBLP : journals / corr / SzegedyVISW15 . This hurts perplexity , as the model learns to be more unsure , but improves accuracy and BLEU score . Szegedy et al . [ reference ] introduced label - smoothing regularization for training deep Inception models . In this paper , we bring together concepts from pairwise learning and label confusion and take a step towards solving the problems of overfitting and samplespecific artifacts when training neural networks for FGVC tasks . We additionally compare the performance of our optimization technique with other regularization methods as well . We first compare Pairwise Comparison with with Label - Smoothing Regularization ( LSR ) on all six FGVC datasets for VGG - Net16 , ResNet - 50 and DenseNet - 161 . These results are summarized in Table S7 . We propose to use the prior distribution over labels as . In our experiments , we used the uniform distribution , so that We refer to this change in ground - truth label distribution as label - smoothing regularization , or LSR . Note that LSR achieves the desired goal of preventing the largest logit from becoming much larger than all others . This problem reduces the benefit of generated samples under full - camera systems where the relatively abundant data has a lower over - fitting risk . To alleviate this problem , we apply the label smoothing regularization ( LSR ) [ reference ] on the style - transferred images to softly distribute their labels . That is , we assign less confidence on the ground - truth label and assign small weights to the other classes . Overfitting is a crucial problem in deep neural networks , even in the latest network architectures . In this paper , to relieve the overfitting effect of ResNet and its improvements ( i.e. , Wide ResNet , PyramidNet , and ResNeXt ) , we propose a new regularization method called ShakeDrop regularization . ShakeDrop is inspired by Shake-Shake , which is an effective regularization method , but can be applied to ResNeXt only . Because effective regularization often causes unstable training , we introduce a training stabilizer , which is an unusual use of an existing regularizer . Through experiments under various conditions , we demonstrate the conditions under which ShakeDrop works well . In our experiments , we search for the best convolutional layer ( or " cell " ) on the CIFAR-10 dataset and then apply this cell to the ImageNet dataset by stacking together more copies of this cell , each with their own parameters to design a convolutional architecture , which we name a " NASNet architecture " . We also introduce a new regularization technique called ScheduledDropPath that significantly improves generalization in the NASNet models . On CIFAR-10 itself , a NASNet found by our method achieves 2.4 % error rate , which is state-of-the-art .
Based on the approximate formulation , efficient algorithms for solving least squares problems can be applied to scale the formulation to very large data sets . In addition , existing regularization techniques for least squares can be incorporated into the model for improved generalization performance . We have conducted experiments using large-scale benchmark data sets , and experimental results show that the proposed hypergraph spectral learning formulation is effective in capturing the high-order relations in multi-label problems .
We propose a new regularization framework for sparse online learning . We focus on regularization terms , and we enhance the state-of-the-art regularization approach by integrating information on all previous subgradients of the loss function into a regularization term . The resulting algorithms enable online learning to adjust the intensity of each feature ’s truncations without pre-processing and eventually eliminate the bias of L1-regularization .
This paper presents an Artificial Neural Networks ( ANN ) as a machine learning approach to intrusion detection system for Denial of Service Attacks ( DoS ) and Distributed Denial of Service Attacks ( DDoS ) . The proposed method used the Bayesian Regularization ( BR ) backpropagation and scaled conjugate gradient ( SCG ) descent backpropagation algorithm . The network was trained and tested with subset of the CICID2017 dataset that meets the real world criteria .
SRRS seeks low-rank representations from the noisy data , and learns a discriminative subspace from the recovered clean data jointly . A supervised regularization function is designed to make use of the class label information and therefore to enhance the discriminability of subspace . Our approach is formulated as a constrained rank minimization problem .
SRRS seeks low-rank representations from the noisy data , and learns a discriminative subspace from the recovered clean data jointly . A supervised regularization function is designed to make use of the class label information , and therefore to enhance the discriminability of subspace . Our approach is formulated as a constrained rank-minimization problem .
Prox-regularization algorithms for solving generalized fractional programs ( GFP ) were already considered by several authors . Since the standard dual of a generalized fractional program has not generally the form of GFP , these approaches can not apply directly to the dual problem . In this paper , we propose a primal-dual algorithm for solving convex generalized fractional programs . That is , we use a prox-regularization method to the dual problem that generates a sequence of auxiliary dual problems with unique solutions . So we can avoid the numerical difficulties that can occur if the fractional program does not have a unique solution .
Second , the authors implement the regularization method with the variational iteration method for the same purpose . The effectiveness of the regularization-Homotopy method and the regularization-variational method is shown by using them for several illustrative examples , which have been solved by other authors using the so-called regularization-Adomian method . Findings
The large-scale focusing inversion of gravity and magnetic potential field data using $ L_1$-norm regularization is considered . The use of the randomized singular value decomposition methodology facilitates tackling the computational challenge that arises in the solution of these large-scale inverse problems .
From the typical anisotropic diffusion equations , the PDEs for image super-resolution are deduced and , then a robust regularization-type reconstruction algorithm is devised . The proposed approach can suppress the system noise and preserve more details of the estimated HR image by orientation-selective regularization . The experimental results show that the PDE-based approach makes a promising progress in image super-resolution .
In this paper , we propose to solve the partial multi-modal sparse coding problem via multi-modal similarity structure regularization . Specifically , we propose a partial multi-modal sparse coding framework termed Adaptive Partial Multi-Modal Similarity Structure Regularization for Sparse Coding ( AdaPM2SC ) , which preserves the similarity structure within the same modality and between different modalities . Experimental results conducted on two real-world datasets demonstrate that AdaPM2SC significantly outperforms the state-of-the-art methods under partial multi-modality scenario .
In Sanger data analysis , we introduce a strategy to identify expression modules indicating gene regulatory mechanisms via the principal component analysis ( PCA ) , and perform logistic regression modeling based on not a single gene but gene expression modules . We can see through Monte Carlo experiments and real world example that the proposed adaptive penalized logistic regression outperforms feature selection and classification compared with existing L < inline-formula><tex-math notation="LaTeX " > $ _ {1}$</tex-math><alternatives><inline-graphic xlink : href="park-ieq2 - 2561937.gif"/></alternatives></inline-formula > -type regularization . The discriminately imposed penalty based on WRST effectively performs crucial gene selection , and thus our method can improve classification accuracy without interruption of noise genes .
Adaptative cubic regularization ( ARC ) methods for unconstrained optimization compute steps from linear systems with a shifted Hessian in the spirit of the modified Newton method . In the simplest case , the shift is a multiple of the identity , which is typically identified by trial and error .
Key statistics for these datasets is presented in Table [ reference ] . As mentioned in the introduction , some of the best existing results on these datasets are obtained by using extensive regularization techniques on relatively large LSTMs ( ) . We apply our regularization technique to these models , the so called AWD - LSTM .
Consequently , more insights on CNN architecture design can be obtained . Due to the ill - posed nature of SISR , regularization needs to be imposed to constrain the solution . Mathematically , the HR counterpart of an LR image y can be estimated by solving the following MAP problem where is the data fidelity term , is the regularization term ( or prior term ) and is the trade - off parameter .
| to approximately match the magnitudes of the derivatives of J and J aug ( see ( 4.3 ) ) . Since we aim to achieve the minimum training loss possible , and the goal is to show a particular result rather than to achieve good generalization , we do not use any kind of regularization in the neural network ( e.g. weight decay , dropout ) . For this set of experiments , we also constrain each row of the input embedding matrix to have a norm of 1 because training becomes difficult without this constraint when only augmented loss is used .
Then , acting as a “ student , ” it is retrained on the new training set . Many recent approaches ( including the consistentency regularization methods discussed below and our own method ) train the student with soft targets from the teacher ’s output distribution rather than a hard label , making the procedure more akin to knowledge distillation hinton2015distilling . It is also possible to use multiple models or prediction modules for the teacher , such as in tri - training zhou2005tri , ruder2018strong . Consistency Regularization . Recent works add noise ( e.g. , drawn from a Gaussian distribution ) or apply stochastic transformations ( e.g. , horizontally flipping an image ) to the student ’s inputs . This trains the model to give consistent predictions to nearby data points , encouraging distributional smoothness in the model . Consistency regularization has been very successful for computer vision applications bachman2014learning , laine2016temporal , tarvainen2017weight . However , stochastic input alterations are more difficult to apply to discrete data like text , making consistency regularization less used for natural language processing . One solution is to add noise to the model
Multi - view learning on data where features can be separated into distinct subsets has been well studied Xu2013ASO . Particularly relevant are co - training blum1998combining and co - regularization Sindhwani2005ACA , which trains two models with disjoint views of the input . On unlabeled data , each one acts as a “ teacher ” for the other model .
– based dropout Gal2015 and zoneout Krueger2016 . These schemes extend dropout to the recurrent setting by taking advantage of the repeating structure of recurrent networks , providing more powerful and less destructive regularization . Variational inference – based dropout locks the dropout mask used for the recurrent connections across timesteps , so a single RNN pass uses a single stochastic subset of the recurrent weights .
It is also important to note that the cuDNN library ’s RNN primitives do not natively support any form of recurrent dropout . That is , running an LSTM that uses a state - of - the - art regularization scheme at cuDNN - like speeds would likely require an entirely custom kernel . Batch size
First , for categorical image generation , we apply the proposed method on DCGAN using the CIFAR - 10 dataset . Second , for image - to - image translation , we embed the proposed regularization scheme in Pix2Pix and DRIT using the facades , maps , Yosemite , and cat dog datasets . Third , for text - to - image synthesis , we incorporate StackGAN ++ with the proposed regularization term using the CUB - 200 - 2011 dataset .
We make the following three contributions towards this goal : We demonstrate that GANs benefit dramatically from scaling , and train models with two to four times as many parameters and eight times the batch size compared to prior art . We introduce two simple , general architectural changes that improve scalability , and modify a regularization scheme to improve conditioning , demonstrably boosting performance . As a side effect of our modifications , our models become amenable to the “ truncation trick , ” a simple sampling technique that allows explicit , fine - grained control of the trade - off between sample variety and fidelity .
document : Loss - Sensitive Generative Adversarial Networks on Lipschitz Densities In this paper , we present the Lipschitz regularization theory and algorithms for a novel Loss - Sensitive Generative Adversarial Network ( LS - GAN ) . Specifically , it trains a loss function to distinguish between real and fake samples by designated margins , while learning a generator alternately to produce realistic samples by minimizing their losses .
( 2 ) We present a novel flow regularization layer to ameliorate the issue of outliers and vague flow boundaries by using a feature - driven local convolution . ( 3 ) Our network owns an effective structure for pyramidal feature extraction and embraces feature warping rather than image warping as practiced in FlowNet2 .
In particular , if we are willing to sacrifice the ability to perform ancestral sampling , can we improve our performance ? The regularization view of the elbo ( Eq . 6 ) introduces a trade - off between how well we can fit the data and how close the approximate posterior stays to the prior during learning . We propose using β 1 .
For models such as NNs for which we can evaluate the gradient of the output with respect to the input , however , virtual adversarial perturbation admits an approximation that can be computed efficiently with the power method [ reference ] . This property enables us to implement VAT for NNs with no more than three times the computational cost of the standard , regularization - free training . This approximation step is an important part of the VAT algorithm that makes it readily applicable for various settings and model architectures .
A new label smoothing method that makes use of prior knowledge of a language at human level , homophone , is proposed in this paper for automatic speech recognition ( ASR ) . Compared with its forerunners , the proposed method uses pronunciation knowledge of homophones in a more complex way .
Most recent computer vision approaches against label noise are relatively complex , requiring complex networks or extra data resources . In this work , we evaluate simple and efficient model-agnostic approaches to handling noisy labels when training sound event classifiers , namely label smoothing regularization , mixup and noise-robust loss functions . The main advantage of these methods is that they can be easily incorporated to existing deep learning pipelines without need for network modifications or extra resources .
Second , a novel domain adaptive network is designed to decompose the representations to person-related and domain-related part . The former aims at learning domain-invariant and discriminative representation by a adversarial loss and a Re-ID loss with the label smoothing regularization . And the latter further improve a model ’s ability of extracting domain-invariant features by separating the domain unique features .
However , the fabric smoothness labels contain sequence information , and the problem shall be defined as an ordinal classification problem . This article presents an effective method including an image preprocessing algorithm , a compact convolutional neural network(CNN ) model , and a label smoothing process . Compared with the commonly used CNN frameworks , the proposed compact CNN model is more suitable for this small-sample and low-abstraction problem . The image processing algorithm can improve the model ’s illumination adaptability , and the label smoothing process can modify the model to satisfy the ordinal classification problems better . In the experiments , the method is tested on a fabric image set including 385 graded fabric specimens .
We can adjust the hyperparameter cardinality instead of the network depth to extract more discriminative features of HSIs and improve the classification accuracy . On the other hand , in order to improve the classification accuracies of classes with small numbers of samples , we enrich the input of the 3D-ResNeXt spectral-spatial feature learning network by additional spectral feature learning , and finally use a loss function modified by label smoothing strategy to solve the imbalance of classes . The experimental results on three popular HSI datasets demonstrate the superiority of our proposed network and an effective improvement in the accuracies especially for the classes with small numbers of training samples .
However , while conceptually similar , theoretical relation between LPA and GCN has not yet been investigated . Here we study the relationship between LPA and GCN in terms of two aspects : ( 1 ) feature/label smoothing where we analyze how the feature/label of one node is spread over its neighbors ; And , ( 2 ) feature/label influence of how much the initial feature/label of one node influences the final feature/label of another node . Based on our theoretical analysis , we propose an end-to-end model that unifies GCN and LPA for node classification .
Under a Bayesian parameter estimation framework , we employ different probability distributions depending on the correspondence labels . We incorporate models for image similarity , an image intensity prior , label map smoothing , and a transformation prior that encourages deformation near the estimated tumor location . Our proposed algorithm increases registration accuracy compared to a traditional voxel-based registration method as shown using both synthetic and real patient images .
For occlusion invariance , a feature-level GAN is incorporated to generate more challenging feature-level GAN data by creating random occlusion masks in deep feature space . To facilitate the online learning process , a label smoothing loss regularization is introduced to achieve model regularization and over-fitting reduction by integrating the unlabeled GAN-generated training data with the realistically labeled ones . In addition , a re-detection correlation filter conservatively trained with reliable training data is employed to integrate a classification score metric to perform reliable model updates and avoid heavy degradation .
In this paper , we propose a new label smoothing regularization scheme for person re-identification . We first use an unsupervised method for discriminative learning representation .
However , stabilizing and training cGAN systems is difficult and they still fall short of the performance achieved by the spectral enhancement approaches . This paper investigates whether different normalization strategies and one-sided label smoothing can further stabilize the cGAN-based speech enhancement model . In addition , we propose incorporating a Gammatone-based auditory filtering layer and a trainable pre-emphasis layer to further improve the performance of the cGAN framework .
On the other hand , our training criterion can also help the existing model adapt to new speech characteristics from unlabeled speech . In our experiments of phonetic classification , there is a consistent reduction of error rate from MLE to MMIE with I-smoothing , and then to MMIE with unlabeled-smoothing . Error rates can be further reduced by transductive-MMIE .
For the base model , we use a rate of . paragraph : Label Smoothing During training , we employed label smoothing of value DBLP : journals / corr / SzegedyVISW15 . This hurts perplexity , as the model learns to be more unsure , but improves accuracy and BLEU score .
Szegedy et al . [ reference ] introduced label - smoothing regularization for training deep Inception models . In this paper , we bring together concepts from pairwise learning and label confusion and take a step towards solving the problems of overfitting and samplespecific artifacts when training neural networks for FGVC tasks .
We additionally compare the performance of our optimization technique with other regularization methods as well . We first compare Pairwise Comparison with with Label - Smoothing Regularization ( LSR ) on all six FGVC datasets for VGG - Net16 , ResNet - 50 and DenseNet - 161 . These results are summarized in Table S7 .
We propose to use the prior distribution over labels as . In our experiments , we used the uniform distribution , so that We refer to this change in ground - truth label distribution as label - smoothing regularization , or LSR . Note that LSR achieves the desired goal of preventing the largest logit from becoming much larger than all others .
This problem reduces the benefit of generated samples under full - camera systems where the relatively abundant data has a lower over - fitting risk . To alleviate this problem , we apply the label smoothing regularization ( LSR ) [ reference ] on the style - transferred images to softly distribute their labels . That is , we assign less confidence on the ground - truth label and assign small weights to the other classes .
Overfitting is a crucial problem in deep neural networks , even in the latest network architectures . In this paper , to relieve the overfitting effect of ResNet and its improvements ( i.e. , Wide ResNet , PyramidNet , and ResNeXt ) , we propose a new regularization method called ShakeDrop regularization . ShakeDrop is inspired by Shake-Shake , which is an effective regularization method , but can be applied to ResNeXt only .
Because effective regularization often causes unstable training , we introduce a training stabilizer , which is an unusual use of an existing regularizer . Through experiments under various conditions , we demonstrate the conditions under which ShakeDrop works well .
In our experiments , we search for the best convolutional layer ( or " cell " ) on the CIFAR-10 dataset and then apply this cell to the ImageNet dataset by stacking together more copies of this cell , each with their own parameters to design a convolutional architecture , which we name a " NASNet architecture " . We also introduce a new regularization technique called ScheduledDropPath that significantly improves generalization in the NASNet models . On CIFAR-10 itself , a NASNet found by our method achieves 2.4 % error rate , which is state-of-the-art .
Hypergraph spectral learning for multi-label classification Feature-aware regularization for sparse online learning Towards DoS/DDoS Attack Detection Using Artificial Neural Networks Robust Subspace Learning Learning Robust and Discriminative Subspace With Low-Rank Constraints Prox-dual regularization algorithm for generalized fractional programs Two efficient methods for solving Schlömilch's integral equation Improving the use of the randomized singular value decomposition for the inversion of gravity and magnetic data Image Super-resolution Using PDE Regularization Partial Multi-Modal Sparse Coding via Adaptive Similarity Structure Regularization A Novel Adaptive Penalized Logistic Regression for Uncovering Biomarker Associated with Anti-Cancer Drug Sensitivity Scalable adaptative cubic regularization methods Learning a Single Convolutional Super-Resolution Network for Multiple Degradations Tying Word Vectors and Word Classifiers: A Loss Framework for Language Modeling Semi-Supervised Sequence Modeling with Cross-View Training Quasi-Recurrent Neural Networks Mode Seeking Generative Adversarial Networks for Diverse Image Synthesis Large Scale GAN Training for High Fidelity Natural Image Synthesis Loss-Sensitive Generative Adversarial Networks on Lipschitz Densities LiteFlowNet: A Lightweight Convolutional Neural Network for Optical Flow Estimation Variational Autoencoders for Collaborative Filtering Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning Homophone-based Label Smoothing in End-to-End Automatic Speech Recognition Model-Agnostic Approaches To Handling Noisy Labels When Training Sound Event Classifiers Unsupervised Person Re-identification Based on Clustering and Domain-Invariant Network Automatic Assessment of Fabric Smoothness Appearance Based on a Compact Convolutional Neural Network With Label Smoothing Three-Dimensional ResNeXt Network Using Feature Fusion and Label Smoothing for Hyperspectral Image Classification Unifying Graph Convolutional Neural Networks and Label Propagation Non-rigid registration of longitudinal brain tumor treatment MRI Robust Visual Tracking based on Adversarial Unlabeled Instance Generation with Label Smoothing Loss Regularization Person Re-identification through Clustering and Partial Label Smoothing Regularization iSEGAN: Improved Speech Enhancement Generative Adversarial Networks Maximum mutual information estimation with unlabeled data for phonetic classification Attention is All you Need Pairwise Confusion for Fine-Grained Visual Classification Rethinking the Inception Architecture for Computer Vision Camera Style Adaptation for Person Re-identification Shakedrop Regularization for Deep Residual Learning Learning Transferable Architectures for Scalable Image Recognition
2da7a7a2ddc2b9dd27181bc6feb0f5129dbe3fd3 083a07c1d68e755c2f7bf738ba4ed1b893e45e55 1317c07bd1a486fcfe10c6fb91fb0a2f029a8ca6 395065c4d500e5a21cd67b3abe6920da589af385 7f7e7719a88a35eadd7739bc50bb59b9bc94ef6c eac348202b68415aaea47d9ad17cd4e2aaa525e1 f453bbb1b83855b544ffaee4bc3a2a42e25ad0a1 31676974ae8688c91cbfb3a279b93a43be0ef187 479fe1b6ea86f58ba1c32c9ef0cd48489ba2aa2f 645996bc6da54ba056ad741f5f8161fc5fd88dbb 0f20a0543fc28c1d26ba8ff5bbd46cbdda00207b 80d1cafede7f35decf5e9b473885cc47c579ae58 f092a38ab8370caa1e078afc3d39463c4cb3f1c4 c841631ffebea5928e7609123abc86d204147e6a e2e9b1224786aac104916979f47c26d7010ed9cc 788703c4db226e463867131a3c19fca4ab5487aa 04582f1f4cd31c51468250dff1cf86be0572e2d2 c397fdc468eb878db233dc80bf65a35a5b87a80e 06bcf2b799586d98001db1b5803f03c00ce03b52 78dd13bb4ae85690f66c67ab8abfd020ddc8ffe3 b3bbf0efbd272831e89fbb09634636c70bcf2dfa 29bdd183402a94e3cca7531ced412bb427e9285a cfc4cd3f84969d59b0c853b29ffd34374932ee4f b842cd963a5e61c48b467c92aee035485256ee12 d0611891b9e8a7c5731146097b6f201578f47b2f
["Computer Science"] ["Mathematics","Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science","Medicine"] ["Mathematics"] ["Computer Science"] ["Physics","Computer Science","Mathematics","Geology"] ["Computer Science"] ["Computer Science"] ["Computer Science","Medicine"] ["Computer Science"] ["Computer Science","Engineering"] ["Computer Science","Engineering","Mathematics"] ["Computer Science"] ["Computer Science"] ["Engineering","Medicine","Computer Science"] ["Mathematics","Computer Science"] ["Computer Science","Medicine"] ["Mathematics","Computer Science"] ["Computer Science"] ["Computer Science","Engineering"] ["Computer Science"] ["Computer Science"] ["Computer Science"]
2008 2014 2018 2017 2016 2017 2017 2019 2007 2016 2017 2015 0 2018 2017 2018 2017 2019 2019 2019 2018 2018 2019 2020 2019 2019 2020 2020 2020 2011 2020 2019 2020 2008 2017 2018 2016 2018 2019 2017
KDD UEMCON ACM Multimedia 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition ICLR EMNLP ICLR 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) ICLR International Journal of Computer Vision 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition WWW IEEE Transactions on Pattern Analysis and Machine Intelligence WASPAA ICIG (3) EMBC INTERSPEECH NIPS ECCV 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
https://api.semanticscholar.org/2da7a7a2ddc2b9dd27181bc6feb0f5129dbe3fd3 https://api.semanticscholar.org/083a07c1d68e755c2f7bf738ba4ed1b893e45e55 https://api.semanticscholar.org/1317c07bd1a486fcfe10c6fb91fb0a2f029a8ca6 https://api.semanticscholar.org/395065c4d500e5a21cd67b3abe6920da589af385 https://api.semanticscholar.org/7f7e7719a88a35eadd7739bc50bb59b9bc94ef6c https://api.semanticscholar.org/eac348202b68415aaea47d9ad17cd4e2aaa525e1 https://api.semanticscholar.org/f453bbb1b83855b544ffaee4bc3a2a42e25ad0a1 https://api.semanticscholar.org/31676974ae8688c91cbfb3a279b93a43be0ef187 https://api.semanticscholar.org/479fe1b6ea86f58ba1c32c9ef0cd48489ba2aa2f https://api.semanticscholar.org/645996bc6da54ba056ad741f5f8161fc5fd88dbb https://api.semanticscholar.org/0f20a0543fc28c1d26ba8ff5bbd46cbdda00207b https://api.semanticscholar.org/80d1cafede7f35decf5e9b473885cc47c579ae58 https://www.semanticscholar.org/paper/b26b4c9da872d3c6122dbf23b9d6e063dc6456b5 https://www.semanticscholar.org/paper/424aef7340ee618132cc3314669400e23ad910ba https://www.semanticscholar.org/paper/0c47cad9729c38d9db1f75491b1ee4bd883a5d4e https://www.semanticscholar.org/paper/2d876ed1dd2c58058d7197b734a8e4d349b8f231 https://www.semanticscholar.org/paper/cec3bffdb0968cd820863005af54cc519704c24a https://www.semanticscholar.org/paper/22aab110058ebbd198edb1f1e7b4f69fb13c0613 https://www.semanticscholar.org/paper/4b71ab38d3ac6eee0da5e68ef666d33d1028bad8 https://www.semanticscholar.org/paper/051b3763c2ad4e4271db712b0e9a4cfe298d05db https://www.semanticscholar.org/paper/23f5854b38a15c2ae201e751311665f7995b5e10 https://www.semanticscholar.org/paper/4b1c6f6521da545892f3f5dc39461584d4a27ec0 https://api.semanticscholar.org/f092a38ab8370caa1e078afc3d39463c4cb3f1c4 https://api.semanticscholar.org/c841631ffebea5928e7609123abc86d204147e6a https://api.semanticscholar.org/e2e9b1224786aac104916979f47c26d7010ed9cc https://api.semanticscholar.org/788703c4db226e463867131a3c19fca4ab5487aa https://api.semanticscholar.org/04582f1f4cd31c51468250dff1cf86be0572e2d2 https://api.semanticscholar.org/c397fdc468eb878db233dc80bf65a35a5b87a80e https://api.semanticscholar.org/06bcf2b799586d98001db1b5803f03c00ce03b52 https://api.semanticscholar.org/78dd13bb4ae85690f66c67ab8abfd020ddc8ffe3 https://api.semanticscholar.org/b3bbf0efbd272831e89fbb09634636c70bcf2dfa https://api.semanticscholar.org/29bdd183402a94e3cca7531ced412bb427e9285a https://api.semanticscholar.org/cfc4cd3f84969d59b0c853b29ffd34374932ee4f https://www.semanticscholar.org/paper/204e3073870fae3d05bcbc2f6a8e263d9b72e776 https://www.semanticscholar.org/paper/20cc4bfdb648fd7947c71252589fc867d4d16933 https://www.semanticscholar.org/paper/23ffaa0fe06eae05817f527a47ac3291077f9e58 https://www.semanticscholar.org/paper/1822ca8db58b0382b0c64f310840f0f875ea02c0 https://api.semanticscholar.org/b842cd963a5e61c48b467c92aee035485256ee12 https://api.semanticscholar.org/d0611891b9e8a7c5731146097b6f201578f47b2f