Waveform-based location methods become more and more popular for analysis of both natural and induced seismicity . We utilize stochastic optimization algorithms to speed up microseismic location . Two waveform-based location methods ( i.e. diffraction stacking and cross correlation stacking ) are adopted to test the performance of three algorithms ( i.e. particle swarm optimization , differential evolution , and neighbourhood algorithm ) . The uncertainties of the plug-in time and state of charge of the battery of the electric vehicle are modeled using a Markovian process and a Monte-Carlo simulation . The results showed that the proposed stochastic optimization model can be solved using dynamic programming and deployed as a continuous optimal control within SOFW . The system will be deployed shortly in Italy within one use case of the Storage 4 Grid ( S4 G ) project . Governments borrow funds to finance the excess of cash payments or interest payments over receipts , usually by issuing fixed income debt and index-linked debt . The goal of this work is to propose a stochastic optimization-based approach to determine the composition of the portfolio issued over a series of government auctions for the fixed income debt , to minimize the cost of servicing debt while controlling risk and maintaining market liquidity . We show that this debt issuance problem can be modeled as a mixed integer linear programming problem with a receding horizon . The problem is exacerbated by uncertainties in demand and availability of resources . In this paper , we present a stochastic optimization based framework for robust decision making in the selection of distributed resources over a planning horizon under demand uncertainty . We present a novel two-stage stochastic programming model for resource selection , and implement an L-shaped decomposition algorithm to solve this model . A linear feedback control law and a quadratic cost function are used , so that the results are comparable with the classical linear quadratic regulator approach . The same problem has been solved with two other stochastic based optimization algorithms , namely a Genetic Algorithm and a Differential Evolution and the results are used for comparison . The numerical simulation shows that sufficient vibration suppression can be achieved by means of this method . The BP ( back-propagation algorithm ) and GA ( genetic algorithm ) are among the most effective algorithms of neural networks ( NN ) . As a deterministic gradient-descent algorithm and a stochastic optimizing algorithm respectively , there exits great compensability between their advantages and disadvantages . The proposed hybrid BP/GA learning method for a multilayer feedforward net blends the merits of both the BP and GA . The system power consumption , which includes the local execution power and the offloading transmission power , is designated as the main system performance index . First , we formulate the power consumption minimization problem with the battery queue stability and QoS constraints as a stochastic optimization programming , which is difficult to solve due to the time-coupling constraints . Then , we adopt the Lyapunov optimization approach to tackle the problem by reformulating it into a problem with relaxed queue stability constraints . In particular it is shown that broadening less than ~2.5:1 results in reduced beam efficiency for both linear and circular arrays , while a broadening of greater than ~2.5:1 results in improved beam efficiency , and that , therefore , certain broadening configurations will be preferable to the radar designer ( e.g. 4:1 in one plane rather than 2:1 in two planes , due to a large difference in efficiency ) . The algorithms used to achieve the broadening patterns presented here are based on a combination of homotopy from known optimality with a stochastic gradient descent approach using a carefully constructed one parameter family of penalty functions With this in mind we propose a class of objective functions which primarily represent a smooth surrogate for the real AUC , and in a special case we show how to prioritise the top of the list . This loss is differentiable and is optimised through a carefully designed stochastic gradient-descent-based algorithm which scales linearly with the size of the data . We mitigate sample bias present in the data by sampling observations according to a certain power-law based distribution . The generalization performance of their algorithm is sensitive to the number of parties and the relative fractions of data owned by the different parties . In this paper , we describe a new differentially private algorithm for the multiparty setting that uses a stochastic gradient descent based procedure to directly optimize the overall multiparty objective rather than combining classifiers learned from optimizing local objectives . The algorithm achieves a slightly weaker form of differential privacy than that of [ 7 ] , but provides improved generalization guarantees that do not depend on the number of parties or the relative sizes of the individual data sets . In our approach , we use Quantum annealers for sampling Boltzmann states . These states are used to approximate gradients in a stochastic gradient descent scheme . The approach is used to demonstrate logic circuits in the discriminative sense and a specialized two-phase distribution using generative BM . In this study , we investigate expected CP decomposition— a special case of CP decomposition in which a tensor to be decomposed is given as the sum or average of tensor samples Χ(t ) for t = 1 , . . . , T. To determine this decomposition , we develope stochastic-gradient-descent-type algorithms with four appealing features : efficient memory use , ability to work in an online setting , robustness of parameter tuning , and simplicity . Our theoretical analysis show that the solutions do not diverge to infinity for any initial value or step size . To apply this technique to large data sets , parallel algorithms are needed . In this paper , we create a parallel version of a BPDN algorithm [ 11 ] that can compute exact solutions for an interval of values for the ℓ1 penalty parameter that exploits the sparsity of the solutions , unlike stochastic gradient descent-type algorithms . It is found that the spike timings of neurons in an SNN can be exploited to estimate the gradients that are associated with each synapse . With the proposed method of estimating gradients , learning similar to the stochastic gradient descent process employed in a conventional artificial neural network ( ANN ) can be achieved . In addition to the conventional layer-by-layer backpropagation , a one-pass direct backpropagation is possible using the proposed learning algorithm . In order to predict user preferences in the collaborative filtering way , the work reconstructs user sparse ratings with autoencoders to estimate unobserved user preferences . To further protect user privacy , the Gaussian mechanism is combined in the stochastic gradient descent process to ensure that the training process meets the requirements of approximate differential privacy . However , the convergence of the EM algorithm to a global maximum is conditioned on proper parameter initialization , a large enough training sample set , and several iterations over this training set . In this work , a Sample Iterative Likelihood Maximization ( SILM ) algorithm based on a stochastic descent gradient method is proposed . Simulation results showed that our algorithm can attain high log-likelihoods with fewer iterations in comparison to the EM algorithm . One remarkable characteristic of this multimodal representation is that it connects textual and visual content allowing to solve queries with only visual content by implicitly completing the missing textual content . Another important characteristic of the method is that the multimodal representation is learned online using an efficient stochastic gradient descent formulation . Experiments were conducted in a dataset of 5,000 images to evaluate the convergence speed and search performance . Backpropagation involves computing the gradient of a loss function , with respect to the weights of the network , to update the weights and thus minimize loss . Although the mean square error is often used as a loss function , the general stochastic gradient descent principle does not immediately connect with a specific loss function . Another drawback of backpropagation has been the search for optimal values of two important training parameters , learning rate and momentum weight , which are determined empirically in most systems . For linear systems , I propose modifications to deterministic methods to allow the use of random samples and maintain the stochastic convergence , which is particularly challenging when the unknown system is singular or nearly singular . For variational inequalities and optimization problems , I propose a class of methods that combine elements of incremental constraint projection , stochastic gradient/subgradient descent , and proximal algorithm . These methods can be applied with various sampling schemes that are suitable for applications involving distributed implementation , large data set , or online learning . Initial learning rate was set to 1 , and a power decay was applied . We used stochastic gradient descent with a batch size of 50 . subsection : Performance section : Training The neural language model is trained to minimize the negative log - likelihood of a training sentence s with stochastic gradient descent ( SGD ; [ reference ] : section : Experiments subsection : Training We used the stochastic gradient descent ( SGD ) algorithm . Adadelta was used to automatically adapt the learning rate of each parameter ( and ) . The algorithm is schematically illustrated in the first layer of Figure [ reference ] . The network is trained by calculating the gradient of this loss and using a stochastic gradient descent algorithm . Stochastic back - propagation through a sampling function is done as in variational auto - encoders . But , in order to reduce parameters , ReLU is used after each convolution layer in our very deep network that has got a satisfactory result for comparison . We use the adaptive moment estimation ( Adam ) to optimize all network parameters instead of the commonly used stochastic gradient descent one . section : EXPERIMENTAL RESULTS There is almost certainly a different network for each presentation of each training case but all of these networks share the same weights for the hidden units that are present . We use the standard , stochastic gradient descent procedure for training the dropout neural networks on mini - batches of training cases , but we modify the penalty term that is normally used to prevent the weights from growing too large . Instead of penalizing the squared length ( L2 norm ) of the whole weight vector , we set an upper bound on the L2 norm of the incoming weight vector for each individual hidden unit . while Note : In this pseudo - code , e ( y ) refers to a " one - hot " vector , consisting of all 0s except for a 1 at position y , and is the element - wise product . We propose to tackle this problem with a simple stochastic gradient procedure , in which updates are made in the opposite direction of the gradient of Equation ( 9 ) for the minimizing parameters , and in the direction of the gradient for the maximizing parameters . Stochastic estimates of the gradient are made , using a subset of the training samples to compute the averages . Due to its efficiency and ease to implement , stochastic gradient descent ( SGD ) has been widely used in machine learning . In particular , SGD is one of the most popular optimization methods for distributed learning . Recently , quantized SGD ( QSGD ) , which adopts quantization to reduce the communication cost in SGD-based distributed learning , has attracted much attention . In this paper , we propose a differentially private algorithm for the adaptation of the learning rate for differentially private stochastic gradient descent ( SGD ) that avoids the need for validation set use . The idea for the adaptiveness comes from the technique of extrapolation in classical numerical analysis : to get an estimate for the error against the gradient flow which underlies SGD , we compare the result obtained by one full step and two half-steps . We prove the privacy of the method using the moments accountant mechanism . We also develop a variance-reduction variant of the algorithm that converges linearly . Numerical experiments suggest that our methods are faster than existing ones such as proximal SGD , SVRG and SAGA on high-dimensional problems . Conventionally , DNNs are trained by using mini-batch stochastic gradient descent ( SGD ) which is stable but prone to be trapped into local optimum . A recent work based on Nesterov ’s accelerated gradient descent ( NAG ) algorithm is developed by merging the current momentum information into correction of SGD updating . NAG less likely jumps into local minimum so that convergence rate is improved . A possible mitigation of such events is to slow down the learning process . This paper presents a novel approach to control the SGD learning rate , that uses two statistical tests . The first one , aimed at fast learning , compares the momentum of the normalized gradient vectors to that of random unit vectors and accordingly gracefully increases or decreases the learning rate . We consider three specific research questions : ( 1 ) what is the relationship between the Hessian of the loss and the second moment of stochastic gradients ( SGs ) ? ( 2 ) how can we characterize the stochastic optimization dynamics of SGD with fixed and adaptive step sizes and diagonal pre-conditioning based on the first and second moments of SGs ? and ( 3 ) how can we characterize a scale-invariant generalization bound of deep nets based on the Hessian of the loss , which by itself is not scale invariant ? For $ \ell_2 $ regression , pwSGD returns an approximate solution with $ \epsilon$ relative error in the objective value and the solution vector measured in prediction norm in $ \mathcal{O}(\log n \cdot \text{nnz}(A ) + \text{poly}(d ) \log(1/\epsilon ) /\epsilon)$ time . We also provide lower bounds on the coreset complexity for more general regression problems , indicating that still new ideas will be needed to extend similar RLA preconditioning ideas to weighted SGD algorithms for more general regression problems . Finally , the effectiveness of such algorithms is illustrated numerically on both synthetic and real datasets . Different classifiers ( i.e. multilayer perceptron ( MLP ) , stochastic gradient descent ( SGD ) , logistic regression and radial basis function ) have been developed and the effects of feature selection algorithms ( i.e. Chi2 , support vector machine-recursive feature elimination ( SVM-RFE ) , minimum redundancy maximum relevance and ReliefF ) for cyberbullying detection have also been investigated . The experimental results of the study proved that SGD and MLP classifiers with 500 selected features using SVM-RFE algorithm showed the best results ( F_measure value is more than 0.930 ) by means of classification time and accuracy . In this paper , we make further contributions along the line . First , we develop an epoch-projection SGD method that only makes a constant number of projections less than $ \log_2T$ but achieves an optimal convergence rate $ O(1/T)$ for { \it strongly convex optimization}. Second , we present a proximal extension to utilize the structure of the objective function that could further speed-up the computation and convergence for sparse regularized loss minimization problems . Finally , we consider an application of the proposed techniques to solving the high dimensional large margin nearest neighbor classification problem , yielding a speed-up of orders of magnitude . Originality . The study of the parameters of the neural network on the basis of the calculation of the average harmonic with different activation functions ( Linear , Sigmoid , Tanh , Softplus , ReLU , L-ReLU ) on training samples of different volumes ( 140 , 1400 , 14000 , 49000 examples ) and various training algorithms Neural Network ( BGD , MB SGD , Adam , Adamax , Nadam ) . Practical value . In these algorithms , people use a constant to indicate the communication period for model/gradient exchange . We find that this type of communication pattern could incur unnecessary and inefficient data transmission for some training methods e.g. , elastic SGD and gossiping SGD . In this paper , we propose an adaptive communication method to improve the performance of gossiping SGD . Unlike past work , we use two identically structured attention layers , one to compute the standard generation probabilities ( gen or p gen ) , and one to produce the scores used in copy or p copy . We train the generation models using SGD and truncated BPTT [ reference ][ reference ] , as in language modeling . That is , we split each y 1:T into contiguous blocks of length 100 , and backprop both the gradients with respect to the current block as well as with respect to the encoder parameters for each block . We use a step width of 3.125 · 10 −4 and early stopping based on validation perplexity ( Kingma and Ba , 2014 ) . For non - recurrent encoders , we obtain best results with stochastic gradient descent ( SGD ) and annealing : we use a learning rate of 0.1 and once the validation perplexity stops improving , we reduce the learning rate by an order of magnitude each epoch until it falls below 10 −4 . For all models , we use mini - batches of 32 sentences for IWSLT'14 and 64 for WMT . and are the training set and the validation set respectively . is the set of parameters including weights and biases of our network MCL , which is updated using Mini - Batch Stochastic Gradient Descent ( SGD ) at each iteration . The face alignment loss is defined as where is the weight of the - th landmark , and denote the ground - truth x - coordinate and y - coordinate of the - th landmark respectively , and is the ground truth inter - ocular distance between the eye centers . [ reference ] for example again . In an SGD iteration , the backward gradients are : where denotes the derivative of to its input . When effective depth , both terms in Eqn . Bootstrapping , and its variants in the literature , rely on the aforementioned alternation template : ( a ) for some period of time a fixed model is used to find new examples to add to the active training set ; ( b ) then , for some period of time the model is trained on the fixed active training set . Training deep ConvNet detectors with SGD typically requires hundreds of thousands of SGD steps and freezing the model for even a few iterations at a time would dramatically slow progress . What is needed , instead , is a purely online form of hard example selection . The algorithm is a simple modification to SGD in which training examples are sampled according to a non - uniform , non - stationary distribution that depends on the current loss of each example under consideration . The method takes advantage of detection - specific problem structure in which each SGD mini - batch consists of only one or two images , but thousands of candidate examples . The candidate examples are subsampled according to a distribution that favors diverse , high loss instances . Our main observation is that these alternating steps can be combined with how FRCN is trained using online SGD . The key is that although each SGD iteration samples only a small number of images , each image contains thousands of example RoIs from which we can select the hard examples rather than a heuristically sampled subset . This strategy fits the alternation template to SGD by “ freezing ” the model for only one mini - batch . First , large - scale person search datasets would have a large number of identities ( more than in our training set ) , while each identity only has several instances and each image only contains a few identities . We need to learn more than discriminant functions simultaneously , but during each SGD iteration we only have positive samples from tens of classes . The classifier matrix suffers from large variance of gradients and thus can not be learned effectively , even with proper pre - training and high momentum . Mini - batch sampling . During fine - tuning , each SGD mini - batch is constructed from N = 2 images , chosen uniformly at random ( as is common practice , we actually iterate over permutations of the dataset ) . We use mini - batches of size R = 128 , sampling 64 RoIs from each image . In the RoI generating layer , each location in a heat map generates 7 boxes with 7 different aspect ratios for KITTI and 5 aspect ratios for PASCAL , where each number indicates the ratio between the height and the width of the bounding box . In training the RPN , each SGD mini - batch is constructed from a single image , chosen uniformly at random . A mini - batch has size 128 , with 64 positive RoIs and 64 negative RoIs , where the IoU threshold is 70 % for both KITTI and PASCAL . All bi - LSTM models were implemented in CNN / pycnn , a flexible neural network library . For all models we use the same hyperparameters , which were set on English dev , i.e. , SGD training with cross - entropy loss , no mini - batches , 20 epochs , default learning rate ( 0.1 ) , 128 dimensions for word embeddings , 100 for character and byte embeddings , 100 hidden states and Gaussian noise with = 0.2 . As training is stochastic in nature , we use a fixed seed throughout . These scores are added together and normalized for the final pixel - wise class probabilities . For training , we use mini - batch SGD with momentum . Our model is trained for 30k SGD iterations using a mini - batch of 6 images , momentum of 0.9 , an initialize learning rate ( LR ) of and ” polynomial ” learning rate policy . subsubsection : Optimization strategy The system might be trained by means of an off - the - shelf mini - batch Stochastic Gradient Descent ( SGD ) method with adaptive learning rates such as variance - based SGD ( vSGD ) . Algorithm [ reference ] details the latter training process . The input image is resized such that its shorter side has 800 pixels . We adopt synchronized SGD training on 8 GPUs . A mini - batch involves 2 images per GPU and 256 anchors per image . The acoustic training data is comprised by LDC corpora 97S62 , 2004S13 , 2005S13 , 2004S11 and 2004S09 ; see for a full description . subsection : 1 - bit SGD Training All presented models are costly to train . Its performance depends crucially on a damping coefficient $ \beta$. Large $ \beta$ values can potentially deliver much larger speedups , but are prone to oscillations and instability ; hence one typically resorts to small values such as 0.5 or 0.9 . We propose Aggregated Momentum ( AggMo ) , a variant of momentum which combines multiple velocity vectors with different $ \beta$ parameters . AggMo is trivial to implement , but significantly dampens oscillations , enabling it to remain stable even for aggressive $ \beta$ values such as 0.999 . We reinterpret Nesterov 's accelerated gradient descent as a special case of AggMo and provide theoretical convergence bounds for online convex optimization . Empirically , we find that AggMo is a suitable drop-in replacement for other momentum methods , and frequently delivers faster convergence . We describe numerous connections to and identities with other algorithms , and we characterize the set of two-state optimization algorithms that QHM can recover . Finally , we propose a QH variant of Adam called QHAdam , and we empirically demonstrate that our algorithms lead to significantly improved training in a variety of settings , including a new state-of-the-art result on WMT16 EN-DE . We hope that these empirical results , combined with the conceptual and practical simplicity of QHM and QHAdam , will spur interest from both practitioners and researchers . Code is immediately available .
Waveform-based location methods become more and more popular for analysis of both natural and induced seismicity . We utilize stochastic optimization algorithms to speed up microseismic location . Two waveform-based location methods ( i.e. diffraction stacking and cross correlation stacking ) are adopted to test the performance of three algorithms ( i.e. particle swarm optimization , differential evolution , and neighbourhood algorithm ) .
The uncertainties of the plug-in time and state of charge of the battery of the electric vehicle are modeled using a Markovian process and a Monte-Carlo simulation . The results showed that the proposed stochastic optimization model can be solved using dynamic programming and deployed as a continuous optimal control within SOFW . The system will be deployed shortly in Italy within one use case of the Storage 4 Grid ( S4 G ) project .
Governments borrow funds to finance the excess of cash payments or interest payments over receipts , usually by issuing fixed income debt and index-linked debt . The goal of this work is to propose a stochastic optimization-based approach to determine the composition of the portfolio issued over a series of government auctions for the fixed income debt , to minimize the cost of servicing debt while controlling risk and maintaining market liquidity . We show that this debt issuance problem can be modeled as a mixed integer linear programming problem with a receding horizon .
The problem is exacerbated by uncertainties in demand and availability of resources . In this paper , we present a stochastic optimization based framework for robust decision making in the selection of distributed resources over a planning horizon under demand uncertainty . We present a novel two-stage stochastic programming model for resource selection , and implement an L-shaped decomposition algorithm to solve this model .
A linear feedback control law and a quadratic cost function are used , so that the results are comparable with the classical linear quadratic regulator approach . The same problem has been solved with two other stochastic based optimization algorithms , namely a Genetic Algorithm and a Differential Evolution and the results are used for comparison . The numerical simulation shows that sufficient vibration suppression can be achieved by means of this method .
The BP ( back-propagation algorithm ) and GA ( genetic algorithm ) are among the most effective algorithms of neural networks ( NN ) . As a deterministic gradient-descent algorithm and a stochastic optimizing algorithm respectively , there exits great compensability between their advantages and disadvantages . The proposed hybrid BP/GA learning method for a multilayer feedforward net blends the merits of both the BP and GA .
The system power consumption , which includes the local execution power and the offloading transmission power , is designated as the main system performance index . First , we formulate the power consumption minimization problem with the battery queue stability and QoS constraints as a stochastic optimization programming , which is difficult to solve due to the time-coupling constraints . Then , we adopt the Lyapunov optimization approach to tackle the problem by reformulating it into a problem with relaxed queue stability constraints .
In particular it is shown that broadening less than ~2.5:1 results in reduced beam efficiency for both linear and circular arrays , while a broadening of greater than ~2.5:1 results in improved beam efficiency , and that , therefore , certain broadening configurations will be preferable to the radar designer ( e.g. 4:1 in one plane rather than 2:1 in two planes , due to a large difference in efficiency ) . The algorithms used to achieve the broadening patterns presented here are based on a combination of homotopy from known optimality with a stochastic gradient descent approach using a carefully constructed one parameter family of penalty functions
With this in mind we propose a class of objective functions which primarily represent a smooth surrogate for the real AUC , and in a special case we show how to prioritise the top of the list . This loss is differentiable and is optimised through a carefully designed stochastic gradient-descent-based algorithm which scales linearly with the size of the data . We mitigate sample bias present in the data by sampling observations according to a certain power-law based distribution .
The generalization performance of their algorithm is sensitive to the number of parties and the relative fractions of data owned by the different parties . In this paper , we describe a new differentially private algorithm for the multiparty setting that uses a stochastic gradient descent based procedure to directly optimize the overall multiparty objective rather than combining classifiers learned from optimizing local objectives . The algorithm achieves a slightly weaker form of differential privacy than that of [ 7 ] , but provides improved generalization guarantees that do not depend on the number of parties or the relative sizes of the individual data sets .
In our approach , we use Quantum annealers for sampling Boltzmann states . These states are used to approximate gradients in a stochastic gradient descent scheme . The approach is used to demonstrate logic circuits in the discriminative sense and a specialized two-phase distribution using generative BM .
In this study , we investigate expected CP decomposition— a special case of CP decomposition in which a tensor to be decomposed is given as the sum or average of tensor samples Χ(t ) for t = 1 , . . . , T. To determine this decomposition , we develope stochastic-gradient-descent-type algorithms with four appealing features : efficient memory use , ability to work in an online setting , robustness of parameter tuning , and simplicity . Our theoretical analysis show that the solutions do not diverge to infinity for any initial value or step size .
To apply this technique to large data sets , parallel algorithms are needed . In this paper , we create a parallel version of a BPDN algorithm [ 11 ] that can compute exact solutions for an interval of values for the ℓ1 penalty parameter that exploits the sparsity of the solutions , unlike stochastic gradient descent-type algorithms .
It is found that the spike timings of neurons in an SNN can be exploited to estimate the gradients that are associated with each synapse . With the proposed method of estimating gradients , learning similar to the stochastic gradient descent process employed in a conventional artificial neural network ( ANN ) can be achieved . In addition to the conventional layer-by-layer backpropagation , a one-pass direct backpropagation is possible using the proposed learning algorithm .
In order to predict user preferences in the collaborative filtering way , the work reconstructs user sparse ratings with autoencoders to estimate unobserved user preferences . To further protect user privacy , the Gaussian mechanism is combined in the stochastic gradient descent process to ensure that the training process meets the requirements of approximate differential privacy .
However , the convergence of the EM algorithm to a global maximum is conditioned on proper parameter initialization , a large enough training sample set , and several iterations over this training set . In this work , a Sample Iterative Likelihood Maximization ( SILM ) algorithm based on a stochastic descent gradient method is proposed . Simulation results showed that our algorithm can attain high log-likelihoods with fewer iterations in comparison to the EM algorithm .
One remarkable characteristic of this multimodal representation is that it connects textual and visual content allowing to solve queries with only visual content by implicitly completing the missing textual content . Another important characteristic of the method is that the multimodal representation is learned online using an efficient stochastic gradient descent formulation . Experiments were conducted in a dataset of 5,000 images to evaluate the convergence speed and search performance .
Backpropagation involves computing the gradient of a loss function , with respect to the weights of the network , to update the weights and thus minimize loss . Although the mean square error is often used as a loss function , the general stochastic gradient descent principle does not immediately connect with a specific loss function . Another drawback of backpropagation has been the search for optimal values of two important training parameters , learning rate and momentum weight , which are determined empirically in most systems .
For linear systems , I propose modifications to deterministic methods to allow the use of random samples and maintain the stochastic convergence , which is particularly challenging when the unknown system is singular or nearly singular . For variational inequalities and optimization problems , I propose a class of methods that combine elements of incremental constraint projection , stochastic gradient/subgradient descent , and proximal algorithm . These methods can be applied with various sampling schemes that are suitable for applications involving distributed implementation , large data set , or online learning .
Initial learning rate was set to 1 , and a power decay was applied . We used stochastic gradient descent with a batch size of 50 . subsection : Performance
section : Training The neural language model is trained to minimize the negative log - likelihood of a training sentence s with stochastic gradient descent ( SGD ; [ reference ] : section : Experiments
subsection : Training We used the stochastic gradient descent ( SGD ) algorithm . Adadelta was used to automatically adapt the learning rate of each parameter ( and ) .
The algorithm is schematically illustrated in the first layer of Figure [ reference ] . The network is trained by calculating the gradient of this loss and using a stochastic gradient descent algorithm . Stochastic back - propagation through a sampling function is done as in variational auto - encoders .
But , in order to reduce parameters , ReLU is used after each convolution layer in our very deep network that has got a satisfactory result for comparison . We use the adaptive moment estimation ( Adam ) to optimize all network parameters instead of the commonly used stochastic gradient descent one . section : EXPERIMENTAL RESULTS
There is almost certainly a different network for each presentation of each training case but all of these networks share the same weights for the hidden units that are present . We use the standard , stochastic gradient descent procedure for training the dropout neural networks on mini - batches of training cases , but we modify the penalty term that is normally used to prevent the weights from growing too large . Instead of penalizing the squared length ( L2 norm ) of the whole weight vector , we set an upper bound on the L2 norm of the incoming weight vector for each individual hidden unit .
while Note : In this pseudo - code , e ( y ) refers to a " one - hot " vector , consisting of all 0s except for a 1 at position y , and is the element - wise product . We propose to tackle this problem with a simple stochastic gradient procedure , in which updates are made in the opposite direction of the gradient of Equation ( 9 ) for the minimizing parameters , and in the direction of the gradient for the maximizing parameters . Stochastic estimates of the gradient are made , using a subset of the training samples to compute the averages .
Due to its efficiency and ease to implement , stochastic gradient descent ( SGD ) has been widely used in machine learning . In particular , SGD is one of the most popular optimization methods for distributed learning . Recently , quantized SGD ( QSGD ) , which adopts quantization to reduce the communication cost in SGD-based distributed learning , has attracted much attention .
In this paper , we propose a differentially private algorithm for the adaptation of the learning rate for differentially private stochastic gradient descent ( SGD ) that avoids the need for validation set use . The idea for the adaptiveness comes from the technique of extrapolation in classical numerical analysis : to get an estimate for the error against the gradient flow which underlies SGD , we compare the result obtained by one full step and two half-steps . We prove the privacy of the method using the moments accountant mechanism .
We also develop a variance-reduction variant of the algorithm that converges linearly . Numerical experiments suggest that our methods are faster than existing ones such as proximal SGD , SVRG and SAGA on high-dimensional problems .
Conventionally , DNNs are trained by using mini-batch stochastic gradient descent ( SGD ) which is stable but prone to be trapped into local optimum . A recent work based on Nesterov ’s accelerated gradient descent ( NAG ) algorithm is developed by merging the current momentum information into correction of SGD updating . NAG less likely jumps into local minimum so that convergence rate is improved .
A possible mitigation of such events is to slow down the learning process . This paper presents a novel approach to control the SGD learning rate , that uses two statistical tests . The first one , aimed at fast learning , compares the momentum of the normalized gradient vectors to that of random unit vectors and accordingly gracefully increases or decreases the learning rate .
We consider three specific research questions : ( 1 ) what is the relationship between the Hessian of the loss and the second moment of stochastic gradients ( SGs ) ? ( 2 ) how can we characterize the stochastic optimization dynamics of SGD with fixed and adaptive step sizes and diagonal pre-conditioning based on the first and second moments of SGs ? and ( 3 ) how can we characterize a scale-invariant generalization bound of deep nets based on the Hessian of the loss , which by itself is not scale invariant ?
For $ \ell_2 $ regression , pwSGD returns an approximate solution with $ \epsilon$ relative error in the objective value and the solution vector measured in prediction norm in $ \mathcal{O}(\log n \cdot \text{nnz}(A ) + \text{poly}(d ) \log(1/\epsilon ) /\epsilon)$ time . We also provide lower bounds on the coreset complexity for more general regression problems , indicating that still new ideas will be needed to extend similar RLA preconditioning ideas to weighted SGD algorithms for more general regression problems . Finally , the effectiveness of such algorithms is illustrated numerically on both synthetic and real datasets .
Different classifiers ( i.e. multilayer perceptron ( MLP ) , stochastic gradient descent ( SGD ) , logistic regression and radial basis function ) have been developed and the effects of feature selection algorithms ( i.e. Chi2 , support vector machine-recursive feature elimination ( SVM-RFE ) , minimum redundancy maximum relevance and ReliefF ) for cyberbullying detection have also been investigated . The experimental results of the study proved that SGD and MLP classifiers with 500 selected features using SVM-RFE algorithm showed the best results ( F_measure value is more than 0.930 ) by means of classification time and accuracy .
In this paper , we make further contributions along the line . First , we develop an epoch-projection SGD method that only makes a constant number of projections less than $ \log_2T$ but achieves an optimal convergence rate $ O(1/T)$ for { \it strongly convex optimization}. Second , we present a proximal extension to utilize the structure of the objective function that could further speed-up the computation and convergence for sparse regularized loss minimization problems . Finally , we consider an application of the proposed techniques to solving the high dimensional large margin nearest neighbor classification problem , yielding a speed-up of orders of magnitude .
Originality . The study of the parameters of the neural network on the basis of the calculation of the average harmonic with different activation functions ( Linear , Sigmoid , Tanh , Softplus , ReLU , L-ReLU ) on training samples of different volumes ( 140 , 1400 , 14000 , 49000 examples ) and various training algorithms Neural Network ( BGD , MB SGD , Adam , Adamax , Nadam ) . Practical value .
In these algorithms , people use a constant to indicate the communication period for model/gradient exchange . We find that this type of communication pattern could incur unnecessary and inefficient data transmission for some training methods e.g. , elastic SGD and gossiping SGD . In this paper , we propose an adaptive communication method to improve the performance of gossiping SGD .
Unlike past work , we use two identically structured attention layers , one to compute the standard generation probabilities ( gen or p gen ) , and one to produce the scores used in copy or p copy . We train the generation models using SGD and truncated BPTT [ reference ][ reference ] , as in language modeling . That is , we split each y 1:T into contiguous blocks of length 100 , and backprop both the gradients with respect to the current block as well as with respect to the encoder parameters for each block .
We use a step width of 3.125 · 10 −4 and early stopping based on validation perplexity ( Kingma and Ba , 2014 ) . For non - recurrent encoders , we obtain best results with stochastic gradient descent ( SGD ) and annealing : we use a learning rate of 0.1 and once the validation perplexity stops improving , we reduce the learning rate by an order of magnitude each epoch until it falls below 10 −4 . For all models , we use mini - batches of 32 sentences for IWSLT'14 and 64 for WMT .
and are the training set and the validation set respectively . is the set of parameters including weights and biases of our network MCL , which is updated using Mini - Batch Stochastic Gradient Descent ( SGD ) at each iteration . The face alignment loss is defined as where is the weight of the - th landmark , and denote the ground - truth x - coordinate and y - coordinate of the - th landmark respectively , and is the ground truth inter - ocular distance between the eye centers .
[ reference ] for example again . In an SGD iteration , the backward gradients are : where denotes the derivative of to its input . When effective depth , both terms in Eqn .
Bootstrapping , and its variants in the literature , rely on the aforementioned alternation template : ( a ) for some period of time a fixed model is used to find new examples to add to the active training set ; ( b ) then , for some period of time the model is trained on the fixed active training set . Training deep ConvNet detectors with SGD typically requires hundreds of thousands of SGD steps and freezing the model for even a few iterations at a time would dramatically slow progress . What is needed , instead , is a purely online form of hard example selection .
The algorithm is a simple modification to SGD in which training examples are sampled according to a non - uniform , non - stationary distribution that depends on the current loss of each example under consideration . The method takes advantage of detection - specific problem structure in which each SGD mini - batch consists of only one or two images , but thousands of candidate examples . The candidate examples are subsampled according to a distribution that favors diverse , high loss instances .
Our main observation is that these alternating steps can be combined with how FRCN is trained using online SGD . The key is that although each SGD iteration samples only a small number of images , each image contains thousands of example RoIs from which we can select the hard examples rather than a heuristically sampled subset . This strategy fits the alternation template to SGD by “ freezing ” the model for only one mini - batch .
First , large - scale person search datasets would have a large number of identities ( more than in our training set ) , while each identity only has several instances and each image only contains a few identities . We need to learn more than discriminant functions simultaneously , but during each SGD iteration we only have positive samples from tens of classes . The classifier matrix suffers from large variance of gradients and thus can not be learned effectively , even with proper pre - training and high momentum .
Mini - batch sampling . During fine - tuning , each SGD mini - batch is constructed from N = 2 images , chosen uniformly at random ( as is common practice , we actually iterate over permutations of the dataset ) . We use mini - batches of size R = 128 , sampling 64 RoIs from each image .
In the RoI generating layer , each location in a heat map generates 7 boxes with 7 different aspect ratios for KITTI and 5 aspect ratios for PASCAL , where each number indicates the ratio between the height and the width of the bounding box . In training the RPN , each SGD mini - batch is constructed from a single image , chosen uniformly at random . A mini - batch has size 128 , with 64 positive RoIs and 64 negative RoIs , where the IoU threshold is 70 % for both KITTI and PASCAL .
All bi - LSTM models were implemented in CNN / pycnn , a flexible neural network library . For all models we use the same hyperparameters , which were set on English dev , i.e. , SGD training with cross - entropy loss , no mini - batches , 20 epochs , default learning rate ( 0.1 ) , 128 dimensions for word embeddings , 100 for character and byte embeddings , 100 hidden states and Gaussian noise with = 0.2 . As training is stochastic in nature , we use a fixed seed throughout .
These scores are added together and normalized for the final pixel - wise class probabilities . For training , we use mini - batch SGD with momentum . Our model is trained for 30k SGD iterations using a mini - batch of 6 images , momentum of 0.9 , an initialize learning rate ( LR ) of and ” polynomial ” learning rate policy .
subsubsection : Optimization strategy The system might be trained by means of an off - the - shelf mini - batch Stochastic Gradient Descent ( SGD ) method with adaptive learning rates such as variance - based SGD ( vSGD ) . Algorithm [ reference ] details the latter training process .
The input image is resized such that its shorter side has 800 pixels . We adopt synchronized SGD training on 8 GPUs . A mini - batch involves 2 images per GPU and 256 anchors per image .
The acoustic training data is comprised by LDC corpora 97S62 , 2004S13 , 2005S13 , 2004S11 and 2004S09 ; see for a full description . subsection : 1 - bit SGD Training All presented models are costly to train .
Its performance depends crucially on a damping coefficient $ \beta$. Large $ \beta$ values can potentially deliver much larger speedups , but are prone to oscillations and instability ; hence one typically resorts to small values such as 0.5 or 0.9 . We propose Aggregated Momentum ( AggMo ) , a variant of momentum which combines multiple velocity vectors with different $ \beta$ parameters . AggMo is trivial to implement , but significantly dampens oscillations , enabling it to remain stable even for aggressive $ \beta$ values such as 0.999 . We reinterpret Nesterov 's accelerated gradient descent as a special case of AggMo and provide theoretical convergence bounds for online convex optimization . Empirically , we find that AggMo is a suitable drop-in replacement for other momentum methods , and frequently delivers faster convergence .
We describe numerous connections to and identities with other algorithms , and we characterize the set of two-state optimization algorithms that QHM can recover . Finally , we propose a QH variant of Adam called QHAdam , and we empirically demonstrate that our algorithms lead to significantly improved training in a variety of settings , including a new state-of-the-art result on WMT16 EN-DE . We hope that these empirical results , combined with the conceptual and practical simplicity of QHM and QHAdam , will spur interest from both practitioners and researchers . Code is immediately available .
Waveform-based microseismic location using stochastic optimization algorithms: A parameter tuning workflow Stochastic optimization framework for online scheduling of an EV charging station in a residential place with photovoltaics and energy storage system A mixed integer linear programming model for optimal sovereign debt issuance Stochastic Programming Approach for Resource Selection Under Demand Uncertainty Vibration control of beams with piezoelectric sensors and actuators using particle swarm optimization Hybrid back-propagation/genetic algorithm for multilayer feedforward neural networks Distributed Energy Management for Multiuser Mobile-Edge Computing Systems With Energy Harvesting Devices and QoS Constraints Extreme Beam Broadening using Phase Only Pattern Synthesis Collaborative Filtering with Localised Ranking A Differentially Private Stochastic Gradient Descent Algorithm for Multiparty Classification Machine learning in quantum computers via general Boltzmann Machines: Generative and Discriminative training through annealing Expected Tensor Decomposition with Stochastic Gradient Descent Parallelizing Basis Pursuit Denoising Online Supervised Learning for Hardware-Based Multilayer Spiking Neural Networks Through the Modulation of Weight-Dependent Spike-Timing-Dependent Plasticity Differentially Private Recommender System with Autoencoders Sample iterative likelihood maximization for speaker verification systems Online Matrix Factorization for Multimodal Image Retrieval The Golden Ratio of Learning and Momentum Stochastic methods for large-scale linear problems, variational inequalities, and convex optimization Natural Language Inference by Tree-Based Convolution and Heuristic Matching Neural Text Generation from Structured Data with Application to the Biography Domain Neural Machine Translation by Jointly Learning to Align and Translate Towards Conceptual Compression Single image super-resolution with dilated convolution based multi-scale information learning inception module Improving neural networks by preventing co-adaptation of feature detectors Domain-Adversarial Training of Neural Networks Quantized Epoch-SGD for Communication-Efficient Distributed Learning Learning rate adaptation for differentially private stochastic gradient descent Stochastic Primal-Dual Method for Empirical Risk Minimization with O(1) Per-Iteration Complexity Hybrid Accelerated Optimization for Speech Recognition Stochastic Gradient Descent: Going As Fast As Possible But Not Faster Hessian based analysis of SGD for Deep Nets: Dynamics and Generalization Weighted SGD for $\ell_p$ Regression with Randomized Preconditioning AUTOMATIC DETECTION OF CYBERBULLYING IN FORMSPRING.ME, MYSPACE AND YOUTUBE SOCIAL NETWORKS Efficient Stochastic Gradient Descent for Strongly Convex Optimization OPTIMAL ROUTE DEFINITION IN THE NETWORK BASED ON THE MULTILAYER NEURAL MODEL Adaptive Communication for Distributed Deep Learning on Commodity GPU Cluster Challenges in Data-to-Document Generation A Convolutional Encoder Model for Neural Machine Translation Deep Multi-Center Learning for Face Alignment Wider or Deeper: Revisiting the ResNet Model for Visual Recognition Training Region-Based Object Detectors with Online Hard Example Mining Joint Detection and Identification Feature Learning for Person Search Fast R-CNN Subcategory-Aware Convolutional Neural Networks for Object Proposals and Detection Multilingual Part-of-Speech Tagging with Bidirectional Long Short-Term Memory Models and Auxiliary Loss Revisiting Unreasonable Effectiveness of Data in Deep Learning Era No more meta-parameter tuning in unsupervised sparse feature learning Feature Pyramid Networks for Object Detection The microsoft 2016 conversational speech recognition system Aggregated Momentum: Stability Through Passive Damping Quasi-hyperbolic momentum and Adam for deep learning
436e2f918eb95c4d4293670f12d509a27ca46e25 abf2633276bed1c946d9f92d1749cdae85a78d57 1bd84fd676de05fb7793e602ab085a0317a1c1ce c2b3eed05d3bd936bcd52dbb9f90ead708d11c39 e90ad935ed08604e43976a6c088c60842fa51ca7 f8280a167e0643edf696cee7363688e6df98edba 02fd9efc086e3711ac47994ee2c729f2eb2c38a1 7a66bacf2a4446bad1fc6d93fdddd0d83220e910 586918721e3f0c968531d44154ececf5229e5e68 de2c619af6388f5d07ce87f1a3a960e0d807ee41 729c7e67156ed65155ffedab40863de9d4be52ca aeb5ec712b8874cd908f8d9fac52b17acac9ba22 a8b0de2765e9f74571e57d651dacd92fb8b9bca8 298d0d9323023c91f6ca379c09a7fdcc7f46264f 9442e0858ffac9cad5e015a1191182bbe1d88d83 5f32b67910a51cd5d166f15705485f452f7a58d5 2385ac6d414fb81524b8ebef4da156b2d9eb7baf 6bbb7e025c9ca1c85ed6aa5da289b313ce82d3aa 5216a84904e7cacd7b42f1d4e5562a500e4348a2 a5c620e6f7a96083f7975271a0c70c717d9f5140 767c70abd453a6f2ba854ce5f293291d0854f88f 3bb0fbe66c51fcf48aa02a5175c3d11bd41b3ee8 a1fb43ffead2252b4d4766a0e500ee53baf6ee46 c730bec01ec8124c16e0aa96bf0038a36612c6e5 fd96765ab009b1c408e3afcbf6d806af831c6b6b 0b5af8d20009f30ec66088e2cd090531a62a7239 69b77e32f95c1b9f1676e4d61869931e15e705a0 5966ee80c468f8285089cf33d12c3225ff098cb3 eee012a4d52810ca607a40ca7bdf033e45e8bb7f 639fadecf2079a76332cbf9bdf157871c4d89c7d 1de35bfced22b716db622d8946a66ac9c5d39690 8c62519ba7567b0c0016f9d49eefa2584e5df2cb
["Computer Science"] ["Computer Science"] ["Computer Science","Economics","Chemistry"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Mathematics","Physics","Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science","Medicine"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science","Mathematics"] ["Computer Science"] ["Mathematics","Computer Science"] ["Mathematics","Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science","Mathematics"] ["Mathematics","Computer Science"] ["Computer Science","Mathematics"] ["Computer Science"] ["Mathematics","Computer Science"] ["Computer Science"] ["Computer Science"] ["Mathematics","Computer Science"] ["Mathematics","Computer Science"]
2019 2019 2018 2011 2000 2019 2006 2015 2012 2020 2016 2019 2018 2019 2010 2012 2020 2013 2016 2016 2015 2016 2017 2012 2016 2019 2018 2018 2016 2017 2019 2019 2013 2019 2018 2017 2017 2020 2019 2016 2017 2015 2017 2016 2017 2014 2017 2017 2018 2018
JSSPP AAAI AISTATS AAAI IJCNN iThings/GreenCom/CPSCom/SmartData EUSIPCO CIARP ACL EMNLP ICLR NIPS 2017 IEEE International Conference on Image Processing (ICIP) ArXiv J. Mach. Learn. Res. NeurIPS INTERSPEECH CCGrid EMNLP ACL Neurocomputing Pattern Recognit. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2015 IEEE International Conference on Computer Vision (ICCV) 2017 IEEE Winter Conference on Applications of Computer Vision (WACV) ACL 2017 IEEE International Conference on Computer Vision (ICCV) ArXiv 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
https://api.semanticscholar.org/436e2f918eb95c4d4293670f12d509a27ca46e25 https://api.semanticscholar.org/abf2633276bed1c946d9f92d1749cdae85a78d57 https://api.semanticscholar.org/1bd84fd676de05fb7793e602ab085a0317a1c1ce https://api.semanticscholar.org/c2b3eed05d3bd936bcd52dbb9f90ead708d11c39 https://api.semanticscholar.org/e90ad935ed08604e43976a6c088c60842fa51ca7 https://api.semanticscholar.org/f8280a167e0643edf696cee7363688e6df98edba https://api.semanticscholar.org/02fd9efc086e3711ac47994ee2c729f2eb2c38a1 https://api.semanticscholar.org/7a66bacf2a4446bad1fc6d93fdddd0d83220e910 https://api.semanticscholar.org/586918721e3f0c968531d44154ececf5229e5e68 https://api.semanticscholar.org/de2c619af6388f5d07ce87f1a3a960e0d807ee41 https://api.semanticscholar.org/729c7e67156ed65155ffedab40863de9d4be52ca https://api.semanticscholar.org/aeb5ec712b8874cd908f8d9fac52b17acac9ba22 https://api.semanticscholar.org/a8b0de2765e9f74571e57d651dacd92fb8b9bca8 https://api.semanticscholar.org/298d0d9323023c91f6ca379c09a7fdcc7f46264f https://api.semanticscholar.org/9442e0858ffac9cad5e015a1191182bbe1d88d83 https://api.semanticscholar.org/5f32b67910a51cd5d166f15705485f452f7a58d5 https://api.semanticscholar.org/2385ac6d414fb81524b8ebef4da156b2d9eb7baf https://api.semanticscholar.org/6bbb7e025c9ca1c85ed6aa5da289b313ce82d3aa https://api.semanticscholar.org/5216a84904e7cacd7b42f1d4e5562a500e4348a2 https://www.semanticscholar.org/paper/ea407573bfcd39f9a478fe33cf6ce0ee1780a5f0 https://www.semanticscholar.org/paper/604764133befe7a0aaa692919545846197e6e065 https://www.semanticscholar.org/paper/fa72afa9b2cbc8f0d7b05d52548906610ffbb9c5 https://www.semanticscholar.org/paper/bda89e0d181eda7e49ea831225eda86d075e111c https://www.semanticscholar.org/paper/dd88e202b12a1650822c4086d091f882b582c7a4 https://www.semanticscholar.org/paper/1366de5bb112746a555e9c0cd00de3ad8628aea8 https://www.semanticscholar.org/paper/1d5972b32a9b5a455a6eef389de5b7fca25771ad https://api.semanticscholar.org/a5c620e6f7a96083f7975271a0c70c717d9f5140 https://api.semanticscholar.org/767c70abd453a6f2ba854ce5f293291d0854f88f https://api.semanticscholar.org/3bb0fbe66c51fcf48aa02a5175c3d11bd41b3ee8 https://api.semanticscholar.org/a1fb43ffead2252b4d4766a0e500ee53baf6ee46 https://api.semanticscholar.org/c730bec01ec8124c16e0aa96bf0038a36612c6e5 https://api.semanticscholar.org/fd96765ab009b1c408e3afcbf6d806af831c6b6b https://api.semanticscholar.org/0b5af8d20009f30ec66088e2cd090531a62a7239 https://api.semanticscholar.org/69b77e32f95c1b9f1676e4d61869931e15e705a0 https://api.semanticscholar.org/5966ee80c468f8285089cf33d12c3225ff098cb3 https://api.semanticscholar.org/eee012a4d52810ca607a40ca7bdf033e45e8bb7f https://api.semanticscholar.org/639fadecf2079a76332cbf9bdf157871c4d89c7d https://www.semanticscholar.org/paper/13395213d47f78672ab4e81573f2b0fa0cfc8c6d https://www.semanticscholar.org/paper/f958d4921951e394057a1c4ec33bad9a34e5dad1 https://www.semanticscholar.org/paper/2d294bde112b892068636f3a48300b3c033d98da https://www.semanticscholar.org/paper/3070a1bd503c3767def898bbd50c7eea2bbf29c9 https://www.semanticscholar.org/paper/63333669bcf694aba2e1928f6060ab1d6a5161fe https://www.semanticscholar.org/paper/fb4b700ba023c08e64c13f8030f40dcc901ac518 https://www.semanticscholar.org/paper/7ffdbc358b63378f07311e883dddacc9faeeaf4b https://www.semanticscholar.org/paper/f082dbb4ec8eb4d6d6d42a50e0cdb5a1c0d65a6e https://www.semanticscholar.org/paper/acd87e4f672f0b92ea4164414c213560c23bee52 https://www.semanticscholar.org/paper/8760bc7631c0cb04e7138254e9fd6451b7def8ca https://www.semanticscholar.org/paper/097851f362667a599208f4a06fa289dac2e91fbe https://www.semanticscholar.org/paper/b9b4e05faa194e5022edd9eb9dd07e3d675c2b36 https://www.semanticscholar.org/paper/ac94ef90be9b0c3bf744d6744e47b38855f9a4c7 https://api.semanticscholar.org/1de35bfced22b716db622d8946a66ac9c5d39690 https://api.semanticscholar.org/8c62519ba7567b0c0016f9d49eefa2584e5df2cb