According to one embodiment of the present disclosure, provided is a learning system that updates a parameter for a neural network, the learning system including: a plurality of differential value calculators; and a parameter update module.
Please help me write a proper abstract based on the patent claims. CLAIM: 1. A learning system that updates a parameter for a neural network, the learning system comprising: a plurality of differential value calculators; and a parameter update module, wherein each of the differential value calculators, which are not synchronized with one another, receives a parameter of a certain time from the parameter update module, calculates a differential value to be used for updating the parameter, based on the received parameter, and transmits the differential value to the parameter update module, the parameter update module receives the differential value from the differential value calculator, updates the parameter based on the received differential value, not in synchronization with differential value calculation by the plurality of differential value calculators, and transmits the updated parameter to the plurality of differential value calculators, and, when calculating the differential value, the differential value calculator takes staleness into account, the staleness corresponding to a number of times the parameter is updated during a period between a time when the parameter is received and a time when the calculated differential value is used by the parameter update module in updating the parameter. 2. The learning system according to claim 1, wherein, taking the staleness into account, the differential value calculator calculates a predicted value of a parameter of a time when the calculated differential value is used by the parameter update module in updating the parameter, and calculates the differential value by differentiating the predicted value. 3. The learning system according to claim 2, wherein the differential value calculator calculates the predicted value, based on the parameter of the certain time and a parameter of an earlier time than the certain time, and the staleness. 4. The learning system according to claim 3, wherein, when the parameter of the certain time τ is represented by W(τ), the parameter of the earlier time (τ−1) than the certain time is represented by W(τ−1), the staleness is represented by sn, and a coefficient γε(0,1), the predicted value P(τ) is expressed by the equation: [ Mathematical   Formula   1 ] P ( τ ) = W ( τ ) + ∑ u = 1 s n + 1   γ u  ( W ( τ ) - W ( τ - 1 ) ) 5. The learning system according to claim 4, wherein the differential value calculator gradually increases the coefficient γ to a large value, as learning progresses. 6. The learning system according to claim 1, wherein the parameter update module updates the parameter using a value obtained by multiplying the differential value by a learning coefficient, and increases the learning coefficient to a large value as learning progresses. 7. The learning system according to claim 1, wherein the parameter update module adjusts an update amount of the parameter, to prevent an absolute value or a norm of the update amount of the parameter from exceeding a first value until the parameter is updated a first number of times since a start of learning. 8. A learning method to update a parameter for a neural network, the learning method comprising: each of a plurality of differential value calculators, which are not synchronized with one another, receiving a parameter of a certain time from a parameter update module, calculating a differential value to be used for updating the parameter, based on the received parameter, transmitting the differential value to the parameter update module; the parameter update module receiving the differential value from the differential calculators; updating the parameter based on the received differential value, not in synchronization with differential value calculation by the plurality of differential value calculators; and transmitting the updated parameter to the plurality of differential value calculators, wherein upon calculating the differential value by the differential value calculator, staleness is taken into account by the differential value calculator, the staleness corresponding to a number of times the parameter is updated during a period between a time when the parameter is received and a time when the calculated differential value is used by the parameter update module in updating the parameter.