In particular , the UAV-IR is used to intelligently reflect mmW beamforming signals from a base station towards a mobile outdoor user , while harvesting energy from mmW signals to power the IR . To maintain a line-of-sight ( LOS ) channel , a reinforcement learning ( RL ) approach , based on Q- learning and neural networks , is proposed to model the propagation environment , such that the location and reflection coefficient of the UAV-IR can be optimized to maximize the downlink transmission capacity . Simulation results show a significant advantage for using a UAV-IR over a static IR , in terms of the average data rate and the achievable downlink LOS probability . The QDSEGA is one of the methods that we have proposed for them . The QDSEGA is realized by combining Q-learning and GA , and it can acquire suitable behaviors by adapting a movement of a robot for a task . In this paper , we focus on the adaptability of the QDSEGA and discuss the robustness of the autonomous redundant robot that is controlled by the QDSEGA . Therefore algorithms have been created to compress the visual information to abstract representations ( Visual Features ) . Neural Q-learning is such a method . It combines the RL algorithm of Q-learning with Artificial Neural Networks ( ANNs ) . Distributed visual navigation based on neural Q-learning for a mobile robot is studied in this paper . First , a general distributed structure based on the multiple processors for visual navigation is established according to the decomposition of the mobile robot visual navigation task . After restarting , units continue to learn via back-propagation . The resulting restart algorithm is tested in a Q-learning network that learns to solve an inverted pendulum problem . Solutions are found faster on average with the restart algorithm than without it . π Our model - based exploration bonuses can be incorporated into any online reinforcement learning algorithm that updates the policy based on state , action , reward tuples of the form , such as Q - learning or actor - critic algorithms . Our method is summarized in Algorithm [ reference ] . document : Deep Reinforcement Learning with Double Q - learning The popular Q - learning algorithm is known to overestimate action values under certain conditions . It was not previously known whether , in practice , such overestimations are common , whether they harm performance , and whether they can generally be prevented . This policy acts greedily with respect to with probability and uniformly at random otherwise . DQN uses a convolutional neural network to parameterize and the Q - learning algorithm to achieve human - level play on the Atari - 57 benchmark . subsection : Distributional RL The coverage penalty is taken into account for sample diversity . With comparison to deep Q network ( DQN ) and prioritized experience replay ( PER ) methods , the DCRL algorithm is evaluated on Atari 2600 games , and the experimental results show that DCRL outperforms DQN and PER on most of these games . More results further show that the proposed curriculum training paradigm of DCRL is also applicable and effective for other memory-based deep reinforcement learning approaches , such as double DQN and dueling network . Then , the weighted sum of the delay and energy-efficient minimization computation offloading problem was formulated , constrained to the maximum latency and server resources . After that , a DQN algorithm based on reinforcement learning is proposed . In order to avoid the problem of excessive state space and overestimation , a DDQN offloading algorithm is proposed . Under the influence of the above navigation model , the multiple geomagnetic component gradually approaches the target value with the movement of the aircraft , thereby achieving the navigation purpose . By comparing with the evolutionary algorithm ( EA ) , it is proved that the DQN algorithm have better convergence ability . In order to do so , we have collected a dataset of game frames that we used to train our agent on . We present different approaches and settings for DQN agent . We evaluate our agent using results of the previous participants of AIBirds competition , results of volunteer human players and present the results of AIBirds 2018 competition . Among the many variants of RL , an important class of problems is where the state and action spaces are continuous — autonomous robots , autonomous vehicles , optimal control are all examples of such problems that can lend themselves naturally to reinforcement based algorithms , and have continuous state and action spaces . In this paper , we introduce a prioritized form of a combination of state-of-the-art approaches such as Deep Q-learning ( DQN ) and Deep Deterministic Policy Gradient ( DDPG ) to outperform the earlier results for continuous state and action space problems . Our experiments also involve the use of parameter noise during training resulting in more robust deep RL models outperforming the earlier results significantly . Learning from the historical experience , a Deep Q-Network ( DQN ) based approach is proposed to generate transmission decisions at the transmitter . In addition , a variant of DQN , termed adaptive DQN , is introduced to cope with the change of jamming conditions . The simulation results demonstrate that the proposed scheme can learn an optimal policy to guide the transmitter to avoid jamming more quickly and efficiently than a Q-learning baseline . Moreover , the effectiveness and robustness of the adaptive DQN is also numerically verified . We test our methods in two path following tasks — straight line and sinusoids curve following of AUV by simulating in the Gazebo platform . Our experimental results show that with our proposed deep interactive RL method , AUV can converge faster than a DQN learner from only environmental reward . Moreover , AUV learning with our deep RL from both human and environmental rewards can also achieve a similar or even better performance than that with deep interactive RL and can adapt to the actual environment by further learning from environmental rewards . However , the traditional Deep Q-Networks ( DQN ) algorithm has the problem of overestimation of state-action values and high approximation error variance of the target values . To fit the motion coordination problem scenario , we further propose a new algorithm Multi-Loss Double DQN ( MLDDQN ) based on the Double DQN ( DDQN ) , which minimizes the weighted sum of loss of different target networks we have learned before . We evaluate the performance of our algorithm through the different motion coordination tasks . Abstract Prediction of ground responses is important for improving performance of tunneling . This study proposes a novel reinforcement learning ( RL ) based optimizer with the integration of deep-Q network ( DQN ) and particle swarm optimization ( PSO ) . Such optimizer is used to improve the extreme learning machine ( ELM ) based tunneling-induced settlement prediction model . The proposed APA is proven to be convergent even with a more aggressive learning rate , making the DRL agent have a faster learning speed . Furthermore , to combine the accelerated algorithm with deep Q-network ( DQN ) , Double DQN and deep deterministic policy gradient ( DDPG ) , we proposed three novel DRL algorithms : APA-DQN , APA-Double DQN , and APA-DDPG , which demonstrates the adaptability of the accelerated algorithm with DRL algorithms . We have tested the proposed algorithms on both discrete-action and continuous-action tasks . Figure [ reference ] depicts the result of our experiment , averaged across 5 trials . Although optimistic initialization helps in Freeway , it otherwise yields performance similar to DQN . By contrast , the count - based exploration bonus enables us to make quick progress on a number of games , most dramatically in Montezuma ’s Revenge and Venture . An overview of the neural network we used to learn the agent ’s value function while also learning the feature representation and the SR is depicted in Figure [ reference ] . The layers used to compute the state - action value function , , are structured as in DQN Mnih15 , but with different numbers of parameters ( i .. e , filter sizes , stride , and number of nodes ) . The ALE is a particularly interesting testbed in our context , because the difficulty of exploration varies greatly between games . Random strategies often work well , and it is in these games that Deep Q - Networks ( DQN ) with - greedy is able to achieve so - called human - level performance . In others , however , DQN with - greedy does not improve upon a random policy , and its inability to explore efficiently is one of the key determinants of this failure . Table [ reference ] compares our evaluation scores to Double DQN ( DDQN ) , Double DQN with pseudocount ( DDQN - PC ) , A3C + , DQN Pop - Art ( DQN - PA ) , Dueling Network ( Dueling ) , Gorila , DQN with Model Prediction Exploration Bonuses ( MP - EB ) , Trust Region Policy Optimisation ( TRPO ) , and TRPO - AE - SimHash ( TRPO - Hash ) . The most interesting comparisons for our purposes are with TRPO - Hash , DDQN - PC , A3C + , and MP - EB , because these algorithms all use exploration strategies that drive the agent to reduce its uncertainty . Previous work has tackled this task using Q - learning with epsilon - greedy exploration , as well as Monte Carlo tree search and policy gradient methods . We use Deep Q Networks ( DQN ) as the reinforcement learning algorithm within our method , and compare its performance to the same DQN method using only epsilon - greedy exploration , Boltzman exploration , and a Thompson sampling approach . The results for 14 games in the Arcade Learning Environment are presented in Table 1 . subsubsection : Retrace ( ) Retrace ( ) is a convergent off - policy multi - step algorithm extending the DQN agent munos2016safe . Assume that some trajectory has been generated according to behaviour policy , i.e. , . dabney2017qr propose an alternate pair of choices , parameterizing the distribution by a uniform mixture of Diracs whose locations are adjusted using quantile regression . Their algorithm , QR - DQN , while restricted to a discrete set of quantiles , automatically adapts return quantiles to minimize the Wasserstein distance between the Bellman updated and current return distributions . This flexibility allows QR - DQN to significantly improve on C51 ’s Atari - 57 performance . We now introduce the implicit quantile network ( IQN ) , a deterministic parametric function trained to reparameterize samples from a base distribution , e.g. , to the respective quantile values of a target distribution . IQN provides an effective way to learn an implicit representation of the return distribution , yielding a powerful function approximator for a new DQN - like agent . Let be the quantile function at for the random variable . subsection : Implementation Consider the neural network structure used by the DQN agent . Let be the function computed by the convolutional layers and the subsequent fully - connected layers mapping to the estimated action - values , such that . The idea of Double Q - learning is to reduce overestimations by decomposing the max operation in the target into action selection and action evaluation . Although not fully decoupled , the target network in the DQN architecture provides a natural candidate for the second value function , without having to introduce additional networks . We therefore propose to evaluate the greedy policy according to the online network , but using the target network to estimate its value . Some tuning is appropriate because the hyperparameters were tuned for DQN , which is a different algorithm . For the tuned version of Double DQN , we increased the number of frames between each two copies of the target network from 10 , 000 to 30 , 000 , to reduce overestimations further because immediately after each switch DQN and Double DQN both revert to Q - learning . In addition , we reduced the exploration during learning from to , and then used during evaluation . We applied Gorila DQN to 49 games on the Atari 2600 platform . We outperformed single GPU DQN on 41 games and outperformed human professional on 25 games . Gorila DQN also trained much faster than the non - distributed version in terms of wall - time , reaching the performance of single GPU DQN roughly ten times faster for most games . The current parameters may be updated many times per time - step , and are copied into the old parameters after iterations . At every update iteration the current parameters are updated so as to minimise the mean - squared Bellman error with respect to old parameters , by optimizing the following loss function ( DQN Loss ) , For each update , a tuple of experience ( or a minibatch of such samples ) is sampled uniformly from the replay memory . In detail , our traffic prediction method relies on online supervised learning adopting Recurrent Neural Networks ( RNNs ) that can accurately capture traffic statistics over consecutive frames , and the access control configuration can use either a non-ML based controller or a cooperatively trained Deep Reinforcement Learning ( DRL ) based controller depending on the complexity of different random access schemes . Numerical results show that the proposed two-step cooperative learning optimizer considerably outperforms the conventional Deep Q-Network ( DQN ) in terms of higher training efficiency and better access performance . As a key enzyme in the life-cycle of coronavirus , the 3C-like main protease ( 3CLpro or Mpro ) is the most attractive for antiviral drug design . Based on a recently solved structure ( PDB ID : 6LU7 ) , we developed a novel advanced deep Q-learning network with the fragment-based drug design ( ADQN-FBDD ) for generating potential lead compounds targeting SARS-CoV-2 3CLpro . We obtained a series of derivatives from those lead compounds by our structure-based optimization policy ( SBOP ) . In recent years , deep reinforcement learning has been widely used in games , robots , autonomous driving and other fields , which proves that deep reinforcement learning is powerful in decision making and problem solving . In this paper , according to deep reinforcement learning and the characteristic of thermal process control , a thermal process control method based on DQN ( Deep Q-learning Network ) is proposed . Through matlab and python mixed programming , the process of DQN controller training and the simulation of control system are completed . Each node decides its transmitting/receiving mode and the transmission power based on its local observation of the system state . We formulate a Markov game and develop a multi-agent deep reinforcement learning algorithm based on deep recurrent Q-network . The simulation results show that the proposed approach outperforms the baselines significantly . We formulate our optimization problem as a resource allocation as well as user association problem . We propose a deep Q-Network based algorithm , namely Delay Minimizing Deep Q-Network ( DM-DQN ) to address the low-latency requirement . DM-DQN aims at reducing the delay of CUDs by balancing the trade-off between allocating more RBs and associating devices to base stations with high channel quality . In this paper , we propose a deep Q-network based learning approach for connectivity preservation problem . We adopt a fully connected neural network with nonlinearities as the Q-function in the deep Q-network based framework . Further , we design and implement a simulation environment for connectivity preservation , which provides states and reward feedbacks to deep Q-network . Because of the tremendous research that focuses on human brains and reinforcement learning , scientists have investigated how robots can autonomously tackle complex tasks in the form of making a self-driving agent control in a human-like way . In this study , we propose an end-to-end architecture using novel deep-Q-network architecture in conjunction with a recurrence to resolve the problem in the field of simulated self-driving . The main contribution of this study is that we trained the driving agent using a brain-inspired trial-and-error technique , which was in line with the real world situation . To make our study realistic , the actual climatic data from each district in Thailand together with the actual rice yields were used to construct the yield estimator where the decision tree with Adaboost was employed . Next , we constructed the simulated farming environments for different districts in Thailand and employed the deep Q-learning neural network to find the optimum actions . From our simulated examples , the deep Q-learning may allow farmers to earn more than the average income . This architecture uses four main components : parallel actors that generate new behaviour ; parallel learners that are trained from stored experience ; a distributed neural network to represent the value function or behaviour policy ; and a distributed store of experience . We used our architecture to implement the Deep Q - Network algorithm ( DQN ) . Our distributed algorithm was applied to 49 games from Atari 2600 games from the Arcade Learning Environment , using identical hyperparameters . However , it is well - known that the Q - learning algorithm is highly unstable when combined with non - linear function approximators such as deep neural networks . subsection : Deep Q - Networks Recently , a new RL algorithm has been developed which is in practice much more stable when combined with deep Q - networks . This is complementary to distributing gradient computation , and the two approaches can be combined , but in this work we focus purely on data - generation . We use this distributed architecture to scale up variants of Deep Q - Networks ( DQN ) and Deep Deterministic Policy Gradient ( DDPG ) , and we evaluate these on the Arcade Learning Environment benchmark bellemare2013arcade , and on a range of continuous control tasks . Our architecture achieves a new state of the art performance on Atari games , using a fraction of the wall - clock time compared to the previous state of the art , and without per - game hyperparameter tuning . subsection : Value - based algorithms The Deep Q - Network ( DQN ) framework , introduced by mnih15human , popularised the current line of research into deep reinforcement learning by reaching human - level , and beyond , performance across 57 Atari 2600 games in the ALE . While DQN includes many specific components , the essence of the framework , much of which is shared by Neural Fitted Q - Learning riedmiller2005neural , is to use of a deep convolutional neural network to approximate an action - value function , training this approximate action - value function using the Q - Learning algorithm watkins1992 and mini - batches of one - step transitions ( ) drawn randomly from an experience replay buffer lin1992self . In this paper , we propose a deep reinforcement learning ( DRL ) framework to improve the selection of macro actions . Our framework is based on the combination of the Ape-X DQN and the Long-Short-Term-Memory ( LSTM ) . We use this framework to build our bot , named as LastOrder . In addition , it provides support for multiplayer and multi-agent experiments . We propose three full-game scenarios of varying difficulty with the Football Benchmarks and report baseline results for three commonly used reinforcement algorithms ( IMPALA , PPO , and Ape-X DQN ) . We also provide a diverse set of simpler scenarios with the Football Academy and showcase several promising research directions .
In particular , the UAV-IR is used to intelligently reflect mmW beamforming signals from a base station towards a mobile outdoor user , while harvesting energy from mmW signals to power the IR . To maintain a line-of-sight ( LOS ) channel , a reinforcement learning ( RL ) approach , based on Q- learning and neural networks , is proposed to model the propagation environment , such that the location and reflection coefficient of the UAV-IR can be optimized to maximize the downlink transmission capacity . Simulation results show a significant advantage for using a UAV-IR over a static IR , in terms of the average data rate and the achievable downlink LOS probability .
The QDSEGA is one of the methods that we have proposed for them . The QDSEGA is realized by combining Q-learning and GA , and it can acquire suitable behaviors by adapting a movement of a robot for a task . In this paper , we focus on the adaptability of the QDSEGA and discuss the robustness of the autonomous redundant robot that is controlled by the QDSEGA .
Therefore algorithms have been created to compress the visual information to abstract representations ( Visual Features ) . Neural Q-learning is such a method . It combines the RL algorithm of Q-learning with Artificial Neural Networks ( ANNs ) .
Distributed visual navigation based on neural Q-learning for a mobile robot is studied in this paper . First , a general distributed structure based on the multiple processors for visual navigation is established according to the decomposition of the mobile robot visual navigation task .
After restarting , units continue to learn via back-propagation . The resulting restart algorithm is tested in a Q-learning network that learns to solve an inverted pendulum problem . Solutions are found faster on average with the restart algorithm than without it .
π Our model - based exploration bonuses can be incorporated into any online reinforcement learning algorithm that updates the policy based on state , action , reward tuples of the form , such as Q - learning or actor - critic algorithms . Our method is summarized in Algorithm [ reference ] .
document : Deep Reinforcement Learning with Double Q - learning The popular Q - learning algorithm is known to overestimate action values under certain conditions . It was not previously known whether , in practice , such overestimations are common , whether they harm performance , and whether they can generally be prevented .
This policy acts greedily with respect to with probability and uniformly at random otherwise . DQN uses a convolutional neural network to parameterize and the Q - learning algorithm to achieve human - level play on the Atari - 57 benchmark . subsection : Distributional RL
The coverage penalty is taken into account for sample diversity . With comparison to deep Q network ( DQN ) and prioritized experience replay ( PER ) methods , the DCRL algorithm is evaluated on Atari 2600 games , and the experimental results show that DCRL outperforms DQN and PER on most of these games . More results further show that the proposed curriculum training paradigm of DCRL is also applicable and effective for other memory-based deep reinforcement learning approaches , such as double DQN and dueling network .
Then , the weighted sum of the delay and energy-efficient minimization computation offloading problem was formulated , constrained to the maximum latency and server resources . After that , a DQN algorithm based on reinforcement learning is proposed . In order to avoid the problem of excessive state space and overestimation , a DDQN offloading algorithm is proposed .
Under the influence of the above navigation model , the multiple geomagnetic component gradually approaches the target value with the movement of the aircraft , thereby achieving the navigation purpose . By comparing with the evolutionary algorithm ( EA ) , it is proved that the DQN algorithm have better convergence ability .
In order to do so , we have collected a dataset of game frames that we used to train our agent on . We present different approaches and settings for DQN agent . We evaluate our agent using results of the previous participants of AIBirds competition , results of volunteer human players and present the results of AIBirds 2018 competition .
Among the many variants of RL , an important class of problems is where the state and action spaces are continuous — autonomous robots , autonomous vehicles , optimal control are all examples of such problems that can lend themselves naturally to reinforcement based algorithms , and have continuous state and action spaces . In this paper , we introduce a prioritized form of a combination of state-of-the-art approaches such as Deep Q-learning ( DQN ) and Deep Deterministic Policy Gradient ( DDPG ) to outperform the earlier results for continuous state and action space problems . Our experiments also involve the use of parameter noise during training resulting in more robust deep RL models outperforming the earlier results significantly .
Learning from the historical experience , a Deep Q-Network ( DQN ) based approach is proposed to generate transmission decisions at the transmitter . In addition , a variant of DQN , termed adaptive DQN , is introduced to cope with the change of jamming conditions . The simulation results demonstrate that the proposed scheme can learn an optimal policy to guide the transmitter to avoid jamming more quickly and efficiently than a Q-learning baseline . Moreover , the effectiveness and robustness of the adaptive DQN is also numerically verified .
We test our methods in two path following tasks — straight line and sinusoids curve following of AUV by simulating in the Gazebo platform . Our experimental results show that with our proposed deep interactive RL method , AUV can converge faster than a DQN learner from only environmental reward . Moreover , AUV learning with our deep RL from both human and environmental rewards can also achieve a similar or even better performance than that with deep interactive RL and can adapt to the actual environment by further learning from environmental rewards .
However , the traditional Deep Q-Networks ( DQN ) algorithm has the problem of overestimation of state-action values and high approximation error variance of the target values . To fit the motion coordination problem scenario , we further propose a new algorithm Multi-Loss Double DQN ( MLDDQN ) based on the Double DQN ( DDQN ) , which minimizes the weighted sum of loss of different target networks we have learned before . We evaluate the performance of our algorithm through the different motion coordination tasks .
Abstract Prediction of ground responses is important for improving performance of tunneling . This study proposes a novel reinforcement learning ( RL ) based optimizer with the integration of deep-Q network ( DQN ) and particle swarm optimization ( PSO ) . Such optimizer is used to improve the extreme learning machine ( ELM ) based tunneling-induced settlement prediction model .
The proposed APA is proven to be convergent even with a more aggressive learning rate , making the DRL agent have a faster learning speed . Furthermore , to combine the accelerated algorithm with deep Q-network ( DQN ) , Double DQN and deep deterministic policy gradient ( DDPG ) , we proposed three novel DRL algorithms : APA-DQN , APA-Double DQN , and APA-DDPG , which demonstrates the adaptability of the accelerated algorithm with DRL algorithms . We have tested the proposed algorithms on both discrete-action and continuous-action tasks .
Figure [ reference ] depicts the result of our experiment , averaged across 5 trials . Although optimistic initialization helps in Freeway , it otherwise yields performance similar to DQN . By contrast , the count - based exploration bonus enables us to make quick progress on a number of games , most dramatically in Montezuma ’s Revenge and Venture .
An overview of the neural network we used to learn the agent ’s value function while also learning the feature representation and the SR is depicted in Figure [ reference ] . The layers used to compute the state - action value function , , are structured as in DQN Mnih15 , but with different numbers of parameters ( i .. e , filter sizes , stride , and number of nodes ) .
The ALE is a particularly interesting testbed in our context , because the difficulty of exploration varies greatly between games . Random strategies often work well , and it is in these games that Deep Q - Networks ( DQN ) with - greedy is able to achieve so - called human - level performance . In others , however , DQN with - greedy does not improve upon a random policy , and its inability to explore efficiently is one of the key determinants of this failure .
Table [ reference ] compares our evaluation scores to Double DQN ( DDQN ) , Double DQN with pseudocount ( DDQN - PC ) , A3C + , DQN Pop - Art ( DQN - PA ) , Dueling Network ( Dueling ) , Gorila , DQN with Model Prediction Exploration Bonuses ( MP - EB ) , Trust Region Policy Optimisation ( TRPO ) , and TRPO - AE - SimHash ( TRPO - Hash ) . The most interesting comparisons for our purposes are with TRPO - Hash , DDQN - PC , A3C + , and MP - EB , because these algorithms all use exploration strategies that drive the agent to reduce its uncertainty .
Previous work has tackled this task using Q - learning with epsilon - greedy exploration , as well as Monte Carlo tree search and policy gradient methods . We use Deep Q Networks ( DQN ) as the reinforcement learning algorithm within our method , and compare its performance to the same DQN method using only epsilon - greedy exploration , Boltzman exploration , and a Thompson sampling approach . The results for 14 games in the Arcade Learning Environment are presented in Table 1 .
subsubsection : Retrace ( ) Retrace ( ) is a convergent off - policy multi - step algorithm extending the DQN agent munos2016safe . Assume that some trajectory has been generated according to behaviour policy , i.e. , .
dabney2017qr propose an alternate pair of choices , parameterizing the distribution by a uniform mixture of Diracs whose locations are adjusted using quantile regression . Their algorithm , QR - DQN , while restricted to a discrete set of quantiles , automatically adapts return quantiles to minimize the Wasserstein distance between the Bellman updated and current return distributions . This flexibility allows QR - DQN to significantly improve on C51 ’s Atari - 57 performance .
We now introduce the implicit quantile network ( IQN ) , a deterministic parametric function trained to reparameterize samples from a base distribution , e.g. , to the respective quantile values of a target distribution . IQN provides an effective way to learn an implicit representation of the return distribution , yielding a powerful function approximator for a new DQN - like agent . Let be the quantile function at for the random variable .
subsection : Implementation Consider the neural network structure used by the DQN agent . Let be the function computed by the convolutional layers and the subsequent fully - connected layers mapping to the estimated action - values , such that .
The idea of Double Q - learning is to reduce overestimations by decomposing the max operation in the target into action selection and action evaluation . Although not fully decoupled , the target network in the DQN architecture provides a natural candidate for the second value function , without having to introduce additional networks . We therefore propose to evaluate the greedy policy according to the online network , but using the target network to estimate its value .
Some tuning is appropriate because the hyperparameters were tuned for DQN , which is a different algorithm . For the tuned version of Double DQN , we increased the number of frames between each two copies of the target network from 10 , 000 to 30 , 000 , to reduce overestimations further because immediately after each switch DQN and Double DQN both revert to Q - learning . In addition , we reduced the exploration during learning from to , and then used during evaluation .
We applied Gorila DQN to 49 games on the Atari 2600 platform . We outperformed single GPU DQN on 41 games and outperformed human professional on 25 games . Gorila DQN also trained much faster than the non - distributed version in terms of wall - time , reaching the performance of single GPU DQN roughly ten times faster for most games .
The current parameters may be updated many times per time - step , and are copied into the old parameters after iterations . At every update iteration the current parameters are updated so as to minimise the mean - squared Bellman error with respect to old parameters , by optimizing the following loss function ( DQN Loss ) , For each update , a tuple of experience ( or a minibatch of such samples ) is sampled uniformly from the replay memory .
In detail , our traffic prediction method relies on online supervised learning adopting Recurrent Neural Networks ( RNNs ) that can accurately capture traffic statistics over consecutive frames , and the access control configuration can use either a non-ML based controller or a cooperatively trained Deep Reinforcement Learning ( DRL ) based controller depending on the complexity of different random access schemes . Numerical results show that the proposed two-step cooperative learning optimizer considerably outperforms the conventional Deep Q-Network ( DQN ) in terms of higher training efficiency and better access performance .
As a key enzyme in the life-cycle of coronavirus , the 3C-like main protease ( 3CLpro or Mpro ) is the most attractive for antiviral drug design . Based on a recently solved structure ( PDB ID : 6LU7 ) , we developed a novel advanced deep Q-learning network with the fragment-based drug design ( ADQN-FBDD ) for generating potential lead compounds targeting SARS-CoV-2 3CLpro . We obtained a series of derivatives from those lead compounds by our structure-based optimization policy ( SBOP ) .
In recent years , deep reinforcement learning has been widely used in games , robots , autonomous driving and other fields , which proves that deep reinforcement learning is powerful in decision making and problem solving . In this paper , according to deep reinforcement learning and the characteristic of thermal process control , a thermal process control method based on DQN ( Deep Q-learning Network ) is proposed . Through matlab and python mixed programming , the process of DQN controller training and the simulation of control system are completed .
Each node decides its transmitting/receiving mode and the transmission power based on its local observation of the system state . We formulate a Markov game and develop a multi-agent deep reinforcement learning algorithm based on deep recurrent Q-network . The simulation results show that the proposed approach outperforms the baselines significantly .
We formulate our optimization problem as a resource allocation as well as user association problem . We propose a deep Q-Network based algorithm , namely Delay Minimizing Deep Q-Network ( DM-DQN ) to address the low-latency requirement . DM-DQN aims at reducing the delay of CUDs by balancing the trade-off between allocating more RBs and associating devices to base stations with high channel quality .
In this paper , we propose a deep Q-network based learning approach for connectivity preservation problem . We adopt a fully connected neural network with nonlinearities as the Q-function in the deep Q-network based framework . Further , we design and implement a simulation environment for connectivity preservation , which provides states and reward feedbacks to deep Q-network .
Because of the tremendous research that focuses on human brains and reinforcement learning , scientists have investigated how robots can autonomously tackle complex tasks in the form of making a self-driving agent control in a human-like way . In this study , we propose an end-to-end architecture using novel deep-Q-network architecture in conjunction with a recurrence to resolve the problem in the field of simulated self-driving . The main contribution of this study is that we trained the driving agent using a brain-inspired trial-and-error technique , which was in line with the real world situation .
To make our study realistic , the actual climatic data from each district in Thailand together with the actual rice yields were used to construct the yield estimator where the decision tree with Adaboost was employed . Next , we constructed the simulated farming environments for different districts in Thailand and employed the deep Q-learning neural network to find the optimum actions . From our simulated examples , the deep Q-learning may allow farmers to earn more than the average income .
This architecture uses four main components : parallel actors that generate new behaviour ; parallel learners that are trained from stored experience ; a distributed neural network to represent the value function or behaviour policy ; and a distributed store of experience . We used our architecture to implement the Deep Q - Network algorithm ( DQN ) . Our distributed algorithm was applied to 49 games from Atari 2600 games from the Arcade Learning Environment , using identical hyperparameters .
However , it is well - known that the Q - learning algorithm is highly unstable when combined with non - linear function approximators such as deep neural networks . subsection : Deep Q - Networks Recently , a new RL algorithm has been developed which is in practice much more stable when combined with deep Q - networks .
This is complementary to distributing gradient computation , and the two approaches can be combined , but in this work we focus purely on data - generation . We use this distributed architecture to scale up variants of Deep Q - Networks ( DQN ) and Deep Deterministic Policy Gradient ( DDPG ) , and we evaluate these on the Arcade Learning Environment benchmark bellemare2013arcade , and on a range of continuous control tasks . Our architecture achieves a new state of the art performance on Atari games , using a fraction of the wall - clock time compared to the previous state of the art , and without per - game hyperparameter tuning .
subsection : Value - based algorithms The Deep Q - Network ( DQN ) framework , introduced by mnih15human , popularised the current line of research into deep reinforcement learning by reaching human - level , and beyond , performance across 57 Atari 2600 games in the ALE . While DQN includes many specific components , the essence of the framework , much of which is shared by Neural Fitted Q - Learning riedmiller2005neural , is to use of a deep convolutional neural network to approximate an action - value function , training this approximate action - value function using the Q - Learning algorithm watkins1992 and mini - batches of one - step transitions ( ) drawn randomly from an experience replay buffer lin1992self .
In this paper , we propose a deep reinforcement learning ( DRL ) framework to improve the selection of macro actions . Our framework is based on the combination of the Ape-X DQN and the Long-Short-Term-Memory ( LSTM ) . We use this framework to build our bot , named as LastOrder .
In addition , it provides support for multiplayer and multi-agent experiments . We propose three full-game scenarios of varying difficulty with the Football Benchmarks and report baseline results for three commonly used reinforcement algorithms ( IMPALA , PPO , and Ape-X DQN ) . We also provide a diverse set of simpler scenarios with the Football Academy and showcase several promising research directions .
Reflections in the Sky: Millimeter Wave Communication with UAV-Carried Intelligent Reflectors Emergence of adaptive behaviors by redundant robots - robustness to changes environment and failures Reinforcement Learning of Visual Features Distributed visual navigation based on neural Q-learning for a mobile robot Q-Learning with Hidden-Unit Restarting Incentivizing Exploration In Reinforcement Learning With Deep Predictive Models Deep Reinforcement Learning with Double Q-Learning Implicit Quantile Networks for Distributional Reinforcement Learning Self-Paced Prioritized Curriculum Learning With Coverage Penalty in Deep Reinforcement Learning Computation Offloading Strategy Based on Deep Reinforcement Learning in Cloud-Assisted Mobile Edge Computing Geomagnetic Navigation for AUV based on Deep Reinforcement Learning Algorithm Deep Q-Network for Angry Birds Learning Agents with Prioritization and Parameter Noise in Continuous State and Action Space A New Anti-Jamming Strategy Based on Deep Reinforcement Learning for MANET Deep Interactive Reinforcement Learning for Path Following of Autonomous Underwater Vehicle Motion Coordination of Multiple Robots Based on Deep Reinforcement Learning Reinforcement learning based optimizer for improvement of predicting tunneling-induced ground responses Approximate Policy-Based Accelerated Deep Reinforcement Learning Unifying Count-Based Exploration and Intrinsic Motivation Count-Based Exploration in Feature Space for Reinforcement Learning Incentivizing Exploration In Reinforcement Learning With Deep Predictive Models The Reactor: A Sample-Efficient Actor-Critic Architecture Implicit Quantile Networks for Distributional Reinforcement Learning Deep Reinforcement Learning with Double Q-Learning Massively Parallel Methods for Deep Reinforcement Learning Traffic Prediction and Random Access Control Optimization: Learning and Non-learning based Approaches AI-aided design of novel targeted covalent inhibitors against SARS-CoV-2 The Application of DQN in Thermal Process Control Age of Information Minimization for Wireless Ad Hoc Networks: A Deep Reinforcement Learning Approach Deep Q-Learning for Low-Latency Tactile Applications: Microgrid Communications Deep Q-Learning to Preserve Connectivity in Multi-robot Systems Toward a Brain-Inspired System: Deep Recurrent Reinforcement Learning for a Simulated Self-Driving Agent Rice Cultivation Planning Using A Deep Learning Neural Network Massively Parallel Methods for Deep Reinforcement Learning Distributed Prioritized Experience Replay The Reactor: A Sample-Efficient Actor-Critic Architecture Macro action selection with deep reinforcement learning in StarCraft Google Research Football: A Novel Reinforcement Learning Environment
3e3698404a4169c15b0856164c4aeac2e742d83c 0d9111453b6a6a8460c763d1d5700a95614abc10 6dfd49f1c355fc58fd27b0c119428b812807b100 77b13f8aefc6a77f941d938ffd33b5852358604c bf8ee51dca04d65a76a8eecb3b7ab1dbdb54bd79 43c0f5ebd97225369e88418b4e957c3d592c0389 989d3307f37c17aa5282366bfa8d7fa972666b9f 8a8e48cae9b4873af59369799ad88cdce9df4a39 449a3e01b6c42adf7431cd8fdd70b1b575267f6e ca7128d9412a349991325a560544db4dd147013b 72df46417b2d84dadd32f1f3b20e8be60fed86ad 0d82360a4da311a277607db355dda3f196e8eb3d 6a8b9d285b63dcf083540bace0e195f429e78167 bdfa708fcd7f8f46a260232cba8175b574d0b8a6 0a56bb37ab28a39c31fe2572595368398d7a2e75 1f2f6aace741896ae8a8d2c20697d7e52a22de5b 69ef7f16339b70d6971ac2fe2b9d286014cd712a 4c2106b1aff591968f610d72e75cc465ba8a72c7 284b4a673f311ea7c644865e24807deabb89d583 3b256bd25158aa5e0592462c49c245a872465839 fddb55f708c7796c88b0b83d4f0f63bb6ca4efdf 4a9b976b1447512ff40e586014505953800af111 04e3249f5ed044bd5fcfe4d4eef570cc262932fc ceaee39e1d40bf781f54f4f99601f2e115a460bd 114e87eb358b75ac0c2ce224d4f9f142f50cdd8b
["Computer Science","Mathematics","Engineering"] ["Computer Science","Materials Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science","Medicine"] ["Computer Science"] ["Engineering","Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Engineering","Computer Science"] ["Computer Science","Medicine"] ["Computer Science"] ["Biology","Medicine"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science","Medicine"] ["Computer Science"] ["Computer Science"] ["Computer Science","Mathematics"]
2019 2003 2016 2006 1992 2015 2016 2018 2018 2020 2019 2019 2019 2019 2020 2019 2020 2020 2016 0 2017 2015 2017 2018 2016 2015 2020 2020 2019 2019 2018 2017 2019 2019 2015 2018 2017 2018 2019
NIPS ArXiv AAAI ICML ROBIO ISNN (1) VTC Spring ICTAI NIPS IJCAI ArXiv ArXiv ICML AAAI ArXiv GLOBECOM SmartGridComm ICSPS ArXiv ICLR ArXiv
https://api.semanticscholar.org/3e3698404a4169c15b0856164c4aeac2e742d83c https://api.semanticscholar.org/0d9111453b6a6a8460c763d1d5700a95614abc10 https://api.semanticscholar.org/6dfd49f1c355fc58fd27b0c119428b812807b100 https://api.semanticscholar.org/77b13f8aefc6a77f941d938ffd33b5852358604c https://api.semanticscholar.org/bf8ee51dca04d65a76a8eecb3b7ab1dbdb54bd79 https://www.semanticscholar.org/paper/2470fcf0f89082de874ac9133ccb3a8667dd89a8 https://www.semanticscholar.org/paper/3b9732bb07dc99bde5e1f9f75251c6ea5039373e https://www.semanticscholar.org/paper/d85623ffae865f9ef386644dd02d0ea2d6a8c8de https://api.semanticscholar.org/43c0f5ebd97225369e88418b4e957c3d592c0389 https://api.semanticscholar.org/989d3307f37c17aa5282366bfa8d7fa972666b9f https://api.semanticscholar.org/8a8e48cae9b4873af59369799ad88cdce9df4a39 https://api.semanticscholar.org/449a3e01b6c42adf7431cd8fdd70b1b575267f6e https://api.semanticscholar.org/ca7128d9412a349991325a560544db4dd147013b https://api.semanticscholar.org/72df46417b2d84dadd32f1f3b20e8be60fed86ad https://api.semanticscholar.org/0d82360a4da311a277607db355dda3f196e8eb3d https://api.semanticscholar.org/6a8b9d285b63dcf083540bace0e195f429e78167 https://api.semanticscholar.org/bdfa708fcd7f8f46a260232cba8175b574d0b8a6 https://api.semanticscholar.org/0a56bb37ab28a39c31fe2572595368398d7a2e75 https://www.semanticscholar.org/paper/6e90fd78e8a3b98af3954aae5209703aa966603e https://www.semanticscholar.org/paper/0f810eb4777fd05317951ebaa7a3f5835ee84cf4 https://www.semanticscholar.org/paper/2470fcf0f89082de874ac9133ccb3a8667dd89a8 https://www.semanticscholar.org/paper/052282998bc24db695891f755a00e3cebd3fd796 https://www.semanticscholar.org/paper/d85623ffae865f9ef386644dd02d0ea2d6a8c8de https://www.semanticscholar.org/paper/3b9732bb07dc99bde5e1f9f75251c6ea5039373e https://www.semanticscholar.org/paper/d5ed07113ddcd038062525a5a54550c012ac9a74 https://api.semanticscholar.org/1f2f6aace741896ae8a8d2c20697d7e52a22de5b https://api.semanticscholar.org/69ef7f16339b70d6971ac2fe2b9d286014cd712a https://api.semanticscholar.org/4c2106b1aff591968f610d72e75cc465ba8a72c7 https://api.semanticscholar.org/284b4a673f311ea7c644865e24807deabb89d583 https://api.semanticscholar.org/3b256bd25158aa5e0592462c49c245a872465839 https://api.semanticscholar.org/fddb55f708c7796c88b0b83d4f0f63bb6ca4efdf https://api.semanticscholar.org/4a9b976b1447512ff40e586014505953800af111 https://api.semanticscholar.org/04e3249f5ed044bd5fcfe4d4eef570cc262932fc https://www.semanticscholar.org/paper/d5ed07113ddcd038062525a5a54550c012ac9a74 https://www.semanticscholar.org/paper/1f08598381af9146d0fd9a61b30d0e51a7331689 https://www.semanticscholar.org/paper/052282998bc24db695891f755a00e3cebd3fd796 https://api.semanticscholar.org/ceaee39e1d40bf781f54f4f99601f2e115a460bd https://api.semanticscholar.org/114e87eb358b75ac0c2ce224d4f9f142f50cdd8b