In this paper , a critical situation learning module is combined with the evolved fuzzy systems , i.e. , reflective action module . The critical situation learning module is composed of Q-learning with CMAC . Location information of surrounding ghosts and the existence of power-pills are given to PacMan as state . Q -- learning algorithm is one of the most popular model -- free reinforcement learning algorithms . This paper extends Q -- learning algorithm to multi -- agent cooperative team domain properly , and proposes a shared experience tuples multi -- agent cooperative reinforcement learning algorithm , in which a new knowledge representation form composed of sequential pair as 〈 state-- value , action -- valu 〉 is proposed . And experience tuples are shared with other agents through similarity transformation according to homogeneous subtasks . We prove that neural Q-learning finds the optimal policy with $ O(1/\sqrt{T})$ convergence rate if the neural function approximator is sufficiently overparameterized , where $ T$ is the number of iterations . To our best knowledge , our result is the first finite-time analysis of neural Q-learning under non-i.i.d . data assumption . In this work we present the results of preliminary studies on how neural networks can be utilized to path planning on square grids , e.g. how well they can cope with path finding tasks by themselves within the well-known reinforcement problem statement . Conducted experiments show that the agent using neural Q-learning algorithm robustly learns to achieve the goal on small maps and demonstrate promising results on the maps have ben never seen by him before . The sequentially stacked layers of Recurrent Neural network is fed by the data parameters . The Q- learning network constructs a crop yield prediction environment based on the input parameters . A linear layer maps the Recurrent Neural Network output values to the Q-values . The application of the network and the intelligent terminal opens a new mode of medical treatment , in the diagnosis and treatment of mental disease , how to build a smart , efficient model is a major issue in front of scientific research . In this paper , the energy consumption of the mobile terminal is reduced based on the Q learning network and the wireless body area network routing method , improve the accuracy of data reception and the battery life . Dynamic data obtained by real-time data exchange , BP Adaboost strong classifier algorithm is used to accurately model the psychological problems of patients . In a Smart home environment Home Energy Management plays a critical role in finding a suitable and reliable solution to curtail the peak demand and achieve energy conservation . In this paper , a new method named as Home Energy Management as a Service ( HEMaaS ) is proposed which is based on neural network based Q-learning algorithm . Although several attempts have been made in the past to address similar problems , the models developed do not cater to maximize the user convenience and robustness of the system . However , when the state of Markov Decision Process is very big or continuous , the computation load and the memory load will become very big and can not be solved then . Therefore , this paper proposed a neural network based Q learning algorithm to solve this problem known as “ Curse of Dimensionality ” . This new method realized generalization of conventional Q learnig algorithm in huge and continuous state space as neural network is a very effective value function approximator . The task investigated in this thesis is a modified form of the Block Design ( BD ) and Block Design Multiple Choice ( BDMC ) subtests , used by the Fourth Edition of the Wechsler Intelligence Scale for Children ( WISC-IV ) to partially assess childrens ’ learning abilities . This thesis investigates the implementation , training , and performance of two reinforcement learning architectures for this problem : Soar-RL , a production system capable of reinforcement learning , and a Q-learning neural network . The objective is to help define the advantages and disadvantages of solving problems using these architectures . Based on the analysis , we propose a new paradigm toward deep fusion of computational and symbolic processing and show the new model as the first step of the paradigm . The model is realized by “ Symbol Emergence Method for Q-Learning Neural Network ” . We testified the validity of the new method . This study applies machine learning methods to determine green times in order to minimize in an isolated intersection . Q-learning and neural networks are applied here to set signal light times and minimize total delays . It is assumed that an intersection behaves in a similar fashion to an intelligent agent learning how to set green times in each cycle based on traffic information . We achieve the complexity reduction by considering only a limited set of basic beams which we flexibly combine with each other in order to obtain the overall beam pattern of the actual transmission . We propose a Nelder-Mead and a Q-Learningbased approach in order to adjust the remaining parameters , i. e. vertical and horizontal angles of every basic beam . Based on system level simulations we can conclude that 2-dimensional antenna arrays can significantly outperform linear arrays when carrying out concurrent coverage and capacity optimization . Further , to achieve collaborative recommendations for the entire user community , we propose a novel CF-based MDP by encoding the states of all users into a shared latent vector space . Finally , we propose an effective Q-network learning method to learn the agent 's optimal policy based on the CF-based MDP . The capability of CFRL is demonstrated by comparing its performance against a variety of existing methods on real-world datasets . Since this seminal work , we have seen numerous extensions and improvements that all share the same underlying framework . Double DQN van2016deep , attempts to correct for the over - estimation bias inherent in Q - Learning by changing the second term of eq : tderr to . The dueling architecture wang2015dueling , changes the network to estimate action - values using separate network heads and with Recently , rainbow introduced Rainbow , a value - based reinforcement learning agent combining many of these improvements into a single agent and demonstrating that they are largely complementary . Without this penalty the values of rare actions can be severely overestimated , which biases the gradient estimates and can cause divergence . A similar trick was used in the context of learning simple algorithms with Q - learning zaremba2015learning . paragraph : Reward shaping The parameters of the so - called Q - network are optimized so as to approximately solve the Bellman equation . For example , the Q - learning algorithm iteratively updates the action - value function towards a sample of the Bellman target , . However , it is well - known that the Q - learning algorithm is highly unstable when combined with non - linear function approximators such as deep neural networks . Though relatively costly and human-intensive , a GT-based approach has several merits , for example : ( a ) it is general by design ; ( b ) it can be partially automated ; ( c ) it yields evidence-based results rooted of the system being examined . This article presents one theoretical formulation of a general architecture recovery method – called REM – and reports on the evaluation of REM in the context of a large architecture recovery campaign performed for the European Space Agency . Our results illustrate some intriguing properties and opportunities of GT-based architecture recovery approaches and point out lessons learned and venues for further research . RBD is most often associated with elderly and in those with neurodegenerative disorders such as Parkinson ’s disease , multiple system atrophy and Lewy body dementia . A portable and user friendly REM sleep detecting system is proposed based on differential shift of the eyeball localization using infrared optical sensor in this study . The system consists of an optical source/detector , current regulator , differential amplification , data acquisition , feature extraction , and classification parts , where a 730 nm light emitting diode and two photo detectors were used in the optical source/detector part , and wavelet transformation was applied in the feature extraction and the classification parts . In the feature extraction part , sleep time series data obtained from the data acquisition part was resampled to a sampling frequency of 256 Hz and then filtered with a 1st order 0.16 Hz high pass filter to remove dc offset and a 2nd order 60 Hz band limit filter . Finally , the filtered data was transformed using Matlab dwt function to detect periods of REM sleep . The performance of this REM sleep detecting system was evaluated with overnight recordings of 5 subjects . The results showed sensitivity of 85 % and specificity of 92 % , suggesting that this system can be a very practically efficient in automatic detection of REM sleep stages in a mobile environment . The invention discloses a method and system for searching map information in a wireless environment . The method includes the steps that a wireless environment map ( REM ) system directly or indirectly informs fixed point measurement equipment and / or movable point measurement equipment of fulfillment of REM measurement ; the REM system carries out environment assessment according to measurement data reported by the fixed point measurement equipment and / or the movable point measurement equipment to obtain REM information . According to the method and system , in the method that the fixed point measurement equipment is combined with the movable point measurement equipment , signal conditions of various areas can be fully obtained , so a network side can comprehensively learn about environment conditions , reliability and precision of resource reallocation are improved , and accordingly the resource utilization rate is improved . Motivated by FCC 's recent ruling for TVWS access with a database , we propose a database-like Radio Environment Map ( REM ) based framework to improve cognitive networks ' capacity . At first , through our measured UHF TV data , we find TV channels exhibit constant patterns which mean channel availabilities in future can be predicted for cognitive access with the REM system . We design a detailed REM implementation framework in a multi-service ( between UHF TV and cellular services ) spectrum reuse scenario to solve cellular networks ' spectrum scarce problem . The authors contend that this is more practical and efficient than the generally assumed fail-stop processor . Results of an implementation in the REM ( Remote Execution Manager ) environment are presented.<<ETX > > Radio Environment Map ( REM ) is very useful as it provides insight of the spatial characteristics of radio environment . To achieve higher accuracy , a novel REM methodology is proposed in this paper . Noticing the inability of systematic sampling to tackle inaccessible areas , a sampling scheme named simulated annealing assisted electrons repulsion is formulated . This feature helps in improving power quality in REMs . REM modeling/simulations are performed in MATLAB/Simulink ® and validated with an experimental prototype through dSPACE ® . The efficacy of the proposed scheme is assessed by computing various power quality indices . A three-layer BP neural network is employed to classify these features to one appropriate stage . For a four-stage classification , consisting of Awake , Stage 1 + REM , Stage 2 and slow wave stage ( SWS ) , of one single Pz-Oz channel EEG signal alone from 7 human subjects , the average stage recognition rate of the proposed method can achieve Awake 95.2 % , Stage 1+Rem 87.1 % , Stage 2 82.0 % , SWS 92.9 % . The experiment results show the method is effective and promising in automatic sleep states classification . We trained and tested our system using both healthy and disordered sleep collected from 41 controls and 42 primary insomnia patients . When compared with manual assessments , an NREM + REM based classifier had an overall discrimination accuracy of 92 % and 86 % between two groups using both two and one EEG channels , respectively . These results demonstrate that deep learning can be used to assist in the diagnosis of sleep disorders such as insomnia . This paper examines the benefit of using a Radio Environment Map ( REM ) exploiting measurement based signal level prediction as a means to detect and avoid the hidden node problem within wireless networks . Practical implementation considerations are taken into account , such as support for multi-Radio Access Technology ( RAT ) deployments , unknown transmit power levels , radio measurement inaccuracies and propagation anomalies ( such as fading ) , which can reduce the reliability of REM based predictions . ( 2 ) PCBB is able to dynamically remap after the reconfiguration of the architecture . ( 3 ) General quantitative evaluation for reliability , communication energy and performance are made respectively before integrated into the unified model REM , whereas other similar models only touch upon two of them quantitatively . It is shown to achieve significantly better performance as compared with the previous hand-crafted states . We further find that double DQN and dueling DQN improve the naive version . To address the nonlinearity of the measurement model , nonlinear MASE is factorized into two linear estimation stages with a nonlinear transformation in between . A-DQN is then applied to solve the two linear estimation stages in which the decomposed solution for optimal step length is derived in closed form by exploiting the problem structure . With this method , each area performs a local state estimation with limited information exchange with its neighbors ; thus , no central coordinator is needed . Experimental results show that the proposed approach based on semantic dependency for intent detection increases the accuracy by 4.3 % . For dialog policy decision , the PD-DQN achieves 87.57 % task success rate , which is 13.9 % higher than the baseline Double DQN ( 73.67 % ) . Finally , using the attention-based Transformer for response template generation obtains a Bleu score of 13.6 , improved by 1.5 compared to the Sequence-to-Sequence model . This thesis focuses on comparing these two aforementioned deep RL algorithms on their performances in learning to play different Atari 2600 games provided by the OpenAI gym . The impact of different exploration strategies on the learning performance of DQN and DQV will be examined , more specifically , a diversity-driven approach ( Div-DQN and Div-DQV ) and a noisy network approach ( NoisyNet-DQN and NoisyNet-DQV ) will be compared to traditional implementations of DQN and DQV . The results show that the standard DQV algorithm outperforms DQN and that DQV-based variants in general slightly outperform DQN-based variants . Weighting factor design in the cost function of FCS-MPC algorithm is one of the widely reported problems which still under investigation . In this paper , the prediction stage of FCS-MPC for asymmetric six-phase induction motor has been reformulated based on the double d-q ( 2dq ) model of the machine . Unlike the reported FCS-MPC algorithm in literature , this will results in four control variables ( stator currents in the double stator circuits ) which have the same priority . The performance was compared with that of the Pure-Pursuit Control ( PPC ) algorithm . The results showed that the Double DQN-based control dramatically reduced the settling time and the overshoot at the corner at higher forward speeds at a minor expense of slightly increased rise time and steady state error . For robotic system 's steady exploration , we designed the safety constraints according to working condition . Our experiments show that our double-task DQN with multiple views performs better than the single-task and single-view model . Combining our DMDQN and data augment , the robotic system can reach the object in an exploration way . We propose an approach for introducing the origin shift into the multiple dqd algorithm for computing the eigenvalues of a totally nonnegative matrix . Numerical experiments show that the shift speeds up the convergence while retaining the accuracy of the computed eigenvalue . Additionally , the TBSS-DQN incorporates both the target branch and context branch to exploit contextual relations . The experimental results on the UCF-Sports , UCF-101 , ActivityNet , JHMDB , and sub-JHMDB datasets demonstrate that our ST-DQN achieves promising localization performance with a very small number of proposals . The results also demonstrate that exploiting contextual information and temporal dependencies contributes to accurate detection of the spatio-temporal boundary . Note that in this section we use the risk - neutral variant of the IQN , that is , the policy of the IQN agent is the regular - greedy policy with respect to the mean of the state - action return distribution . It is important to remember that Rainbow builds upon the distributional RL algorithm C51 , but also includes prioritized experience replay , Double DQN , Dueling Network architecture , Noisy Networks , and multi - step updates . In particular , besides the distributional update , - step updates and prioritized experience replay were found to have significant impact on the performance of Rainbow . We further derive the asymptotic growth exponent of the AWD 's , which are then used to bound the ensemble performance under maximum likelihood ( ML ) decoding . It is shown that simple nonsystematic regular RA ensembles outperform systematic regular RA and regular low-density parity-check ( LDPC ) ensembles and have distance spectrum which closely resembles that of the random ensemble . This problem has applications in wireless interference management and recommendation systems . We determine necessary and sufficient conditions for the design of each matrix , such that the random ensemble will almost surely lose rank by a certain amount . The result is proved by converting the problem to determining rank loss conditions for the union of some specific matroids , and then using tools from matroid and graph theories to derive the necessary and sufficient conditions . We tested the performance of the COB model on 32 datasets from the UCI repository . The experiments use three different ensemble methods ( bagging , random forests , and a randomized ensemble ) , two different numbers of individual classifiers ( 7 and 51 ) , and three different individual machine learning algorithms ( decision trees , k-nearest neighbors , and support vector machines ) . All 24 experiments showed less than 5 % average absolute errors for 32 datasets between the accuracies by the COB model and the actual accuracies of ensembles . Our experiments suggest that the benefits of distributional RL mainly stem from better exploitation . We present a simple and novel variant of ensemble Q-learning called Random Ensemble Mixture ( REM ) , which enforces optimal Bellman consistency on random convex combinations of the Q-heads of a multi-head Q-network . The batch REM agent trained offline on DQN data outperforms the batch QR-DQN and online C51 algorithms . In order to improve the accuracy of river hydraulic model , ensemble Kalman filtering method based on the concept of ensemble was used for real-time updating model states . The key of ensemble Kalman filter lied in the set of initial state ensemble , so that Box Muller method was adopted to generate a set of normally distributed random ensemble . A simulation of a river network composed of 14 channels was used to systematically analyze the data assimilation effect about size and standard deviation of ensemble , gaining preliminary conclusions which were applied to a real case . It is a stack of random model ensembles . Each random model ensemble is a special k-means ensemble that discards the expectation-maximization optimization of each base k-means but only preserves the default initialization method of the base k-means . ( iv ) We propose to select the most powerful representation among the layers by applying DRME to clustering where the single-linkage is used as the clustering algorithm . In this paper , we propose Max-diverse.α , which has a mechanism to control the degrees of randomness in decision tree ensembles . This control gives an ensemble the means to balance the two conflicting functions of a random random ensemble , i.e. , the abilities to model non-axis-parallel boundary and eliminate irrelevant features . We find that this control is more sensitive to the one provided by Random Forests . With comparison to deep Q network ( DQN ) and prioritized experience replay ( PER ) methods , the DCRL algorithm is evaluated on Atari 2600 games , and the experimental results show that DCRL outperforms DQN and PER on most of these games . More results further show that the proposed curriculum training paradigm of DCRL is also applicable and effective for other memory-based deep reinforcement learning approaches , such as double DQN and dueling network . All the experimental results demonstrate that DCRL can achieve improved training efficiency and robustness for deep reinforcement learning . To reduce the overestimations of action values during learning , we present a novel combination of double Q-learning and dueling DQN algorithm , and design an algorithm called Variant of Double dueling DQN ( V-D D3QN ) . We focus on the idea behind V-D D3QN algorithm and propose the feasible idea of using two dueling DQN networks to reduce the overestimations of action values during training , and the specific approach is to randomly select one dueling DQN network at each time step to update its parameters , by exploiting the remaining dueling DQN network to determine the update targets . And then we do our experiments in the customized virtual environment-gridmap . We show the performance of a connectionist Q-Learning algorithm with a target network and experience replay , and compare it to three other algorithms . One that uses the on-policy algorithm SARSA , one with the addition of a dueling network architecture and one with both modifications . To accelerate learning , we use a state representation in which the system receives as input three versions of the full map size each showing only a single feature such as agent location and fire locations . VANETs are vulnerable to malicious nodes and result in performance degradation because of dynamicity and infrastructure-less . In this paper , we propose a trust based dueling deep reinforcement learning approach ( T-DDRL ) for communication of connected vehicles , we deploy a dueling network architecture into a logically centralized controller of software-defined networking ( SDN ) . Specifically , the SDN controller is used as an agent to learn the most trusted routing path by deep neural network ( DNN ) in VANETs , where the trust model is designed to evaluate neighbors ' behaviour of forwarding routing information . Note that in this section we use the risk - neutral variant of the IQN , that is , the policy of the IQN agent is the regular - greedy policy with respect to the mean of the state - action return distribution . It is important to remember that Rainbow builds upon the distributional RL algorithm C51 , but also includes prioritized experience replay , Double DQN , Dueling Network architecture , Noisy Networks , and multi - step updates . In particular , besides the distributional update , - step updates and prioritized experience replay were found to have significant impact on the performance of Rainbow . First , we combined it with a variant of DQN dqn with some of the components of Rainbow rainbow . More specifically , we used double Q - learning doubleq , deepdoubleqlearning with multi - step bootstrap targets [ cf. ][] Sutton:1988 , SuttonBarto:1998 , SuttonBarto:2017 , a3c as the learning algorithm , and a dueling network architecture dueling as the function approximator . This results in computing for all elements in the batch the loss with where is a time index for an experience sampled from the replay starting with state and action , and denotes parameters of the target network dqn , a slow moving copy of the online parameters . Table [ reference ] compares our evaluation scores to Double DQN ( DDQN ) , Double DQN with pseudocount ( DDQN - PC ) , A3C + , DQN Pop - Art ( DQN - PA ) , Dueling Network ( Dueling ) , Gorila , DQN with Model Prediction Exploration Bonuses ( MP - EB ) , Trust Region Policy Optimisation ( TRPO ) , and TRPO - AE - SimHash ( TRPO - Hash ) . The most interesting comparisons for our purposes are with TRPO - Hash , DDQN - PC , A3C + , and MP - EB , because these algorithms all use exploration strategies that drive the agent to reduce its uncertainty .
In this paper , a critical situation learning module is combined with the evolved fuzzy systems , i.e. , reflective action module . The critical situation learning module is composed of Q-learning with CMAC . Location information of surrounding ghosts and the existence of power-pills are given to PacMan as state .
Q -- learning algorithm is one of the most popular model -- free reinforcement learning algorithms . This paper extends Q -- learning algorithm to multi -- agent cooperative team domain properly , and proposes a shared experience tuples multi -- agent cooperative reinforcement learning algorithm , in which a new knowledge representation form composed of sequential pair as 〈 state-- value , action -- valu 〉 is proposed . And experience tuples are shared with other agents through similarity transformation according to homogeneous subtasks .
We prove that neural Q-learning finds the optimal policy with $ O(1/\sqrt{T})$ convergence rate if the neural function approximator is sufficiently overparameterized , where $ T$ is the number of iterations . To our best knowledge , our result is the first finite-time analysis of neural Q-learning under non-i.i.d . data assumption .
In this work we present the results of preliminary studies on how neural networks can be utilized to path planning on square grids , e.g. how well they can cope with path finding tasks by themselves within the well-known reinforcement problem statement . Conducted experiments show that the agent using neural Q-learning algorithm robustly learns to achieve the goal on small maps and demonstrate promising results on the maps have ben never seen by him before .
The sequentially stacked layers of Recurrent Neural network is fed by the data parameters . The Q- learning network constructs a crop yield prediction environment based on the input parameters . A linear layer maps the Recurrent Neural Network output values to the Q-values .
The application of the network and the intelligent terminal opens a new mode of medical treatment , in the diagnosis and treatment of mental disease , how to build a smart , efficient model is a major issue in front of scientific research . In this paper , the energy consumption of the mobile terminal is reduced based on the Q learning network and the wireless body area network routing method , improve the accuracy of data reception and the battery life . Dynamic data obtained by real-time data exchange , BP Adaboost strong classifier algorithm is used to accurately model the psychological problems of patients .
In a Smart home environment Home Energy Management plays a critical role in finding a suitable and reliable solution to curtail the peak demand and achieve energy conservation . In this paper , a new method named as Home Energy Management as a Service ( HEMaaS ) is proposed which is based on neural network based Q-learning algorithm . Although several attempts have been made in the past to address similar problems , the models developed do not cater to maximize the user convenience and robustness of the system .
However , when the state of Markov Decision Process is very big or continuous , the computation load and the memory load will become very big and can not be solved then . Therefore , this paper proposed a neural network based Q learning algorithm to solve this problem known as “ Curse of Dimensionality ” . This new method realized generalization of conventional Q learnig algorithm in huge and continuous state space as neural network is a very effective value function approximator .
The task investigated in this thesis is a modified form of the Block Design ( BD ) and Block Design Multiple Choice ( BDMC ) subtests , used by the Fourth Edition of the Wechsler Intelligence Scale for Children ( WISC-IV ) to partially assess childrens ’ learning abilities . This thesis investigates the implementation , training , and performance of two reinforcement learning architectures for this problem : Soar-RL , a production system capable of reinforcement learning , and a Q-learning neural network . The objective is to help define the advantages and disadvantages of solving problems using these architectures .
Based on the analysis , we propose a new paradigm toward deep fusion of computational and symbolic processing and show the new model as the first step of the paradigm . The model is realized by “ Symbol Emergence Method for Q-Learning Neural Network ” . We testified the validity of the new method .
This study applies machine learning methods to determine green times in order to minimize in an isolated intersection . Q-learning and neural networks are applied here to set signal light times and minimize total delays . It is assumed that an intersection behaves in a similar fashion to an intelligent agent learning how to set green times in each cycle based on traffic information .
We achieve the complexity reduction by considering only a limited set of basic beams which we flexibly combine with each other in order to obtain the overall beam pattern of the actual transmission . We propose a Nelder-Mead and a Q-Learningbased approach in order to adjust the remaining parameters , i. e. vertical and horizontal angles of every basic beam . Based on system level simulations we can conclude that 2-dimensional antenna arrays can significantly outperform linear arrays when carrying out concurrent coverage and capacity optimization .
Further , to achieve collaborative recommendations for the entire user community , we propose a novel CF-based MDP by encoding the states of all users into a shared latent vector space . Finally , we propose an effective Q-network learning method to learn the agent 's optimal policy based on the CF-based MDP . The capability of CFRL is demonstrated by comparing its performance against a variety of existing methods on real-world datasets .
Since this seminal work , we have seen numerous extensions and improvements that all share the same underlying framework . Double DQN van2016deep , attempts to correct for the over - estimation bias inherent in Q - Learning by changing the second term of eq : tderr to . The dueling architecture wang2015dueling , changes the network to estimate action - values using separate network heads and with Recently , rainbow introduced Rainbow , a value - based reinforcement learning agent combining many of these improvements into a single agent and demonstrating that they are largely complementary .
Without this penalty the values of rare actions can be severely overestimated , which biases the gradient estimates and can cause divergence . A similar trick was used in the context of learning simple algorithms with Q - learning zaremba2015learning . paragraph : Reward shaping
The parameters of the so - called Q - network are optimized so as to approximately solve the Bellman equation . For example , the Q - learning algorithm iteratively updates the action - value function towards a sample of the Bellman target , . However , it is well - known that the Q - learning algorithm is highly unstable when combined with non - linear function approximators such as deep neural networks .
Though relatively costly and human-intensive , a GT-based approach has several merits , for example : ( a ) it is general by design ; ( b ) it can be partially automated ; ( c ) it yields evidence-based results rooted of the system being examined . This article presents one theoretical formulation of a general architecture recovery method – called REM – and reports on the evaluation of REM in the context of a large architecture recovery campaign performed for the European Space Agency . Our results illustrate some intriguing properties and opportunities of GT-based architecture recovery approaches and point out lessons learned and venues for further research .
RBD is most often associated with elderly and in those with neurodegenerative disorders such as Parkinson ’s disease , multiple system atrophy and Lewy body dementia . A portable and user friendly REM sleep detecting system is proposed based on differential shift of the eyeball localization using infrared optical sensor in this study . The system consists of an optical source/detector , current regulator , differential amplification , data acquisition , feature extraction , and classification parts , where a 730 nm light emitting diode and two photo detectors were used in the optical source/detector part , and wavelet transformation was applied in the feature extraction and the classification parts . In the feature extraction part , sleep time series data obtained from the data acquisition part was resampled to a sampling frequency of 256 Hz and then filtered with a 1st order 0.16 Hz high pass filter to remove dc offset and a 2nd order 60 Hz band limit filter . Finally , the filtered data was transformed using Matlab dwt function to detect periods of REM sleep . The performance of this REM sleep detecting system was evaluated with overnight recordings of 5 subjects . The results showed sensitivity of 85 % and specificity of 92 % , suggesting that this system can be a very practically efficient in automatic detection of REM sleep stages in a mobile environment .
The invention discloses a method and system for searching map information in a wireless environment . The method includes the steps that a wireless environment map ( REM ) system directly or indirectly informs fixed point measurement equipment and / or movable point measurement equipment of fulfillment of REM measurement ; the REM system carries out environment assessment according to measurement data reported by the fixed point measurement equipment and / or the movable point measurement equipment to obtain REM information . According to the method and system , in the method that the fixed point measurement equipment is combined with the movable point measurement equipment , signal conditions of various areas can be fully obtained , so a network side can comprehensively learn about environment conditions , reliability and precision of resource reallocation are improved , and accordingly the resource utilization rate is improved .
Motivated by FCC 's recent ruling for TVWS access with a database , we propose a database-like Radio Environment Map ( REM ) based framework to improve cognitive networks ' capacity . At first , through our measured UHF TV data , we find TV channels exhibit constant patterns which mean channel availabilities in future can be predicted for cognitive access with the REM system . We design a detailed REM implementation framework in a multi-service ( between UHF TV and cellular services ) spectrum reuse scenario to solve cellular networks ' spectrum scarce problem .
The authors contend that this is more practical and efficient than the generally assumed fail-stop processor . Results of an implementation in the REM ( Remote Execution Manager ) environment are presented.<<ETX > >
Radio Environment Map ( REM ) is very useful as it provides insight of the spatial characteristics of radio environment . To achieve higher accuracy , a novel REM methodology is proposed in this paper . Noticing the inability of systematic sampling to tackle inaccessible areas , a sampling scheme named simulated annealing assisted electrons repulsion is formulated .
This feature helps in improving power quality in REMs . REM modeling/simulations are performed in MATLAB/Simulink ® and validated with an experimental prototype through dSPACE ® . The efficacy of the proposed scheme is assessed by computing various power quality indices .
A three-layer BP neural network is employed to classify these features to one appropriate stage . For a four-stage classification , consisting of Awake , Stage 1 + REM , Stage 2 and slow wave stage ( SWS ) , of one single Pz-Oz channel EEG signal alone from 7 human subjects , the average stage recognition rate of the proposed method can achieve Awake 95.2 % , Stage 1+Rem 87.1 % , Stage 2 82.0 % , SWS 92.9 % . The experiment results show the method is effective and promising in automatic sleep states classification .
We trained and tested our system using both healthy and disordered sleep collected from 41 controls and 42 primary insomnia patients . When compared with manual assessments , an NREM + REM based classifier had an overall discrimination accuracy of 92 % and 86 % between two groups using both two and one EEG channels , respectively . These results demonstrate that deep learning can be used to assist in the diagnosis of sleep disorders such as insomnia .
This paper examines the benefit of using a Radio Environment Map ( REM ) exploiting measurement based signal level prediction as a means to detect and avoid the hidden node problem within wireless networks . Practical implementation considerations are taken into account , such as support for multi-Radio Access Technology ( RAT ) deployments , unknown transmit power levels , radio measurement inaccuracies and propagation anomalies ( such as fading ) , which can reduce the reliability of REM based predictions .
( 2 ) PCBB is able to dynamically remap after the reconfiguration of the architecture . ( 3 ) General quantitative evaluation for reliability , communication energy and performance are made respectively before integrated into the unified model REM , whereas other similar models only touch upon two of them quantitatively .
It is shown to achieve significantly better performance as compared with the previous hand-crafted states . We further find that double DQN and dueling DQN improve the naive version .
To address the nonlinearity of the measurement model , nonlinear MASE is factorized into two linear estimation stages with a nonlinear transformation in between . A-DQN is then applied to solve the two linear estimation stages in which the decomposed solution for optimal step length is derived in closed form by exploiting the problem structure . With this method , each area performs a local state estimation with limited information exchange with its neighbors ; thus , no central coordinator is needed .
Experimental results show that the proposed approach based on semantic dependency for intent detection increases the accuracy by 4.3 % . For dialog policy decision , the PD-DQN achieves 87.57 % task success rate , which is 13.9 % higher than the baseline Double DQN ( 73.67 % ) . Finally , using the attention-based Transformer for response template generation obtains a Bleu score of 13.6 , improved by 1.5 compared to the Sequence-to-Sequence model .
This thesis focuses on comparing these two aforementioned deep RL algorithms on their performances in learning to play different Atari 2600 games provided by the OpenAI gym . The impact of different exploration strategies on the learning performance of DQN and DQV will be examined , more specifically , a diversity-driven approach ( Div-DQN and Div-DQV ) and a noisy network approach ( NoisyNet-DQN and NoisyNet-DQV ) will be compared to traditional implementations of DQN and DQV . The results show that the standard DQV algorithm outperforms DQN and that DQV-based variants in general slightly outperform DQN-based variants .
Weighting factor design in the cost function of FCS-MPC algorithm is one of the widely reported problems which still under investigation . In this paper , the prediction stage of FCS-MPC for asymmetric six-phase induction motor has been reformulated based on the double d-q ( 2dq ) model of the machine . Unlike the reported FCS-MPC algorithm in literature , this will results in four control variables ( stator currents in the double stator circuits ) which have the same priority .
The performance was compared with that of the Pure-Pursuit Control ( PPC ) algorithm . The results showed that the Double DQN-based control dramatically reduced the settling time and the overshoot at the corner at higher forward speeds at a minor expense of slightly increased rise time and steady state error .
For robotic system 's steady exploration , we designed the safety constraints according to working condition . Our experiments show that our double-task DQN with multiple views performs better than the single-task and single-view model . Combining our DMDQN and data augment , the robotic system can reach the object in an exploration way .
We propose an approach for introducing the origin shift into the multiple dqd algorithm for computing the eigenvalues of a totally nonnegative matrix . Numerical experiments show that the shift speeds up the convergence while retaining the accuracy of the computed eigenvalue .
Additionally , the TBSS-DQN incorporates both the target branch and context branch to exploit contextual relations . The experimental results on the UCF-Sports , UCF-101 , ActivityNet , JHMDB , and sub-JHMDB datasets demonstrate that our ST-DQN achieves promising localization performance with a very small number of proposals . The results also demonstrate that exploiting contextual information and temporal dependencies contributes to accurate detection of the spatio-temporal boundary .
Note that in this section we use the risk - neutral variant of the IQN , that is , the policy of the IQN agent is the regular - greedy policy with respect to the mean of the state - action return distribution . It is important to remember that Rainbow builds upon the distributional RL algorithm C51 , but also includes prioritized experience replay , Double DQN , Dueling Network architecture , Noisy Networks , and multi - step updates . In particular , besides the distributional update , - step updates and prioritized experience replay were found to have significant impact on the performance of Rainbow .
We further derive the asymptotic growth exponent of the AWD 's , which are then used to bound the ensemble performance under maximum likelihood ( ML ) decoding . It is shown that simple nonsystematic regular RA ensembles outperform systematic regular RA and regular low-density parity-check ( LDPC ) ensembles and have distance spectrum which closely resembles that of the random ensemble .
This problem has applications in wireless interference management and recommendation systems . We determine necessary and sufficient conditions for the design of each matrix , such that the random ensemble will almost surely lose rank by a certain amount . The result is proved by converting the problem to determining rank loss conditions for the union of some specific matroids , and then using tools from matroid and graph theories to derive the necessary and sufficient conditions .
We tested the performance of the COB model on 32 datasets from the UCI repository . The experiments use three different ensemble methods ( bagging , random forests , and a randomized ensemble ) , two different numbers of individual classifiers ( 7 and 51 ) , and three different individual machine learning algorithms ( decision trees , k-nearest neighbors , and support vector machines ) . All 24 experiments showed less than 5 % average absolute errors for 32 datasets between the accuracies by the COB model and the actual accuracies of ensembles .
Our experiments suggest that the benefits of distributional RL mainly stem from better exploitation . We present a simple and novel variant of ensemble Q-learning called Random Ensemble Mixture ( REM ) , which enforces optimal Bellman consistency on random convex combinations of the Q-heads of a multi-head Q-network . The batch REM agent trained offline on DQN data outperforms the batch QR-DQN and online C51 algorithms .
In order to improve the accuracy of river hydraulic model , ensemble Kalman filtering method based on the concept of ensemble was used for real-time updating model states . The key of ensemble Kalman filter lied in the set of initial state ensemble , so that Box Muller method was adopted to generate a set of normally distributed random ensemble . A simulation of a river network composed of 14 channels was used to systematically analyze the data assimilation effect about size and standard deviation of ensemble , gaining preliminary conclusions which were applied to a real case .
It is a stack of random model ensembles . Each random model ensemble is a special k-means ensemble that discards the expectation-maximization optimization of each base k-means but only preserves the default initialization method of the base k-means . ( iv ) We propose to select the most powerful representation among the layers by applying DRME to clustering where the single-linkage is used as the clustering algorithm .
In this paper , we propose Max-diverse.α , which has a mechanism to control the degrees of randomness in decision tree ensembles . This control gives an ensemble the means to balance the two conflicting functions of a random random ensemble , i.e. , the abilities to model non-axis-parallel boundary and eliminate irrelevant features . We find that this control is more sensitive to the one provided by Random Forests .
With comparison to deep Q network ( DQN ) and prioritized experience replay ( PER ) methods , the DCRL algorithm is evaluated on Atari 2600 games , and the experimental results show that DCRL outperforms DQN and PER on most of these games . More results further show that the proposed curriculum training paradigm of DCRL is also applicable and effective for other memory-based deep reinforcement learning approaches , such as double DQN and dueling network . All the experimental results demonstrate that DCRL can achieve improved training efficiency and robustness for deep reinforcement learning .
To reduce the overestimations of action values during learning , we present a novel combination of double Q-learning and dueling DQN algorithm , and design an algorithm called Variant of Double dueling DQN ( V-D D3QN ) . We focus on the idea behind V-D D3QN algorithm and propose the feasible idea of using two dueling DQN networks to reduce the overestimations of action values during training , and the specific approach is to randomly select one dueling DQN network at each time step to update its parameters , by exploiting the remaining dueling DQN network to determine the update targets . And then we do our experiments in the customized virtual environment-gridmap .
We show the performance of a connectionist Q-Learning algorithm with a target network and experience replay , and compare it to three other algorithms . One that uses the on-policy algorithm SARSA , one with the addition of a dueling network architecture and one with both modifications . To accelerate learning , we use a state representation in which the system receives as input three versions of the full map size each showing only a single feature such as agent location and fire locations .
VANETs are vulnerable to malicious nodes and result in performance degradation because of dynamicity and infrastructure-less . In this paper , we propose a trust based dueling deep reinforcement learning approach ( T-DDRL ) for communication of connected vehicles , we deploy a dueling network architecture into a logically centralized controller of software-defined networking ( SDN ) . Specifically , the SDN controller is used as an agent to learn the most trusted routing path by deep neural network ( DNN ) in VANETs , where the trust model is designed to evaluate neighbors ' behaviour of forwarding routing information .
Note that in this section we use the risk - neutral variant of the IQN , that is , the policy of the IQN agent is the regular - greedy policy with respect to the mean of the state - action return distribution . It is important to remember that Rainbow builds upon the distributional RL algorithm C51 , but also includes prioritized experience replay , Double DQN , Dueling Network architecture , Noisy Networks , and multi - step updates . In particular , besides the distributional update , - step updates and prioritized experience replay were found to have significant impact on the performance of Rainbow .
First , we combined it with a variant of DQN dqn with some of the components of Rainbow rainbow . More specifically , we used double Q - learning doubleq , deepdoubleqlearning with multi - step bootstrap targets [ cf. ][] Sutton:1988 , SuttonBarto:1998 , SuttonBarto:2017 , a3c as the learning algorithm , and a dueling network architecture dueling as the function approximator . This results in computing for all elements in the batch the loss with where is a time index for an experience sampled from the replay starting with state and action , and denotes parameters of the target network dqn , a slow moving copy of the online parameters .
Table [ reference ] compares our evaluation scores to Double DQN ( DDQN ) , Double DQN with pseudocount ( DDQN - PC ) , A3C + , DQN Pop - Art ( DQN - PA ) , Dueling Network ( Dueling ) , Gorila , DQN with Model Prediction Exploration Bonuses ( MP - EB ) , Trust Region Policy Optimisation ( TRPO ) , and TRPO - AE - SimHash ( TRPO - Hash ) . The most interesting comparisons for our purposes are with TRPO - Hash , DDQN - PC , A3C + , and MP - EB , because these algorithms all use exploration strategies that drive the agent to reduce its uncertainty .
Detection of critical situations by CMAC+Q-learning for PacMan agents A Shared Experience Tuples Multi-Agent Cooperative Reinforcement Learning Algorithm A Finite-Time Analysis of Q-Learning with Neural Network Function Approximation Grid Path Planning with Deep Reinforcement Learning: Preliminary Results Crop Yield Prediction Using Deep Reinforcement Learning Model for Sustainable Agrarian Applications Explore the Use of the Network and Mobile Terminal in Solving the Psychological Problem Energy Management in Smart Cities Based on Internet of Things: Peak Demand Reduction and Energy Savings Development of a Neural Network Based Q Learning Algorithm for Traffic Signal Control A Comparison of the Performance of Neural Q-learning and Soar-RL on a Derivative of the Block Design (BD)/Block Design Multiple Choice (BDMC) Subtests on the WISC-IV Intelligence Test New paradigm toward deep fusion of computational and symbolic processing Intelligent Traffic Light Control of Isolated Intersections Using Machine Learning Methods Coverage and Capacity Optimization in Cellular Radio Networks with Advanced Antennas When Collaborative Filtering Meets Reinforcement Learning The Reactor: A Sample-Efficient Actor-Critic Architecture An Actor-Critic Algorithm for Sequence Prediction Massively Parallel Methods for Deep Reinforcement Learning General methods for software architecture recovery: a potential approach and its evaluation A Portable and User Friendly REM Sleep Detection System Based on Differential Movement of Eyeball Using Optical Sensors Method and system for searching map information in wireless environment Cognitive spectrum handoff game in the TV white space with radio environment map Implementing halt on failure processors TV Band Radio Environment Mapping in Beijing Improving power quality in microgrids using virtual motor-generator set based control scheme Automatic Sleep Stage Scoring Using Hilbert-Huang Transform with BP Neural Network Deep Learning and Insomnia: Assisting Clinicians With Their Diagnosis REM based approach for hidden node detection and avoidance in cognitive radio networks A Multi-Objective Model Oriented Mapping Approach for NoC-based Computing Systems Interactive Spoken Content Retrieval by Deep Reinforcement Learning An Adaptive Distributed Quasi-Newton Method for Power System State Estimation Attention-Based Response Generation Using Parallel Double Q-Learning for Dialog Policy Decision in a Conversational System Deep Reinforcement Learning in Atari 2600 Games Weighting Factor Elimination for Predictive Current Control of Asymmetric Six Phase Induction Motor Double-DQN based path smoothing and tracking control method for robotic vehicle navigation Double-Task Deep Q-Learning with Multiple Views Differential qd algorithm for totally nonnegative Hessenberg matrices: introduction of origin shifts and relationship with the discrete hungry Lotka-Volterra system Spatio-Temporal Deep Q-Networks for Human Activity Localization Implicit Quantile Networks for Distributional Reinforcement Learning Asymptotic weight distributions of irregular repeat-accumulate codes When does an ensemble of matrices with randomly scaled rows lose rank? A new model for measuring the accuracies of majority voting ensembles Striving for Simplicity in Off-policy Deep Reinforcement Learning Data Assimilation of River Networks Using Ensemble Kalman Filtering Algorithm Simple Deep Random Model Ensemble Variable Randomness in Decision Tree Ensembles Self-Paced Prioritized Curriculum Learning With Coverage Penalty in Deep Reinforcement Learning V-D D3QN: the Variant of Double Deep Q-Learning Network with Dueling Architecture Forest Fire Control with Connectionist Reinforcement Learning A Deep Reinforcement Learning-based Trust Management Scheme for Software-defined Vehicular Networks Implicit Quantile Networks for Distributional Reinforcement Learning Distributed Prioritized Experience Replay Count-Based Exploration in Feature Space for Reinforcement Learning
89bb2b21aa770613fc7dc039ed8c313f6d89cf31 40a6bb62b6194b2af220b1866bca07207cd44431 5ec2506857e38b628a302ca3fe3ebd2f9dcc0f4d c2508a7de42dbd356f14dd29dddf23a57731fcc9 30c15627b2363cbfe02cd1d4215773494042ad28 ad2d7e580b892c095ef331d0cb507c60908cd5cc 2cbe7bd874864f091e9f995c21b8fe3e488495a5 2d011ccbde6a5b76a0a6989f00eeef06b9660ad0 8856b84fa57232c0f988934bdd68a4646edb3ca5 0cb24d3477e864ba56c6abb4189f846a528699e1 7c98354991d4c2e2d4546760e3b62fc54a32d5e9 76e89a2f87157d61355b58a88e7515daac942ddd 5cc18b5bc1d0944dfb1bf8e1825ffc8fe7c2cac2 961b7a43ab3efca1096be4f33b9e6f836fffa55e 51731c180e5df22a3ed84baad1a06a4a471c5510 cabdfdc6caeaa993872d026bc7853ab981d2031a 528778e82c6b71bcb31aad6cb449e39ca3efdc11 d9bfdb07899f838666e475734b93e966cca9421b 41c31ce15fd931413e9a612ba16190c7e0c4400a 33331b6c1d0d75c095fc5874c29b36833dc9fae8 e21d7b5d0aa72d6fe6ad36e16b5285a7426e31b1 b51dc54ccea04f79480396f9f85c02e4f1e0cd97 fb333d25881042fe2f6c2864ef8f95e86a581be9 8f2219bb4bfe8bc1d17431b20d1118033f62d5a4 8049049144472e58698fe32f428e3a2cd4926e1e 6118528d953800d6b7ddf0be50da5207e1d197e6 7b825cbbfb5a647a28f4c7f7d7d8669a21b04827 214d4d52b72a4980fd42d59dacd2a5a62bc8d691 f643efc871beef8ecdfcb7ae00b33872f66c0a04 302ced2dd5c9da6c9029d5b88407a15173db13f6 ff4137be7bf273dd52e4256f167d6f1ba0124ee2 c6ac75f1ee88bb84ebe741f5bde9d34c981a68ab 10129da014606c70b6fab077319491c772b01c04 fc4584adbe076e5560c973eeab387c9d9abba8b7 2a5b0f88c064147a433cd34e7afac5ef295ca780 3111c6cf468cc25deea8098f5cee7f8e30e5d06c cc4435c2c1ea079721f48d08d0ae3436599d1533 21f77ce93426040614aa4113d79957096af07ff2 01ed5f98913af36eff3b170921c64f2d62f23c79 6124e722c5ab94dc37dfb0908a7b9dff58586453 43c0f5ebd97225369e88418b4e957c3d592c0389 e6a1640c03c50a55ef3e00a0592dbb0851fe33bb 0598b97bcdba896968bb3a09f78ffd8618f907a7 908001d2e56d0b774036f33521a7cb2bcd146309
["Computer Science"] ["Computer Science"] ["Computer Science","Mathematics"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Medicine","Computer Science","Engineering"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Mathematics","Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Engineering","Computer Science"] ["Engineering"] ["Computer Science","Medicine"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Engineering","Computer Science"] ["Computer Science"] ["Computer Science"] ["Engineering"] ["Computer Science","Engineering"] ["Computer Science"] ["Computer Science","Mathematics"] ["Computer Science"] ["Computer Science","Mathematics"] ["Computer Science","Mathematics"] ["Computer Science"] ["Mathematics","Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science","Medicine"] ["Computer Science"] ["Computer Science"] ["Computer Science"]
2009 2005 2019 2018 2020 2015 2017 2011 2011 2001 2013 2015 2019 2017 2017 2015 2018 2017 2012 2012 1993 2014 2016 2010 2017 2012 2017 2016 2019 2020 2019 2020 2019 2017 2010 2020 2018 2005 2012 2019 2014 2013 2006 2018 2018 2019 2018 2018 2018 2017
SMC WSA ArXiv ICLR ArXiv HCI (29) IECON GLOBECOM ICCV Workshops ICML GLOBECOM IJCNN DIVANet@MSWiM ICML ICLR IJCAI
https://api.semanticscholar.org/89bb2b21aa770613fc7dc039ed8c313f6d89cf31 https://api.semanticscholar.org/40a6bb62b6194b2af220b1866bca07207cd44431 https://api.semanticscholar.org/5ec2506857e38b628a302ca3fe3ebd2f9dcc0f4d https://api.semanticscholar.org/c2508a7de42dbd356f14dd29dddf23a57731fcc9 https://api.semanticscholar.org/30c15627b2363cbfe02cd1d4215773494042ad28 https://api.semanticscholar.org/ad2d7e580b892c095ef331d0cb507c60908cd5cc https://api.semanticscholar.org/2cbe7bd874864f091e9f995c21b8fe3e488495a5 https://api.semanticscholar.org/2d011ccbde6a5b76a0a6989f00eeef06b9660ad0 https://api.semanticscholar.org/8856b84fa57232c0f988934bdd68a4646edb3ca5 https://api.semanticscholar.org/0cb24d3477e864ba56c6abb4189f846a528699e1 https://api.semanticscholar.org/7c98354991d4c2e2d4546760e3b62fc54a32d5e9 https://api.semanticscholar.org/76e89a2f87157d61355b58a88e7515daac942ddd https://api.semanticscholar.org/5cc18b5bc1d0944dfb1bf8e1825ffc8fe7c2cac2 https://www.semanticscholar.org/paper/052282998bc24db695891f755a00e3cebd3fd796 https://www.semanticscholar.org/paper/0d24a0695c9fc669e643bad51d4e14f056329dec https://www.semanticscholar.org/paper/d5ed07113ddcd038062525a5a54550c012ac9a74 https://api.semanticscholar.org/961b7a43ab3efca1096be4f33b9e6f836fffa55e https://api.semanticscholar.org/51731c180e5df22a3ed84baad1a06a4a471c5510 https://api.semanticscholar.org/cabdfdc6caeaa993872d026bc7853ab981d2031a https://api.semanticscholar.org/528778e82c6b71bcb31aad6cb449e39ca3efdc11 https://api.semanticscholar.org/d9bfdb07899f838666e475734b93e966cca9421b https://api.semanticscholar.org/41c31ce15fd931413e9a612ba16190c7e0c4400a https://api.semanticscholar.org/33331b6c1d0d75c095fc5874c29b36833dc9fae8 https://api.semanticscholar.org/e21d7b5d0aa72d6fe6ad36e16b5285a7426e31b1 https://api.semanticscholar.org/b51dc54ccea04f79480396f9f85c02e4f1e0cd97 https://api.semanticscholar.org/fb333d25881042fe2f6c2864ef8f95e86a581be9 https://api.semanticscholar.org/8f2219bb4bfe8bc1d17431b20d1118033f62d5a4 https://api.semanticscholar.org/8049049144472e58698fe32f428e3a2cd4926e1e https://api.semanticscholar.org/6118528d953800d6b7ddf0be50da5207e1d197e6 https://api.semanticscholar.org/7b825cbbfb5a647a28f4c7f7d7d8669a21b04827 https://api.semanticscholar.org/214d4d52b72a4980fd42d59dacd2a5a62bc8d691 https://api.semanticscholar.org/f643efc871beef8ecdfcb7ae00b33872f66c0a04 https://api.semanticscholar.org/302ced2dd5c9da6c9029d5b88407a15173db13f6 https://api.semanticscholar.org/ff4137be7bf273dd52e4256f167d6f1ba0124ee2 https://api.semanticscholar.org/c6ac75f1ee88bb84ebe741f5bde9d34c981a68ab https://api.semanticscholar.org/10129da014606c70b6fab077319491c772b01c04 https://www.semanticscholar.org/paper/d85623ffae865f9ef386644dd02d0ea2d6a8c8de https://api.semanticscholar.org/fc4584adbe076e5560c973eeab387c9d9abba8b7 https://api.semanticscholar.org/2a5b0f88c064147a433cd34e7afac5ef295ca780 https://api.semanticscholar.org/3111c6cf468cc25deea8098f5cee7f8e30e5d06c https://api.semanticscholar.org/cc4435c2c1ea079721f48d08d0ae3436599d1533 https://api.semanticscholar.org/21f77ce93426040614aa4113d79957096af07ff2 https://api.semanticscholar.org/01ed5f98913af36eff3b170921c64f2d62f23c79 https://api.semanticscholar.org/6124e722c5ab94dc37dfb0908a7b9dff58586453 https://api.semanticscholar.org/43c0f5ebd97225369e88418b4e957c3d592c0389 https://api.semanticscholar.org/e6a1640c03c50a55ef3e00a0592dbb0851fe33bb https://api.semanticscholar.org/0598b97bcdba896968bb3a09f78ffd8618f907a7 https://api.semanticscholar.org/908001d2e56d0b774036f33521a7cb2bcd146309 https://www.semanticscholar.org/paper/d85623ffae865f9ef386644dd02d0ea2d6a8c8de https://www.semanticscholar.org/paper/1f08598381af9146d0fd9a61b30d0e51a7331689 https://www.semanticscholar.org/paper/0f810eb4777fd05317951ebaa7a3f5835ee84cf4