{"action":"created","issue":{"url":"https://api.github.com/repos/TakuyaHiraoka/Reinforcement-Learning-in-Multi-Party-Trading-Dialog/issues/1","repository_url":"https://api.github.com/repos/TakuyaHiraoka/Reinforcement-Learning-in-Multi-Party-Trading-Dialog","labels_url":"https://api.github.com/repos/TakuyaHiraoka/Reinforcement-Learning-in-Multi-Party-Trading-Dialog/issues/1/labels{/name}","comments_url":"https://api.github.com/repos/TakuyaHiraoka/Reinforcement-Learning-in-Multi-Party-Trading-Dialog/issues/1/comments","events_url":"https://api.github.com/repos/TakuyaHiraoka/Reinforcement-Learning-in-Multi-Party-Trading-Dialog/issues/1/events","html_url":"https://github.com/TakuyaHiraoka/Reinforcement-Learning-in-Multi-Party-Trading-Dialog/issues/1","id":189962736,"number":1,"title":"AssertionError: Total number of weight in initialization is incorrect. it should be same as that of weights loaded from File. check __init__ in ActionValueNetwoek in interface.py ","user":{"login":"loveJasmine","id":18051187,"avatar_url":"https://avatars.githubusercontent.com/u/18051187?v=3","gravatar_id":"","url":"https://api.github.com/users/loveJasmine","html_url":"https://github.com/loveJasmine","followers_url":"https://api.github.com/users/loveJasmine/followers","following_url":"https://api.github.com/users/loveJasmine/following{/other_user}","gists_url":"https://api.github.com/users/loveJasmine/gists{/gist_id}","starred_url":"https://api.github.com/users/loveJasmine/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/loveJasmine/subscriptions","organizations_url":"https://api.github.com/users/loveJasmine/orgs","repos_url":"https://api.github.com/users/loveJasmine/repos","events_url":"https://api.github.com/users/loveJasmine/events{/privacy}","received_events_url":"https://api.github.com/users/loveJasmine/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2016-11-17T07:30:13Z","updated_at":"2016-11-17T11:21:50Z","closed_at":null,"body":"rzai@rzai00:/media/rzai/ai_data/Reinforcement-Learning-in-Multi-Party-Trading-Dialog$ python MDPforMultiPlayerNegotiation.py \r\nCurrent number of opponents=2\r\nCombination of Opponents=HxH\r\nProtocol for System Action is generated.\r\n20 actions are generated.\r\n['Accept', 'Keep', 'Offer_AgentLearner_Apple_Orange', 'Offer_AgentLearner_Apple_Grape', 'Offer_AgentLearner_Orange_Apple', 'Offer_AgentLearner_Orange_Grape', 'Offer_AgentLearner_Grape_Apple', 'Offer_AgentLearner_Grape_Orange', 'Offer_Agent0_Apple_Orange', 'Offer_Agent0_Apple_Grape', 'Offer_Agent0_Orange_Apple', 'Offer_Agent0_Orange_Grape', 'Offer_Agent0_Grape_Apple', 'Offer_Agent0_Grape_Orange', 'Offer_Agent1_Apple_Orange', 'Offer_Agent1_Apple_Grape', 'Offer_Agent1_Orange_Apple', 'Offer_Agent1_Orange_Grape', 'Offer_Agent1_Grape_Apple', 'Offer_Agent1_Grape_Orange']\r\nProtocol for System State is generated.\r\n27 states are generated.\r\nBias always 1\r\nAdditional dialog state is appended\r\n1.CurrentTurn\r\nFull offered/accepted table\r\nFull offered/accepted table\r\nOffer/accept History\r\nReject History\r\nOfferedFrom_TargetAgentName_itemGive_itemGiven:���O�ɂǂ�TargetAgent���畨�X(itemGive��������itemGiven�����炤)������Offer���������B\r\nNumItem_TargetAgentName_item:�e�G�[�W�F���g��Item�̌�\r\nAll shared DS\r\n['OfferedFrom_AgentLearner_Apple_Orange', 'OfferedFrom_AgentLearner_Apple_Grape', 'OfferedFrom_AgentLearner_Orange_Apple', 'OfferedFrom_AgentLearner_Orange_Grape', 'OfferedFrom_AgentLearner_Grape_Apple', 'OfferedFrom_AgentLearner_Grape_Orange', 'OfferedFrom_Agent0_Apple_Orange', 'OfferedFrom_Agent0_Apple_Grape', 'OfferedFrom_Agent0_Orange_Apple', 'OfferedFrom_Agent0_Orange_Grape', 'OfferedFrom_Agent0_Grape_Apple', 'OfferedFrom_Agent0_Grape_Orange', 'OfferedFrom_Agent1_Apple_Orange', 'OfferedFrom_Agent1_Apple_Grape', 'OfferedFrom_Agent1_Orange_Apple', 'OfferedFrom_Agent1_Orange_Grape', 'OfferedFrom_Agent1_Grape_Apple', 'OfferedFrom_Agent1_Grape_Orange', 'NumItem_AgentLearner_Apple', 'NumItem_AgentLearner_Orange', 'NumItem_AgentLearner_Grape', 'NumItem_Agent0_Apple', 'NumItem_Agent0_Orange', 'NumItem_Agent0_Grape', 'NumItem_Agent1_Apple', 'NumItem_Agent1_Orange', 'NumItem_Agent1_Grape', 'Bias', 'CurrentTurn', 'FO_AgentLearner_AgentLearner_Apple_Orange', 'Accept_AgentLearner_AgentLearner_Apple_Orange', 'FO_AgentLearner_AgentLearner_Apple_Grape', 'Accept_AgentLearner_AgentLearner_Apple_Grape', 'FO_AgentLearner_AgentLearner_Orange_Apple', 'Accept_AgentLearner_AgentLearner_Orange_Apple', 'FO_AgentLearner_AgentLearner_Orange_Grape', 'Accept_AgentLearner_AgentLearner_Orange_Grape', 'FO_AgentLearner_AgentLearner_Grape_Apple', 'Accept_AgentLearner_AgentLearner_Grape_Apple', 'FO_AgentLearner_AgentLearner_Grape_Orange', 'Accept_AgentLearner_AgentLearner_Grape_Orange', 'FO_AgentLearner_Agent0_Apple_Orange', 'Accept_AgentLearner_Agent0_Apple_Orange', 'FO_AgentLearner_Agent0_Apple_Grape', 'Accept_AgentLearner_Agent0_Apple_Grape', 'FO_AgentLearner_Agent0_Orange_Apple', 'Accept_AgentLearner_Agent0_Orange_Apple', 'FO_AgentLearner_Agent0_Orange_Grape', 'Accept_AgentLearner_Agent0_Orange_Grape', 'FO_AgentLearner_Agent0_Grape_Apple', 'Accept_AgentLearner_Agent0_Grape_Apple', 'FO_AgentLearner_Agent0_Grape_Orange', 'Accept_AgentLearner_Agent0_Grape_Orange', 'FO_AgentLearner_Agent1_Apple_Orange', 'Accept_AgentLearner_Agent1_Apple_Orange', 'FO_AgentLearner_Agent1_Apple_Grape', 'Accept_AgentLearner_Agent1_Apple_Grape', 'FO_AgentLearner_Agent1_Orange_Apple', 'Accept_AgentLearner_Agent1_Orange_Apple', 'FO_AgentLearner_Agent1_Orange_Grape', 'Accept_AgentLearner_Agent1_Orange_Grape', 'FO_AgentLearner_Agent1_Grape_Apple', 'Accept_AgentLearner_Agent1_Grape_Apple', 'FO_AgentLearner_Agent1_Grape_Orange', 'Accept_AgentLearner_Agent1_Grape_Orange', 'FO_Agent0_AgentLearner_Apple_Orange', 'Accept_Agent0_AgentLearner_Apple_Orange', 'FO_Agent0_AgentLearner_Apple_Grape', 'Accept_Agent0_AgentLearner_Apple_Grape', 'FO_Agent0_AgentLearner_Orange_Apple', 'Accept_Agent0_AgentLearner_Orange_Apple', 'FO_Agent0_AgentLearner_Orange_Grape', 'Accept_Agent0_AgentLearner_Orange_Grape', 'FO_Agent0_AgentLearner_Grape_Apple', 'Accept_Agent0_AgentLearner_Grape_Apple', 'FO_Agent0_AgentLearner_Grape_Orange', 'Accept_Agent0_AgentLearner_Grape_Orange', 'FO_Agent0_Agent0_Apple_Orange', 'Accept_Agent0_Agent0_Apple_Orange', 'FO_Agent0_Agent0_Apple_Grape', 'Accept_Agent0_Agent0_Apple_Grape', 'FO_Agent0_Agent0_Orange_Apple', 'Accept_Agent0_Agent0_Orange_Apple', 'FO_Agent0_Agent0_Orange_Grape', 'Accept_Agent0_Agent0_Orange_Grape', 'FO_Agent0_Agent0_Grape_Apple', 'Accept_Agent0_Agent0_Grape_Apple', 'FO_Agent0_Agent0_Grape_Orange', 'Accept_Agent0_Agent0_Grape_Orange', 'FO_Agent0_Agent1_Apple_Orange', 'Accept_Agent0_Agent1_Apple_Orange', 'FO_Agent0_Agent1_Apple_Grape', 'Accept_Agent0_Agent1_Apple_Grape', 'FO_Agent0_Agent1_Orange_Apple', 'Accept_Agent0_Agent1_Orange_Apple', 'FO_Agent0_Agent1_Orange_Grape', 'Accept_Agent0_Agent1_Orange_Grape', 'FO_Agent0_Agent1_Grape_Apple', 'Accept_Agent0_Agent1_Grape_Apple', 'FO_Agent0_Agent1_Grape_Orange', 'Accept_Agent0_Agent1_Grape_Orange', 'FO_Agent1_AgentLearner_Apple_Orange', 'Accept_Agent1_AgentLearner_Apple_Orange', 'FO_Agent1_AgentLearner_Apple_Grape', 'Accept_Agent1_AgentLearner_Apple_Grape', 'FO_Agent1_AgentLearner_Orange_Apple', 'Accept_Agent1_AgentLearner_Orange_Apple', 'FO_Agent1_AgentLearner_Orange_Grape', 'Accept_Agent1_AgentLearner_Orange_Grape', 'FO_Agent1_AgentLearner_Grape_Apple', 'Accept_Agent1_AgentLearner_Grape_Apple', 'FO_Agent1_AgentLearner_Grape_Orange', 'Accept_Agent1_AgentLearner_Grape_Orange', 'FO_Agent1_Agent0_Apple_Orange', 'Accept_Agent1_Agent0_Apple_Orange', 'FO_Agent1_Agent0_Apple_Grape', 'Accept_Agent1_Agent0_Apple_Grape', 'FO_Agent1_Agent0_Orange_Apple', 'Accept_Agent1_Agent0_Orange_Apple', 'FO_Agent1_Agent0_Orange_Grape', 'Accept_Agent1_Agent0_Orange_Grape', 'FO_Agent1_Agent0_Grape_Apple', 'Accept_Agent1_Agent0_Grape_Apple', 'FO_Agent1_Agent0_Grape_Orange', 'Accept_Agent1_Agent0_Grape_Orange', 'FO_Agent1_Agent1_Apple_Orange', 'Accept_Agent1_Agent1_Apple_Orange', 'FO_Agent1_Agent1_Apple_Grape', 'Accept_Agent1_Agent1_Apple_Grape', 'FO_Agent1_Agent1_Orange_Apple', 'Accept_Agent1_Agent1_Orange_Apple', 'FO_Agent1_Agent1_Orange_Grape', 'Accept_Agent1_Agent1_Orange_Grape', 'FO_Agent1_Agent1_Grape_Apple', 'Accept_Agent1_Agent1_Grape_Apple', 'FO_Agent1_Agent1_Grape_Orange', 'Accept_Agent1_Agent1_Grape_Orange', 'HistoryO_AgentLearner_AgentLearner_Apple_Orange', 'HistoryA_AgentLearner_AgentLearner_Apple_Orange', 'HistoryO_AgentLearner_AgentLearner_Apple_Grape', 'HistoryA_AgentLearner_AgentLearner_Apple_Grape', 'HistoryO_AgentLearner_AgentLearner_Orange_Apple', 'HistoryA_AgentLearner_AgentLearner_Orange_Apple', 'HistoryO_AgentLearner_AgentLearner_Orange_Grape', 'HistoryA_AgentLearner_AgentLearner_Orange_Grape', 'HistoryO_AgentLearner_AgentLearner_Grape_Apple', 'HistoryA_AgentLearner_AgentLearner_Grape_Apple', 'HistoryO_AgentLearner_AgentLearner_Grape_Orange', 'HistoryA_AgentLearner_AgentLearner_Grape_Orange', 'HistoryO_AgentLearner_Agent0_Apple_Orange', 'HistoryA_AgentLearner_Agent0_Apple_Orange', 'HistoryO_AgentLearner_Agent0_Apple_Grape', 'HistoryA_AgentLearner_Agent0_Apple_Grape', 'HistoryO_AgentLearner_Agent0_Orange_Apple', 'HistoryA_AgentLearner_Agent0_Orange_Apple', 'HistoryO_AgentLearner_Agent0_Orange_Grape', 'HistoryA_AgentLearner_Agent0_Orange_Grape', 'HistoryO_AgentLearner_Agent0_Grape_Apple', 'HistoryA_AgentLearner_Agent0_Grape_Apple', 'HistoryO_AgentLearner_Agent0_Grape_Orange', 'HistoryA_AgentLearner_Agent0_Grape_Orange', 'HistoryO_AgentLearner_Agent1_Apple_Orange', 'HistoryA_AgentLearner_Agent1_Apple_Orange', 'HistoryO_AgentLearner_Agent1_Apple_Grape', 'HistoryA_AgentLearner_Agent1_Apple_Grape', 'HistoryO_AgentLearner_Agent1_Orange_Apple', 'HistoryA_AgentLearner_Agent1_Orange_Apple', 'HistoryO_AgentLearner_Agent1_Orange_Grape', 'HistoryA_AgentLearner_Agent1_Orange_Grape', 'HistoryO_AgentLearner_Agent1_Grape_Apple', 'HistoryA_AgentLearner_Agent1_Grape_Apple', 'HistoryO_AgentLearner_Agent1_Grape_Orange', 'HistoryA_AgentLearner_Agent1_Grape_Orange', 'HistoryO_Agent0_AgentLearner_Apple_Orange', 'HistoryA_Agent0_AgentLearner_Apple_Orange', 'HistoryO_Agent0_AgentLearner_Apple_Grape', 'HistoryA_Agent0_AgentLearner_Apple_Grape', 'HistoryO_Agent0_AgentLearner_Orange_Apple', 'HistoryA_Agent0_AgentLearner_Orange_Apple', 'HistoryO_Agent0_AgentLearner_Orange_Grape', 'HistoryA_Agent0_AgentLearner_Orange_Grape', 'HistoryO_Agent0_AgentLearner_Grape_Apple', 'HistoryA_Agent0_AgentLearner_Grape_Apple', 'HistoryO_Agent0_AgentLearner_Grape_Orange', 'HistoryA_Agent0_AgentLearner_Grape_Orange', 'HistoryO_Agent0_Agent0_Apple_Orange', 'HistoryA_Agent0_Agent0_Apple_Orange', 'HistoryO_Agent0_Agent0_Apple_Grape', 'HistoryA_Agent0_Agent0_Apple_Grape', 'HistoryO_Agent0_Agent0_Orange_Apple', 'HistoryA_Agent0_Agent0_Orange_Apple', 'HistoryO_Agent0_Agent0_Orange_Grape', 'HistoryA_Agent0_Agent0_Orange_Grape', 'HistoryO_Agent0_Agent0_Grape_Apple', 'HistoryA_Agent0_Agent0_Grape_Apple', 'HistoryO_Agent0_Agent0_Grape_Orange', 'HistoryA_Agent0_Agent0_Grape_Orange', 'HistoryO_Agent0_Agent1_Apple_Orange', 'HistoryA_Agent0_Agent1_Apple_Orange', 'HistoryO_Agent0_Agent1_Apple_Grape', 'HistoryA_Agent0_Agent1_Apple_Grape', 'HistoryO_Agent0_Agent1_Orange_Apple', 'HistoryA_Agent0_Agent1_Orange_Apple', 'HistoryO_Agent0_Agent1_Orange_Grape', 'HistoryA_Agent0_Agent1_Orange_Grape', 'HistoryO_Agent0_Agent1_Grape_Apple', 'HistoryA_Agent0_Agent1_Grape_Apple', 'HistoryO_Agent0_Agent1_Grape_Orange', 'HistoryA_Agent0_Agent1_Grape_Orange', 'HistoryO_Agent1_AgentLearner_Apple_Orange', 'HistoryA_Agent1_AgentLearner_Apple_Orange', 'HistoryO_Agent1_AgentLearner_Apple_Grape', 'HistoryA_Agent1_AgentLearner_Apple_Grape', 'HistoryO_Agent1_AgentLearner_Orange_Apple', 'HistoryA_Agent1_AgentLearner_Orange_Apple', 'HistoryO_Agent1_AgentLearner_Orange_Grape', 'HistoryA_Agent1_AgentLearner_Orange_Grape', 'HistoryO_Agent1_AgentLearner_Grape_Apple', 'HistoryA_Agent1_AgentLearner_Grape_Apple', 'HistoryO_Agent1_AgentLearner_Grape_Orange', 'HistoryA_Agent1_AgentLearner_Grape_Orange', 'HistoryO_Agent1_Agent0_Apple_Orange', 'HistoryA_Agent1_Agent0_Apple_Orange', 'HistoryO_Agent1_Agent0_Apple_Grape', 'HistoryA_Agent1_Agent0_Apple_Grape', 'HistoryO_Agent1_Agent0_Orange_Apple', 'HistoryA_Agent1_Agent0_Orange_Apple', 'HistoryO_Agent1_Agent0_Orange_Grape', 'HistoryA_Agent1_Agent0_Orange_Grape', 'HistoryO_Agent1_Agent0_Grape_Apple', 'HistoryA_Agent1_Agent0_Grape_Apple', 'HistoryO_Agent1_Agent0_Grape_Orange', 'HistoryA_Agent1_Agent0_Grape_Orange', 'HistoryO_Agent1_Agent1_Apple_Orange', 'HistoryA_Agent1_Agent1_Apple_Orange', 'HistoryO_Agent1_Agent1_Apple_Grape', 'HistoryA_Agent1_Agent1_Apple_Grape', 'HistoryO_Agent1_Agent1_Orange_Apple', 'HistoryA_Agent1_Agent1_Orange_Apple', 'HistoryO_Agent1_Agent1_Orange_Grape', 'HistoryA_Agent1_Agent1_Orange_Grape', 'HistoryO_Agent1_Agent1_Grape_Apple', 'HistoryA_Agent1_Agent1_Grape_Apple', 'HistoryO_Agent1_Agent1_Grape_Orange', 'HistoryA_Agent1_Agent1_Grape_Orange', 'Rejected_AgentLearner_Agent0_Apple_Orange', 'Rejected_AgentLearner_Agent0_Apple_Grape', 'Rejected_AgentLearner_Agent0_Orange_Apple', 'Rejected_AgentLearner_Agent0_Orange_Grape', 'Rejected_AgentLearner_Agent0_Grape_Apple', 'Rejected_AgentLearner_Agent0_Grape_Orange', 'Rejected_AgentLearner_Agent1_Apple_Orange', 'Rejected_AgentLearner_Agent1_Apple_Grape', 'Rejected_AgentLearner_Agent1_Orange_Apple', 'Rejected_AgentLearner_Agent1_Orange_Grape', 'Rejected_AgentLearner_Agent1_Grape_Apple', 'Rejected_AgentLearner_Agent1_Grape_Orange', 'Rejected_Agent0_AgentLearner_Apple_Orange', 'Rejected_Agent0_AgentLearner_Apple_Grape', 'Rejected_Agent0_AgentLearner_Orange_Apple', 'Rejected_Agent0_AgentLearner_Orange_Grape', 'Rejected_Agent0_AgentLearner_Grape_Apple', 'Rejected_Agent0_AgentLearner_Grape_Orange', 'Rejected_Agent0_Agent1_Apple_Orange', 'Rejected_Agent0_Agent1_Apple_Grape', 'Rejected_Agent0_Agent1_Orange_Apple', 'Rejected_Agent0_Agent1_Orange_Grape', 'Rejected_Agent0_Agent1_Grape_Apple', 'Rejected_Agent0_Agent1_Grape_Orange', 'Rejected_Agent1_AgentLearner_Apple_Orange', 'Rejected_Agent1_AgentLearner_Apple_Grape', 'Rejected_Agent1_AgentLearner_Orange_Apple', 'Rejected_Agent1_AgentLearner_Orange_Grape', 'Rejected_Agent1_AgentLearner_Grape_Apple', 'Rejected_Agent1_AgentLearner_Grape_Orange', 'Rejected_Agent1_Agent0_Apple_Orange', 'Rejected_Agent1_Agent0_Apple_Grape', 'Rejected_Agent1_Agent0_Orange_Apple', 'Rejected_Agent1_Agent0_Orange_Grape', 'Rejected_Agent1_Agent0_Grape_Apple', 'Rejected_Agent1_Agent0_Grape_Orange']\r\nDS for learner\r\n['OfferedFrom_Agent0_Apple_Orange', 'OfferedFrom_Agent0_Apple_Grape', 'OfferedFrom_Agent0_Orange_Apple', 'OfferedFrom_Agent0_Orange_Grape', 'OfferedFrom_Agent0_Grape_Apple', 'OfferedFrom_Agent0_Grape_Orange', 'OfferedFrom_Agent1_Apple_Orange', 'OfferedFrom_Agent1_Apple_Grape', 'OfferedFrom_Agent1_Orange_Apple', 'OfferedFrom_Agent1_Orange_Grape', 'OfferedFrom_Agent1_Grape_Apple', 'OfferedFrom_Agent1_Grape_Orange', 'NumItem_AgentLearner_Apple', 'NumItem_AgentLearner_Orange', 'NumItem_AgentLearner_Grape', 'NumItem_Agent0_Apple', 'NumItem_Agent0_Orange', 'NumItem_Agent0_Grape', 'NumItem_Agent1_Apple', 'NumItem_Agent1_Orange', 'NumItem_Agent1_Grape', 'Bias']\r\nLearner is using NFQ\r\nStatistics of learner is calculated.\r\nAction for learner\r\n['Accept', 'Keep', 'Offer_Agent0_Apple_Orange', 'Offer_Agent0_Apple_Grape', 'Offer_Agent0_Orange_Apple', 'Offer_Agent0_Orange_Grape', 'Offer_Agent0_Grape_Apple', 'Offer_Agent0_Grape_Orange', 'Offer_Agent1_Apple_Orange', 'Offer_Agent1_Apple_Grape', 'Offer_Agent1_Orange_Apple', 'Offer_Agent1_Orange_Grape', 'Offer_Agent1_Grape_Apple', 'Offer_Agent1_Grape_Orange']\r\nMaximum turn=20\r\nTraceback (most recent call last):\r\n  File \"MDPforMultiPlayerNegotiation.py\", line 1575, in <module>\r\n    assert numInitialWeight== len(weight), \"Total number of weight in initialization is incorrect. it should be same as that of weights loaded from File. check __init__ in ActionValueNetwoek in interface.py \"\r\nAssertionError: Total number of weight in initialization is incorrect. it should be same as that of weights loaded from File. check __init__ in ActionValueNetwoek in interface.py \r\nrzai@rzai00:/media/rzai/ai_data/Reinforcement-Learning-in-Multi-Party-Trading-Dialog$ \r\n"},"comment":{"url":"https://api.github.com/repos/TakuyaHiraoka/Reinforcement-Learning-in-Multi-Party-Trading-Dialog/issues/comments/261222421","html_url":"https://github.com/TakuyaHiraoka/Reinforcement-Learning-in-Multi-Party-Trading-Dialog/issues/1#issuecomment-261222421","issue_url":"https://api.github.com/repos/TakuyaHiraoka/Reinforcement-Learning-in-Multi-Party-Trading-Dialog/issues/1","id":261222421,"user":{"login":"TakuyaHiraoka","id":13440417,"avatar_url":"https://avatars.githubusercontent.com/u/13440417?v=3","gravatar_id":"","url":"https://api.github.com/users/TakuyaHiraoka","html_url":"https://github.com/TakuyaHiraoka","followers_url":"https://api.github.com/users/TakuyaHiraoka/followers","following_url":"https://api.github.com/users/TakuyaHiraoka/following{/other_user}","gists_url":"https://api.github.com/users/TakuyaHiraoka/gists{/gist_id}","starred_url":"https://api.github.com/users/TakuyaHiraoka/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/TakuyaHiraoka/subscriptions","organizations_url":"https://api.github.com/users/TakuyaHiraoka/orgs","repos_url":"https://api.github.com/users/TakuyaHiraoka/repos","events_url":"https://api.github.com/users/TakuyaHiraoka/events{/privacy}","received_events_url":"https://api.github.com/users/TakuyaHiraoka/received_events","type":"User","site_admin":false},"created_at":"2016-11-17T11:21:50Z","updated_at":"2016-11-17T11:21:50Z","body":"Hi, \r\n\r\nThank you so much for your comments. \r\nIt would help for me to improve source codes. \r\n\r\nRegarding to  this issue, you need edit \"interface.py\" in pybrain library (i.e., \"pybrain/rl/learners/valuebased/interface.py\")\r\n\r\nMore concretely, \"ActionValueNetwork\" class in \"interface.py\" need be changed as follows:\r\n\r\n\r\n    \"\"\" A network that approximates action values for continuous state /\r\n        discrete action RL environments. To receive the maximum action\r\n        for a given state, a forward pass is executed for all discrete\r\n        actions, and the maximal action is returned. This network is used\r\n        for the NFQ algorithm. \"\"\"\r\n\r\n    def __init__(self, dimState, numActions, name=None):\r\n        Module.__init__(self, dimState, 1, name)\r\n        self.network = buildNetwork(dimState + numActions, 100*(dimState + numActions), 1)\r\n        self.numActions = numActions\r\n\r\n    def _forwardImplementation(self, inbuf, outbuf):\r\n        \"\"\" takes the state vector and return the discrete action with\r\n            the maximum value over all actions for this state.\r\n        \"\"\"\r\n        outbuf[0] = self.getMaxAction(asarray(inbuf))\r\n\r\n    def getMaxAction(self, state):\r\n        \"\"\" Return the action with the maximal value for the given state. \"\"\"\r\n        return argmax(self.getActionValues(state))\r\n\r\n    def getActionValues(self, state):\r\n        \"\"\" Run forward activation for each of the actions and returns all values. \"\"\"\r\n        values = array([self.network.activate(r_[state, one_to_n(i, self.numActions)]) for i in range(self.numActions)])\r\n        return values\r\n\r\n    def getValue(self, state, action):\r\n        return self.network.activate(r_[state, one_to_n(action, self.numActions)])\r\n\r\n"}}
{"actor":{"display_login":"TakuyaHiraoka"}}