{"action":"created","comment":{"url":"https://api.github.com/repos/marl-ieee-nitk/marl-ieee-nitk.github.io/pulls/comments/241678038","pull_request_review_id":185008400,"id":241678038,"node_id":"MDI0OlB1bGxSZXF1ZXN0UmV2aWV3Q29tbWVudDI0MTY3ODAzOA==","diff_hunk":"@@ -0,0 +1,288 @@\n+---\n+layout: post\n+title:  \"Introduction to Q-learning!\"\n+date:   2018-12-11 11:42:04 +0530\n+categories: jekyll update\n+author: Madhuparna Bhowmik\n+---\n+\n+In this article, we will learn about Q-learning and implement the algorithm from scratch. We will also make our own small board game to test the algorithm.\n+### **What is Q-learning**\n+\n+Q-learning is an off-policy and model-free Reinforcement Learning algorithm.\n+Model-free means that we need not know the details of the environment i.e how and with what probability the next state is being generated, given current state and action. And off-policy means action-value function, Q, directly approximates the optimal action-value function, independent of the policy being followed.\n+\n+### **Setting up the environment**\n+\n+Let us start by defining a simple problem and then we will see how it can be solved using Q-learning.\n+Consider a simple board game, a $$3\\times3$$ board with a Start cell and a Goal cell. We can also add a blocked cell to our board. Now, the goal is to learn a path from Start cell represented by S to Goal Cell represented by G without going into the blocked cell X.\n+\n+![Game](/assets/game.jpg){:height=\"50%\" width=\"50%\"}\n+\n+The possible actions from each state are:\n+\n+1.UP\n+\n+2.DOWN\n+\n+3.RIGHT\n+\n+4.LEFT\n+\n+\n+Let's set the rewards now,\n+\n+1.A reward of +10 to successfully reach the Goal(G).\n+\n+2.A reward of -10 when it reaches the blocked state.\n+\n+3.A reward of -1 for all other actions.\n+\n+#### Why do you think we define the rewards in this way?\n+\n+Well +10 for the Goal obviously because that is a desirable state and our algorithm aims at maximizing the reward, so it should get a good amount of reward for completing the task.\n+\n+And we don't want to bump into those blocked states so we set the reward for those states to be -10 so that our algorithm learns to not go to those states.\n+\n+And a reward of -1 for taking any other action because we want to find the shortest way from the Start position to the Goal, we don't want to just wander around unnecessarily. But what if we set the reward to +1 for every successful step?\n+\n+Well then our algorithm will learn to maximize the reward just by wandering around that is now it can keep moving between cell (0,1) and cell (1,0) and get +1 for every step and thus maximizing the reward without reaching the Goal!\n+But when the reward is -1 it is being punished for every extra step and thus it will learn to take the shortest path from Start to Goal.\n+Therefore choosing rewards carefully is important.\n+\n+\n+Let's write the code for this game\n+\n+{% highlight ruby %}","path":"_posts/2018-12-11-Q-learning.markdown","position":56,"original_position":56,"commit_id":"406f9dda2642d2c594c21d6f6b43845c7c027450","original_commit_id":"406f9dda2642d2c594c21d6f6b43845c7c027450","user":{"login":"MJ10","id":4126775,"node_id":"MDQ6VXNlcjQxMjY3NzU=","avatar_url":"https://avatars0.githubusercontent.com/u/4126775?v=4","gravatar_id":"","url":"https://api.github.com/users/MJ10","html_url":"https://github.com/MJ10","followers_url":"https://api.github.com/users/MJ10/followers","following_url":"https://api.github.com/users/MJ10/following{/other_user}","gists_url":"https://api.github.com/users/MJ10/gists{/gist_id}","starred_url":"https://api.github.com/users/MJ10/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/MJ10/subscriptions","organizations_url":"https://api.github.com/users/MJ10/orgs","repos_url":"https://api.github.com/users/MJ10/repos","events_url":"https://api.github.com/users/MJ10/events{/privacy}","received_events_url":"https://api.github.com/users/MJ10/received_events","type":"User","site_admin":false},"body":"Should be python instead of ruby.","created_at":"2018-12-14T08:41:51Z","updated_at":"2018-12-14T08:46:48Z","html_url":"https://github.com/marl-ieee-nitk/marl-ieee-nitk.github.io/pull/2#discussion_r241678038","pull_request_url":"https://api.github.com/repos/marl-ieee-nitk/marl-ieee-nitk.github.io/pulls/2","author_association":"CONTRIBUTOR","_links":{"self":{"href":"https://api.github.com/repos/marl-ieee-nitk/marl-ieee-nitk.github.io/pulls/comments/241678038"},"html":{"href":"https://github.com/marl-ieee-nitk/marl-ieee-nitk.github.io/pull/2#discussion_r241678038"},"pull_request":{"href":"https://api.github.com/repos/marl-ieee-nitk/marl-ieee-nitk.github.io/pulls/2"}}},"pull_request":{"url":"https://api.github.com/repos/marl-ieee-nitk/marl-ieee-nitk.github.io/pulls/2","id":238639395,"node_id":"MDExOlB1bGxSZXF1ZXN0MjM4NjM5Mzk1","html_url":"https://github.com/marl-ieee-nitk/marl-ieee-nitk.github.io/pull/2","diff_url":"https://github.com/marl-ieee-nitk/marl-ieee-nitk.github.io/pull/2.diff","patch_url":"https://github.com/marl-ieee-nitk/marl-ieee-nitk.github.io/pull/2.patch","issue_url":"https://api.github.com/repos/marl-ieee-nitk/marl-ieee-nitk.github.io/issues/2","number":2,"state":"open","locked":false,"title":"Q-learning blog added","user":{"login":"Madhuparna04","id":32368658,"node_id":"MDQ6VXNlcjMyMzY4NjU4","avatar_url":"https://avatars1.githubusercontent.com/u/32368658?v=4","gravatar_id":"","url":"https://api.github.com/users/Madhuparna04","html_url":"https://github.com/Madhuparna04","followers_url":"https://api.github.com/users/Madhuparna04/followers","following_url":"https://api.github.com/users/Madhuparna04/following{/other_user}","gists_url":"https://api.github.com/users/Madhuparna04/gists{/gist_id}","starred_url":"https://api.github.com/users/Madhuparna04/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Madhuparna04/subscriptions","organizations_url":"https://api.github.com/users/Madhuparna04/orgs","repos_url":"https://api.github.com/users/Madhuparna04/repos","events_url":"https://api.github.com/users/Madhuparna04/events{/privacy}","received_events_url":"https://api.github.com/users/Madhuparna04/received_events","type":"User","site_admin":false},"body":"","created_at":"2018-12-14T08:03:32Z","updated_at":"2018-12-14T08:46:48Z","closed_at":null,"merged_at":null,"merge_commit_sha":"25b92c4dda19a4102f5c5e3e53439d2b1fea6a80","assignee":null,"assignees":[],"requested_reviewers":[],"requested_teams":[],"labels":[],"milestone":null,"commits_url":"https://api.github.com/repos/marl-ieee-nitk/marl-ieee-nitk.github.io/pulls/2/commits","review_comments_url":"https://api.github.com/repos/marl-ieee-nitk/marl-ieee-nitk.github.io/pulls/2/comments","review_comment_url":"https://api.github.com/repos/marl-ieee-nitk/marl-ieee-nitk.github.io/pulls/comments{/number}","comments_url":"https://api.github.com/repos/marl-ieee-nitk/marl-ieee-nitk.github.io/issues/2/comments","statuses_url":"https://api.github.com/repos/marl-ieee-nitk/marl-ieee-nitk.github.io/statuses/406f9dda2642d2c594c21d6f6b43845c7c027450","head":{"label":"Madhuparna04:master","ref":"master","sha":"406f9dda2642d2c594c21d6f6b43845c7c027450","user":{"login":"Madhuparna04","id":32368658,"node_id":"MDQ6VXNlcjMyMzY4NjU4","avatar_url":"https://avatars1.githubusercontent.com/u/32368658?v=4","gravatar_id":"","url":"https://api.github.com/users/Madhuparna04","html_url":"https://github.com/Madhuparna04","followers_url":"https://api.github.com/users/Madhuparna04/followers","following_url":"https://api.github.com/users/Madhuparna04/following{/other_user}","gists_url":"https://api.github.com/users/Madhuparna04/gists{/gist_id}","starred_url":"https://api.github.com/users/Madhuparna04/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Madhuparna04/subscriptions","organizations_url":"https://api.github.com/users/Madhuparna04/orgs","repos_url":"https://api.github.com/users/Madhuparna04/repos","events_url":"https://api.github.com/users/Madhuparna04/events{/privacy}","received_events_url":"https://api.github.com/users/Madhuparna04/received_events","type":"User","site_admin":false},"repo":{"id":161751211,"node_id":"MDEwOlJlcG9zaXRvcnkxNjE3NTEyMTE=","name":"marl-ieee-nitk.github.io","full_name":"Madhuparna04/marl-ieee-nitk.github.io","private":false,"owner":{"login":"Madhuparna04","id":32368658,"node_id":"MDQ6VXNlcjMyMzY4NjU4","avatar_url":"https://avatars1.githubusercontent.com/u/32368658?v=4","gravatar_id":"","url":"https://api.github.com/users/Madhuparna04","html_url":"https://github.com/Madhuparna04","followers_url":"https://api.github.com/users/Madhuparna04/followers","following_url":"https://api.github.com/users/Madhuparna04/following{/other_user}","gists_url":"https://api.github.com/users/Madhuparna04/gists{/gist_id}","starred_url":"https://api.github.com/users/Madhuparna04/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Madhuparna04/subscriptions","organizations_url":"https://api.github.com/users/Madhuparna04/orgs","repos_url":"https://api.github.com/users/Madhuparna04/repos","events_url":"https://api.github.com/users/Madhuparna04/events{/privacy}","received_events_url":"https://api.github.com/users/Madhuparna04/received_events","type":"User","site_admin":false},"html_url":"https://github.com/Madhuparna04/marl-ieee-nitk.github.io","description":null,"fork":true,"url":"https://api.github.com/repos/Madhuparna04/marl-ieee-nitk.github.io","forks_url":"https://api.github.com/repos/Madhuparna04/marl-ieee-nitk.github.io/forks","keys_url":"https://api.github.com/repos/Madhuparna04/marl-ieee-nitk.github.io/keys{/key_id}","collaborators_url":"https://api.github.com/repos/Madhuparna04/marl-ieee-nitk.github.io/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/Madhuparna04/marl-ieee-nitk.github.io/teams","hooks_url":"https://api.github.com/repos/Madhuparna04/marl-ieee-nitk.github.io/hooks","issue_events_url":"https://api.github.com/repos/Madhuparna04/marl-ieee-nitk.github.io/issues/events{/number}","events_url":"https://api.github.com/repos/Madhuparna04/marl-ieee-nitk.github.io/events","assignees_url":"https://api.github.com/repos/Madhuparna04/marl-ieee-nitk.github.io/assignees{/user}","branches_url":"https://api.github.com/repos/Madhuparna04/marl-ieee-nitk.github.io/branches{/branch}","tags_url":"https://api.github.com/repos/Madhuparna04/marl-ieee-nitk.github.io/tags","blobs_url":"https://api.github.com/repos/Madhuparna04/marl-ieee-nitk.github.io/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/Madhuparna04/marl-ieee-nitk.github.io/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/Madhuparna04/marl-ieee-nitk.github.io/git/refs{/sha}","trees_url":"https://api.github.com/repos/Madhuparna04/marl-ieee-nitk.github.io/git/trees{/sha}","statuses_url":"https://api.github.com/repos/Madhuparna04/marl-ieee-nitk.github.io/statuses/{sha}","languages_url":"https://api.github.com/repos/Madhuparna04/marl-ieee-nitk.github.io/languages","stargazers_url":"https://api.github.com/repos/Madhuparna04/marl-ieee-nitk.github.io/stargazers","contributors_url":"https://api.github.com/repos/Madhuparna04/marl-ieee-nitk.github.io/contributors","subscribers_url":"https://api.github.com/repos/Madhuparna04/marl-ieee-nitk.github.io/subscribers","subscription_url":"https://api.github.com/repos/Madhuparna04/marl-ieee-nitk.github.io/subscription","commits_url":"https://api.github.com/repos/Madhuparna04/marl-ieee-nitk.github.io/commits{/sha}","git_commits_url":"https://api.github.com/repos/Madhuparna04/marl-ieee-nitk.github.io/git/commits{/sha}","comments_url":"https://api.github.com/repos/Madhuparna04/marl-ieee-nitk.github.io/comments{/number}","issue_comment_url":"https://api.github.com/repos/Madhuparna04/marl-ieee-nitk.github.io/issues/comments{/number}","contents_url":"https://api.github.com/repos/Madhuparna04/marl-ieee-nitk.github.io/contents/{+path}","compare_url":"https://api.github.com/repos/Madhuparna04/marl-ieee-nitk.github.io/compare/{base}...{head}","merges_url":"https://api.github.com/repos/Madhuparna04/marl-ieee-nitk.github.io/merges","archive_url":"https://api.github.com/repos/Madhuparna04/marl-ieee-nitk.github.io/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/Madhuparna04/marl-ieee-nitk.github.io/downloads","issues_url":"https://api.github.com/repos/Madhuparna04/marl-ieee-nitk.github.io/issues{/number}","pulls_url":"https://api.github.com/repos/Madhuparna04/marl-ieee-nitk.github.io/pulls{/number}","milestones_url":"https://api.github.com/repos/Madhuparna04/marl-ieee-nitk.github.io/milestones{/number}","notifications_url":"https://api.github.com/repos/Madhuparna04/marl-ieee-nitk.github.io/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/Madhuparna04/marl-ieee-nitk.github.io/labels{/name}","releases_url":"https://api.github.com/repos/Madhuparna04/marl-ieee-nitk.github.io/releases{/id}","deployments_url":"https://api.github.com/repos/Madhuparna04/marl-ieee-nitk.github.io/deployments","created_at":"2018-12-14T07:58:21Z","updated_at":"2018-12-14T08:02:27Z","pushed_at":"2018-12-14T08:02:25Z","git_url":"git://github.com/Madhuparna04/marl-ieee-nitk.github.io.git","ssh_url":"git@github.com:Madhuparna04/marl-ieee-nitk.github.io.git","clone_url":"https://github.com/Madhuparna04/marl-ieee-nitk.github.io.git","svn_url":"https://github.com/Madhuparna04/marl-ieee-nitk.github.io","homepage":null,"size":18,"stargazers_count":0,"watchers_count":0,"language":"CSS","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"archived":false,"open_issues_count":0,"license":null,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"marl-ieee-nitk:master","ref":"master","sha":"eccad337318d34c3cda5eeaa4a02988a8398bd73","user":{"login":"marl-ieee-nitk","id":45727010,"node_id":"MDEyOk9yZ2FuaXphdGlvbjQ1NzI3MDEw","avatar_url":"https://avatars2.githubusercontent.com/u/45727010?v=4","gravatar_id":"","url":"https://api.github.com/users/marl-ieee-nitk","html_url":"https://github.com/marl-ieee-nitk","followers_url":"https://api.github.com/users/marl-ieee-nitk/followers","following_url":"https://api.github.com/users/marl-ieee-nitk/following{/other_user}","gists_url":"https://api.github.com/users/marl-ieee-nitk/gists{/gist_id}","starred_url":"https://api.github.com/users/marl-ieee-nitk/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/marl-ieee-nitk/subscriptions","organizations_url":"https://api.github.com/users/marl-ieee-nitk/orgs","repos_url":"https://api.github.com/users/marl-ieee-nitk/repos","events_url":"https://api.github.com/users/marl-ieee-nitk/events{/privacy}","received_events_url":"https://api.github.com/users/marl-ieee-nitk/received_events","type":"Organization","site_admin":false},"repo":{"id":161008788,"node_id":"MDEwOlJlcG9zaXRvcnkxNjEwMDg3ODg=","name":"marl-ieee-nitk.github.io","full_name":"marl-ieee-nitk/marl-ieee-nitk.github.io","private":false,"owner":{"login":"marl-ieee-nitk","id":45727010,"node_id":"MDEyOk9yZ2FuaXphdGlvbjQ1NzI3MDEw","avatar_url":"https://avatars2.githubusercontent.com/u/45727010?v=4","gravatar_id":"","url":"https://api.github.com/users/marl-ieee-nitk","html_url":"https://github.com/marl-ieee-nitk","followers_url":"https://api.github.com/users/marl-ieee-nitk/followers","following_url":"https://api.github.com/users/marl-ieee-nitk/following{/other_user}","gists_url":"https://api.github.com/users/marl-ieee-nitk/gists{/gist_id}","starred_url":"https://api.github.com/users/marl-ieee-nitk/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/marl-ieee-nitk/subscriptions","organizations_url":"https://api.github.com/users/marl-ieee-nitk/orgs","repos_url":"https://api.github.com/users/marl-ieee-nitk/repos","events_url":"https://api.github.com/users/marl-ieee-nitk/events{/privacy}","received_events_url":"https://api.github.com/users/marl-ieee-nitk/received_events","type":"Organization","site_admin":false},"html_url":"https://github.com/marl-ieee-nitk/marl-ieee-nitk.github.io","description":null,"fork":false,"url":"https://api.github.com/repos/marl-ieee-nitk/marl-ieee-nitk.github.io","forks_url":"https://api.github.com/repos/marl-ieee-nitk/marl-ieee-nitk.github.io/forks","keys_url":"https://api.github.com/repos/marl-ieee-nitk/marl-ieee-nitk.github.io/keys{/key_id}","collaborators_url":"https://api.github.com/repos/marl-ieee-nitk/marl-ieee-nitk.github.io/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/marl-ieee-nitk/marl-ieee-nitk.github.io/teams","hooks_url":"https://api.github.com/repos/marl-ieee-nitk/marl-ieee-nitk.github.io/hooks","issue_events_url":"https://api.github.com/repos/marl-ieee-nitk/marl-ieee-nitk.github.io/issues/events{/number}","events_url":"https://api.github.com/repos/marl-ieee-nitk/marl-ieee-nitk.github.io/events","assignees_url":"https://api.github.com/repos/marl-ieee-nitk/marl-ieee-nitk.github.io/assignees{/user}","branches_url":"https://api.github.com/repos/marl-ieee-nitk/marl-ieee-nitk.github.io/branches{/branch}","tags_url":"https://api.github.com/repos/marl-ieee-nitk/marl-ieee-nitk.github.io/tags","blobs_url":"https://api.github.com/repos/marl-ieee-nitk/marl-ieee-nitk.github.io/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/marl-ieee-nitk/marl-ieee-nitk.github.io/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/marl-ieee-nitk/marl-ieee-nitk.github.io/git/refs{/sha}","trees_url":"https://api.github.com/repos/marl-ieee-nitk/marl-ieee-nitk.github.io/git/trees{/sha}","statuses_url":"https://api.github.com/repos/marl-ieee-nitk/marl-ieee-nitk.github.io/statuses/{sha}","languages_url":"https://api.github.com/repos/marl-ieee-nitk/marl-ieee-nitk.github.io/languages","stargazers_url":"https://api.github.com/repos/marl-ieee-nitk/marl-ieee-nitk.github.io/stargazers","contributors_url":"https://api.github.com/repos/marl-ieee-nitk/marl-ieee-nitk.github.io/contributors","subscribers_url":"https://api.github.com/repos/marl-ieee-nitk/marl-ieee-nitk.github.io/subscribers","subscription_url":"https://api.github.com/repos/marl-ieee-nitk/marl-ieee-nitk.github.io/subscription","commits_url":"https://api.github.com/repos/marl-ieee-nitk/marl-ieee-nitk.github.io/commits{/sha}","git_commits_url":"https://api.github.com/repos/marl-ieee-nitk/marl-ieee-nitk.github.io/git/commits{/sha}","comments_url":"https://api.github.com/repos/marl-ieee-nitk/marl-ieee-nitk.github.io/comments{/number}","issue_comment_url":"https://api.github.com/repos/marl-ieee-nitk/marl-ieee-nitk.github.io/issues/comments{/number}","contents_url":"https://api.github.com/repos/marl-ieee-nitk/marl-ieee-nitk.github.io/contents/{+path}","compare_url":"https://api.github.com/repos/marl-ieee-nitk/marl-ieee-nitk.github.io/compare/{base}...{head}","merges_url":"https://api.github.com/repos/marl-ieee-nitk/marl-ieee-nitk.github.io/merges","archive_url":"https://api.github.com/repos/marl-ieee-nitk/marl-ieee-nitk.github.io/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/marl-ieee-nitk/marl-ieee-nitk.github.io/downloads","issues_url":"https://api.github.com/repos/marl-ieee-nitk/marl-ieee-nitk.github.io/issues{/number}","pulls_url":"https://api.github.com/repos/marl-ieee-nitk/marl-ieee-nitk.github.io/pulls{/number}","milestones_url":"https://api.github.com/repos/marl-ieee-nitk/marl-ieee-nitk.github.io/milestones{/number}","notifications_url":"https://api.github.com/repos/marl-ieee-nitk/marl-ieee-nitk.github.io/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/marl-ieee-nitk/marl-ieee-nitk.github.io/labels{/name}","releases_url":"https://api.github.com/repos/marl-ieee-nitk/marl-ieee-nitk.github.io/releases{/id}","deployments_url":"https://api.github.com/repos/marl-ieee-nitk/marl-ieee-nitk.github.io/deployments","created_at":"2018-12-09T05:47:21Z","updated_at":"2018-12-09T05:52:33Z","pushed_at":"2018-12-14T08:03:32Z","git_url":"git://github.com/marl-ieee-nitk/marl-ieee-nitk.github.io.git","ssh_url":"git@github.com:marl-ieee-nitk/marl-ieee-nitk.github.io.git","clone_url":"https://github.com/marl-ieee-nitk/marl-ieee-nitk.github.io.git","svn_url":"https://github.com/marl-ieee-nitk/marl-ieee-nitk.github.io","homepage":null,"size":18,"stargazers_count":0,"watchers_count":0,"language":"CSS","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":true,"forks_count":2,"mirror_url":null,"archived":false,"open_issues_count":1,"license":null,"forks":2,"open_issues":1,"watchers":0,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/marl-ieee-nitk/marl-ieee-nitk.github.io/pulls/2"},"html":{"href":"https://github.com/marl-ieee-nitk/marl-ieee-nitk.github.io/pull/2"},"issue":{"href":"https://api.github.com/repos/marl-ieee-nitk/marl-ieee-nitk.github.io/issues/2"},"comments":{"href":"https://api.github.com/repos/marl-ieee-nitk/marl-ieee-nitk.github.io/issues/2/comments"},"review_comments":{"href":"https://api.github.com/repos/marl-ieee-nitk/marl-ieee-nitk.github.io/pulls/2/comments"},"review_comment":{"href":"https://api.github.com/repos/marl-ieee-nitk/marl-ieee-nitk.github.io/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/marl-ieee-nitk/marl-ieee-nitk.github.io/pulls/2/commits"},"statuses":{"href":"https://api.github.com/repos/marl-ieee-nitk/marl-ieee-nitk.github.io/statuses/406f9dda2642d2c594c21d6f6b43845c7c027450"}},"author_association":"NONE"}}
{"actor":{"display_login":"MJ10"}}