{"action":"created","issue":{"url":"https://api.github.com/repos/openai/gym/issues/71","repository_url":"https://api.github.com/repos/openai/gym","labels_url":"https://api.github.com/repos/openai/gym/issues/71/labels{/name}","comments_url":"https://api.github.com/repos/openai/gym/issues/71/comments","events_url":"https://api.github.com/repos/openai/gym/issues/71/events","html_url":"https://github.com/openai/gym/issues/71","id":153671868,"number":71,"title":"Env.step() with no action","user":{"login":"keleftheriou","id":19254337,"avatar_url":"https://avatars.githubusercontent.com/u/19254337?v=3","gravatar_id":"","url":"https://api.github.com/users/keleftheriou","html_url":"https://github.com/keleftheriou","followers_url":"https://api.github.com/users/keleftheriou/followers","following_url":"https://api.github.com/users/keleftheriou/following{/other_user}","gists_url":"https://api.github.com/users/keleftheriou/gists{/gist_id}","starred_url":"https://api.github.com/users/keleftheriou/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/keleftheriou/subscriptions","organizations_url":"https://api.github.com/users/keleftheriou/orgs","repos_url":"https://api.github.com/users/keleftheriou/repos","events_url":"https://api.github.com/users/keleftheriou/events{/privacy}","received_events_url":"https://api.github.com/users/keleftheriou/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2016-05-08T19:00:07Z","updated_at":"2016-05-09T04:34:13Z","closed_at":null,"body":"Intuitively, an algorithm should at any point also have the choice of *not* acting at all.\r\n\r\nAt least in the example of control/physics environments, being able to not act might sometimes be the \"best\" choice. Defining \"best\" here can be a long discussion, but as an example an algorithm might be optimizing for *efficiency* (reward/effort), rather than absolute reward."},"comment":{"url":"https://api.github.com/repos/openai/gym/issues/comments/217774098","html_url":"https://github.com/openai/gym/issues/71#issuecomment-217774098","issue_url":"https://api.github.com/repos/openai/gym/issues/71","id":217774098,"user":{"login":"keleftheriou","id":19254337,"avatar_url":"https://avatars.githubusercontent.com/u/19254337?v=3","gravatar_id":"","url":"https://api.github.com/users/keleftheriou","html_url":"https://github.com/keleftheriou","followers_url":"https://api.github.com/users/keleftheriou/followers","following_url":"https://api.github.com/users/keleftheriou/following{/other_user}","gists_url":"https://api.github.com/users/keleftheriou/gists{/gist_id}","starred_url":"https://api.github.com/users/keleftheriou/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/keleftheriou/subscriptions","organizations_url":"https://api.github.com/users/keleftheriou/orgs","repos_url":"https://api.github.com/users/keleftheriou/repos","events_url":"https://api.github.com/users/keleftheriou/events{/privacy}","received_events_url":"https://api.github.com/users/keleftheriou/received_events","type":"User","site_admin":false},"created_at":"2016-05-09T04:34:12Z","updated_at":"2016-05-09T04:34:12Z","body":"I've only been playing with the 'CartPole-v0' environment so far, and that has an `action_space` of `spaces.Discrete(2)` which led me to my comment.\r\n\r\nI wonder if making `Env.step()` have `action=None` as a default could work, letting the environment  complain if it can't accept it. The API would be more general this way, and the potential for API misuse by either environment users or environment authors seems low.\r\n\r\n Alternatively, 'CartPole-v0' could just be changed to take a `Box` `action_space` instead of a `Discrete` one, although it feels like this particular environment might have been purposefully made to not take variable force values.\r\n\r\nGood to know about the mujoco environments utilizing a notion of \"effort\" through the `reward` returned, thanks."}}