{"action":"opened","number":16204,"pull_request":{"url":"https://api.github.com/repos/ansible/ansible/pulls/16204","id":73218770,"html_url":"https://github.com/ansible/ansible/pull/16204","diff_url":"https://github.com/ansible/ansible/pull/16204.diff","patch_url":"https://github.com/ansible/ansible/pull/16204.patch","issue_url":"https://api.github.com/repos/ansible/ansible/issues/16204","number":16204,"state":"open","locked":false,"title":"[Inventory] Cache the result of enumerating groups and host names","user":{"login":"towolf","id":345210,"avatar_url":"https://avatars.githubusercontent.com/u/345210?v=3","gravatar_id":"","url":"https://api.github.com/users/towolf","html_url":"https://github.com/towolf","followers_url":"https://api.github.com/users/towolf/followers","following_url":"https://api.github.com/users/towolf/following{/other_user}","gists_url":"https://api.github.com/users/towolf/gists{/gist_id}","starred_url":"https://api.github.com/users/towolf/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/towolf/subscriptions","organizations_url":"https://api.github.com/users/towolf/orgs","repos_url":"https://api.github.com/users/towolf/repos","events_url":"https://api.github.com/users/towolf/events{/privacy}","received_events_url":"https://api.github.com/users/towolf/received_events","type":"User","site_admin":false},"body":"##### ISSUE TYPE\r\n\r\n - Bugfix Pull Request\r\n\r\n##### ANSIBLE VERSION\r\n<!--- Paste verbatim output from “ansible --version” between quotes below -->\r\n```\r\nansible-playbook 2.2.0 (cache_group_dict 2e1b613c7e) last updated 2016/06/09 15:56:02 (GMT +200)\r\n  lib/ansible/modules/core: (devel a8072f9ef0) last updated 2016/06/07 09:13:22 (GMT +200)\r\n  lib/ansible/modules/extras: (detached HEAD b0aec50b9a) last updated 2016/06/01 14:11:22 (GMT +200)\r\n```\r\n\r\n##### SUMMARY\r\n\r\nCache the result of enumerating groups and host names use inside `VariableManager._get_magic_variables()`.\r\n\r\nThis saves a lot of time re-iterating the nearly always constant global\r\nlist of groups and their members.\r\n\r\nGenerate once and cache, and invalidate cache in case `add_host:` or\r\n`group_by:` are used.\r\n\r\n\r\nBefore\r\n======\r\n\r\n```\r\nTotal time: 485.37 s\r\nFile: /home/towolf/src/local/ansible/lib/ansible/vars/__init__.py\r\nFunction: _get_magic_variables at line 374\r\n\r\nLine #      Hits         Time  Per Hit   % Time  Line Contents\r\n==============================================================\r\n   374                                               @profile\r\n   375                                               def _get_magic_variables(self, loader, play, host, task, include_hostvars, include_delegate_to):\r\n   376                                                   '''\r\n   377                                                   Returns a dictionary of so-called \"magic\" variables in Ansible,\r\n   378                                                   which are special variables we set internally for use.\r\n   379                                                   '''\r\n   380                                           \r\n   381      3010        10132      3.4      0.0          variables = dict()\r\n   382      3010        51137     17.0      0.0          variables['playbook_dir'] = loader.get_basedir()\r\n   383                                           \r\n   384      3010         5181      1.7      0.0          if host:\r\n   385     47808      1080418     22.6      0.2              variables['group_names'] = sorted([group.name for group in host.get_groups() if group.name != 'all'])\r\n   386                                           \r\n   387      2574         5605      2.2      0.0              if self._inventory is not None:\r\n   388      2574         6046      2.3      0.0                  variables['groups']  = dict()\r\n   389  23155704     46359530      2.0      9.6                  for (group_name, group) in iteritems(self._inventory.groups):\r\n   390 171155556    435879828      2.5     89.8                      variables['groups'][group_name] = [h.name for h in group.get_hosts()]\r\n   391      3010         5512      1.8      0.0          if play:\r\n   392      6016        77879     12.9      0.0              variables['role_names'] = [r._role_name for r in play.roles]\r\n   393                                           \r\n   394      3010         5422      1.8      0.0          if task:\r\n   395      3006         6596      2.2      0.0              if task._role:\r\n   396      2863        11964      4.2      0.0                  variables['role_name'] = task._role.get_name()\r\n   397      2863        11069      3.9      0.0                  variables['role_path'] = task._role._role_path\r\n   398      2863        51752     18.1      0.0                  variables['role_uuid'] = text_type(task._role._uuid)\r\n   399                                           \r\n   400      3010         7532      2.5      0.0          if self._inventory is not None:\r\n   401      3010       351708    116.8      0.1              variables['inventory_dir'] = self._inventory.basedir()\r\n   402      3010        75596     25.1      0.0              variables['inventory_file'] = self._inventory.src()\r\n   403      3010         5813      1.9      0.0              if play:\r\n   404                                                           # add the list of hosts in the play, as adjusted for limit/filters\r\n   405                                                           # DEPRECATED: play_hosts should be deprecated in favor of ansible_play_hosts,\r\n   406                                                           #             however this would take work in the templating engine, so for now\r\n   407                                                           #             we'll add both so we can give users something transitional to use\r\n   408    451950      1159038      2.6      0.2                  host_list = [x.name for x in self._inventory.get_hosts()]\r\n   409      3010         5423      1.8      0.0                  variables['play_hosts'] = host_list\r\n   410      3010         5010      1.7      0.0                  variables['ansible_play_hosts'] = host_list\r\n   411                                           \r\n   412                                                   # the 'omit' value alows params to be left out if the variable they are based on is undefined\r\n   413      3010        10783      3.6      0.0          variables['omit'] = self._omit_token\r\n   414      3010       130188     43.3      0.0          variables['ansible_version'] = CLI.version_info(gitinfo=False)\r\n   415                                                   # Set options vars\r\n   416      6020        28844      4.8      0.0          for option, option_value in iteritems(self._options_vars):\r\n   417      3010         5882      2.0      0.0              variables[option] = option_value\r\n   418                                           \r\n   419      3010         6659      2.2      0.0          if self._hostvars is not None and include_hostvars:\r\n   420      3004         5735      1.9      0.0              variables['hostvars'] = self._hostvars\r\n   421                                           \r\n   422      3010         4190      1.4      0.0          return variables\r\n```\r\n\r\n\r\nAfter\r\n====\r\n\r\n```\r\nTotal time: 14.9396 s\r\nFile: /home/towolf/src/local/ansible/lib/ansible/vars/__init__.py\r\nFunction: _get_magic_variables at line 385\r\n\r\nLine #      Hits         Time  Per Hit   % Time  Line Contents\r\n==============================================================\r\n   385                                               def _get_magic_variables(self, loader, play, host, task, include_hostvars, include_delegate_to):\r\n   386                                                   '''\r\n   387                                                   Returns a dictionary of so-called \"magic\" variables in Ansible,\r\n   388                                                   which are special variables we set internally for use.\r\n   389                                                   '''\r\n   390                                           \r\n   391                                                   variables = dict()\r\n   392      3073        72288     23.5      0.5          variables['playbook_dir'] = loader.get_basedir()\r\n   393      3073       193975     63.1      1.3  \r\n   394                                                   if host:\r\n   395      3073         6814      2.2      0.0              variables['group_names'] = sorted([group.name for group in host.get_groups() if group.name != 'all'])\r\n   396     48978      4164175     85.0     27.9  \r\n   397                                                       if self._inventory:\r\n   398      2628        27409     10.4      0.2                  variables['groups']  = self._inventory.get_group_dict()\r\n   399      2628       209298     79.6      1.4  \r\n   400                                                   if play:\r\n   401      3073        10087      3.3      0.1              variables['role_names'] = [r._role_name for r in play.roles]\r\n   402      6142       229651     37.4      1.5  \r\n   403                                                   if task:\r\n   404      3073         5559      1.8      0.0              if task._role:\r\n   405      3069         6333      2.1      0.0                  variables['role_name'] = task._role.get_name()\r\n   406      2923        83104     28.4      0.6                  variables['role_path'] = task._role._role_path\r\n   407      2923        81817     28.0      0.5                  variables['role_uuid'] = text_type(task._role._uuid)\r\n   408      2923       103889     35.5      0.7  \r\n   409                                                   if self._inventory is not None:\r\n   410      3073        13729      4.5      0.1              variables['inventory_dir'] = self._inventory.basedir()\r\n   411      3073      1419979    462.1      9.5              variables['inventory_file'] = self._inventory.src()\r\n   412      3073       494898    161.0      3.3              if play:\r\n   413      3073        34374     11.2      0.2                  # add the list of hosts in the play, as adjusted for limit/filters\r\n   414                                                           # DEPRECATED: play_hosts should be deprecated in favor of ansible_play_hosts,\r\n   415                                                           #             however this would take work in the templating engine, so for now\r\n   416                                                           #             we'll add both so we can give users something transitional to use\r\n   417                                                           host_list = [x.name for x in self._inventory.get_hosts()]\r\n   418    470276      6751722     14.4     45.2                  variables['play_hosts'] = host_list\r\n   419      3073        12900      4.2      0.1                  variables['ansible_play_hosts'] = host_list\r\n   420      3073        13149      4.3      0.1  \r\n   421                                                   # the 'omit' value alows params to be left out if the variable they are based on is undefined\r\n   422                                                   variables['omit'] = self._omit_token\r\n   423      3073        59040     19.2      0.4          variables['ansible_version'] = CLI.version_info(gitinfo=False)\r\n   424      3073       772767    251.5      5.2          # Set options vars\r\n   425                                                   for option, option_value in iteritems(self._options_vars):\r\n   426      6146        53857      8.8      0.4              variables[option] = option_value\r\n   427      3073        31243     10.2      0.2  \r\n   428                                                   if self._hostvars is not None and include_hostvars:\r\n   429      3073        66860     21.8      0.4              variables['hostvars'] = self._hostvars\r\n   430      3067        15350      5.0      0.1  \r\n   431                                                   return variables\r\n```\r\n","created_at":"2016-06-09T13:59:34Z","updated_at":"2016-06-09T13:59:34Z","closed_at":null,"merged_at":null,"merge_commit_sha":"1f210e01f29a274a18a9531ae3e86615b860784f","assignee":null,"milestone":null,"commits_url":"https://api.github.com/repos/ansible/ansible/pulls/16204/commits","review_comments_url":"https://api.github.com/repos/ansible/ansible/pulls/16204/comments","review_comment_url":"https://api.github.com/repos/ansible/ansible/pulls/comments{/number}","comments_url":"https://api.github.com/repos/ansible/ansible/issues/16204/comments","statuses_url":"https://api.github.com/repos/ansible/ansible/statuses/2e1b613c7ee6de05be23e5c554af69ca48ae1514","head":{"label":"towolf:cache_group_dict","ref":"cache_group_dict","sha":"2e1b613c7ee6de05be23e5c554af69ca48ae1514","user":{"login":"towolf","id":345210,"avatar_url":"https://avatars.githubusercontent.com/u/345210?v=3","gravatar_id":"","url":"https://api.github.com/users/towolf","html_url":"https://github.com/towolf","followers_url":"https://api.github.com/users/towolf/followers","following_url":"https://api.github.com/users/towolf/following{/other_user}","gists_url":"https://api.github.com/users/towolf/gists{/gist_id}","starred_url":"https://api.github.com/users/towolf/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/towolf/subscriptions","organizations_url":"https://api.github.com/users/towolf/orgs","repos_url":"https://api.github.com/users/towolf/repos","events_url":"https://api.github.com/users/towolf/events{/privacy}","received_events_url":"https://api.github.com/users/towolf/received_events","type":"User","site_admin":false},"repo":{"id":49649505,"name":"ansible","full_name":"towolf/ansible","owner":{"login":"towolf","id":345210,"avatar_url":"https://avatars.githubusercontent.com/u/345210?v=3","gravatar_id":"","url":"https://api.github.com/users/towolf","html_url":"https://github.com/towolf","followers_url":"https://api.github.com/users/towolf/followers","following_url":"https://api.github.com/users/towolf/following{/other_user}","gists_url":"https://api.github.com/users/towolf/gists{/gist_id}","starred_url":"https://api.github.com/users/towolf/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/towolf/subscriptions","organizations_url":"https://api.github.com/users/towolf/orgs","repos_url":"https://api.github.com/users/towolf/repos","events_url":"https://api.github.com/users/towolf/events{/privacy}","received_events_url":"https://api.github.com/users/towolf/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/towolf/ansible","description":"Ansible is a radically simple IT automation platform that makes your applications and systems easier to deploy. Avoid writing scripts or custom code to deploy and update your applications— automate in a language that approaches plain English, using SSH, with no agents to install on remote systems.","fork":true,"url":"https://api.github.com/repos/towolf/ansible","forks_url":"https://api.github.com/repos/towolf/ansible/forks","keys_url":"https://api.github.com/repos/towolf/ansible/keys{/key_id}","collaborators_url":"https://api.github.com/repos/towolf/ansible/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/towolf/ansible/teams","hooks_url":"https://api.github.com/repos/towolf/ansible/hooks","issue_events_url":"https://api.github.com/repos/towolf/ansible/issues/events{/number}","events_url":"https://api.github.com/repos/towolf/ansible/events","assignees_url":"https://api.github.com/repos/towolf/ansible/assignees{/user}","branches_url":"https://api.github.com/repos/towolf/ansible/branches{/branch}","tags_url":"https://api.github.com/repos/towolf/ansible/tags","blobs_url":"https://api.github.com/repos/towolf/ansible/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/towolf/ansible/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/towolf/ansible/git/refs{/sha}","trees_url":"https://api.github.com/repos/towolf/ansible/git/trees{/sha}","statuses_url":"https://api.github.com/repos/towolf/ansible/statuses/{sha}","languages_url":"https://api.github.com/repos/towolf/ansible/languages","stargazers_url":"https://api.github.com/repos/towolf/ansible/stargazers","contributors_url":"https://api.github.com/repos/towolf/ansible/contributors","subscribers_url":"https://api.github.com/repos/towolf/ansible/subscribers","subscription_url":"https://api.github.com/repos/towolf/ansible/subscription","commits_url":"https://api.github.com/repos/towolf/ansible/commits{/sha}","git_commits_url":"https://api.github.com/repos/towolf/ansible/git/commits{/sha}","comments_url":"https://api.github.com/repos/towolf/ansible/comments{/number}","issue_comment_url":"https://api.github.com/repos/towolf/ansible/issues/comments{/number}","contents_url":"https://api.github.com/repos/towolf/ansible/contents/{+path}","compare_url":"https://api.github.com/repos/towolf/ansible/compare/{base}...{head}","merges_url":"https://api.github.com/repos/towolf/ansible/merges","archive_url":"https://api.github.com/repos/towolf/ansible/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/towolf/ansible/downloads","issues_url":"https://api.github.com/repos/towolf/ansible/issues{/number}","pulls_url":"https://api.github.com/repos/towolf/ansible/pulls{/number}","milestones_url":"https://api.github.com/repos/towolf/ansible/milestones{/number}","notifications_url":"https://api.github.com/repos/towolf/ansible/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/towolf/ansible/labels{/name}","releases_url":"https://api.github.com/repos/towolf/ansible/releases{/id}","deployments_url":"https://api.github.com/repos/towolf/ansible/deployments","created_at":"2016-01-14T13:40:39Z","updated_at":"2016-01-14T13:40:45Z","pushed_at":"2016-06-09T13:56:19Z","git_url":"git://github.com/towolf/ansible.git","ssh_url":"git@github.com:towolf/ansible.git","clone_url":"https://github.com/towolf/ansible.git","svn_url":"https://github.com/towolf/ansible","homepage":"http://ansible.com/","size":35910,"stargazers_count":0,"watchers_count":0,"language":"Python","has_issues":false,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"devel"}},"base":{"label":"ansible:devel","ref":"devel","sha":"02e52c319c921222fb558d5634ebddef17e0c0c0","user":{"login":"ansible","id":1507452,"avatar_url":"https://avatars.githubusercontent.com/u/1507452?v=3","gravatar_id":"","url":"https://api.github.com/users/ansible","html_url":"https://github.com/ansible","followers_url":"https://api.github.com/users/ansible/followers","following_url":"https://api.github.com/users/ansible/following{/other_user}","gists_url":"https://api.github.com/users/ansible/gists{/gist_id}","starred_url":"https://api.github.com/users/ansible/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ansible/subscriptions","organizations_url":"https://api.github.com/users/ansible/orgs","repos_url":"https://api.github.com/users/ansible/repos","events_url":"https://api.github.com/users/ansible/events{/privacy}","received_events_url":"https://api.github.com/users/ansible/received_events","type":"Organization","site_admin":false},"repo":{"id":3638964,"name":"ansible","full_name":"ansible/ansible","owner":{"login":"ansible","id":1507452,"avatar_url":"https://avatars.githubusercontent.com/u/1507452?v=3","gravatar_id":"","url":"https://api.github.com/users/ansible","html_url":"https://github.com/ansible","followers_url":"https://api.github.com/users/ansible/followers","following_url":"https://api.github.com/users/ansible/following{/other_user}","gists_url":"https://api.github.com/users/ansible/gists{/gist_id}","starred_url":"https://api.github.com/users/ansible/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ansible/subscriptions","organizations_url":"https://api.github.com/users/ansible/orgs","repos_url":"https://api.github.com/users/ansible/repos","events_url":"https://api.github.com/users/ansible/events{/privacy}","received_events_url":"https://api.github.com/users/ansible/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/ansible/ansible","description":"Ansible is a radically simple IT automation platform that makes your applications and systems easier to deploy. Avoid writing scripts or custom code to deploy and update your applications— automate in a language that approaches plain English, using SSH, with no agents to install on remote systems.","fork":false,"url":"https://api.github.com/repos/ansible/ansible","forks_url":"https://api.github.com/repos/ansible/ansible/forks","keys_url":"https://api.github.com/repos/ansible/ansible/keys{/key_id}","collaborators_url":"https://api.github.com/repos/ansible/ansible/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/ansible/ansible/teams","hooks_url":"https://api.github.com/repos/ansible/ansible/hooks","issue_events_url":"https://api.github.com/repos/ansible/ansible/issues/events{/number}","events_url":"https://api.github.com/repos/ansible/ansible/events","assignees_url":"https://api.github.com/repos/ansible/ansible/assignees{/user}","branches_url":"https://api.github.com/repos/ansible/ansible/branches{/branch}","tags_url":"https://api.github.com/repos/ansible/ansible/tags","blobs_url":"https://api.github.com/repos/ansible/ansible/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/ansible/ansible/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/ansible/ansible/git/refs{/sha}","trees_url":"https://api.github.com/repos/ansible/ansible/git/trees{/sha}","statuses_url":"https://api.github.com/repos/ansible/ansible/statuses/{sha}","languages_url":"https://api.github.com/repos/ansible/ansible/languages","stargazers_url":"https://api.github.com/repos/ansible/ansible/stargazers","contributors_url":"https://api.github.com/repos/ansible/ansible/contributors","subscribers_url":"https://api.github.com/repos/ansible/ansible/subscribers","subscription_url":"https://api.github.com/repos/ansible/ansible/subscription","commits_url":"https://api.github.com/repos/ansible/ansible/commits{/sha}","git_commits_url":"https://api.github.com/repos/ansible/ansible/git/commits{/sha}","comments_url":"https://api.github.com/repos/ansible/ansible/comments{/number}","issue_comment_url":"https://api.github.com/repos/ansible/ansible/issues/comments{/number}","contents_url":"https://api.github.com/repos/ansible/ansible/contents/{+path}","compare_url":"https://api.github.com/repos/ansible/ansible/compare/{base}...{head}","merges_url":"https://api.github.com/repos/ansible/ansible/merges","archive_url":"https://api.github.com/repos/ansible/ansible/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/ansible/ansible/downloads","issues_url":"https://api.github.com/repos/ansible/ansible/issues{/number}","pulls_url":"https://api.github.com/repos/ansible/ansible/pulls{/number}","milestones_url":"https://api.github.com/repos/ansible/ansible/milestones{/number}","notifications_url":"https://api.github.com/repos/ansible/ansible/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/ansible/ansible/labels{/name}","releases_url":"https://api.github.com/repos/ansible/ansible/releases{/id}","deployments_url":"https://api.github.com/repos/ansible/ansible/deployments","created_at":"2012-03-06T14:58:02Z","updated_at":"2016-06-09T13:58:03Z","pushed_at":"2016-06-09T13:59:34Z","git_url":"git://github.com/ansible/ansible.git","ssh_url":"git@github.com:ansible/ansible.git","clone_url":"https://github.com/ansible/ansible.git","svn_url":"https://github.com/ansible/ansible","homepage":"http://ansible.com/","size":60149,"stargazers_count":17220,"watchers_count":17220,"language":"Python","has_issues":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":5182,"mirror_url":null,"open_issues_count":1421,"forks":5182,"open_issues":1421,"watchers":17220,"default_branch":"devel"}},"_links":{"self":{"href":"https://api.github.com/repos/ansible/ansible/pulls/16204"},"html":{"href":"https://github.com/ansible/ansible/pull/16204"},"issue":{"href":"https://api.github.com/repos/ansible/ansible/issues/16204"},"comments":{"href":"https://api.github.com/repos/ansible/ansible/issues/16204/comments"},"review_comments":{"href":"https://api.github.com/repos/ansible/ansible/pulls/16204/comments"},"review_comment":{"href":"https://api.github.com/repos/ansible/ansible/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/ansible/ansible/pulls/16204/commits"},"statuses":{"href":"https://api.github.com/repos/ansible/ansible/statuses/2e1b613c7ee6de05be23e5c554af69ca48ae1514"}},"merged":false,"mergeable":true,"mergeable_state":"clean","merged_by":null,"comments":0,"review_comments":0,"commits":1,"additions":38,"deletions":11,"changed_files":4}}
{"actor":{"display_login":"towolf"}}