{"action":"created","issue":{"url":"https://api.github.com/repos/ManageIQ/manageiq/issues/9447","repository_url":"https://api.github.com/repos/ManageIQ/manageiq","labels_url":"https://api.github.com/repos/ManageIQ/manageiq/issues/9447/labels{/name}","comments_url":"https://api.github.com/repos/ManageIQ/manageiq/issues/9447/comments","events_url":"https://api.github.com/repos/ManageIQ/manageiq/issues/9447/events","html_url":"https://github.com/ManageIQ/manageiq/pull/9447","id":162279178,"number":9447,"title":"Cap&U split up capture_infra_targets to allow tuning","user":{"login":"kbrock","id":1930,"avatar_url":"https://avatars.githubusercontent.com/u/1930?v=3","gravatar_id":"","url":"https://api.github.com/users/kbrock","html_url":"https://github.com/kbrock","followers_url":"https://api.github.com/users/kbrock/followers","following_url":"https://api.github.com/users/kbrock/following{/other_user}","gists_url":"https://api.github.com/users/kbrock/gists{/gist_id}","starred_url":"https://api.github.com/users/kbrock/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kbrock/subscriptions","organizations_url":"https://api.github.com/users/kbrock/orgs","repos_url":"https://api.github.com/users/kbrock/repos","events_url":"https://api.github.com/users/kbrock/events{/privacy}","received_events_url":"https://api.github.com/users/kbrock/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/ManageIQ/manageiq/labels/metrics","name":"metrics","color":"5319e7"},{"url":"https://api.github.com/repos/ManageIQ/manageiq/labels/performance","name":"performance","color":"fad8c7"},{"url":"https://api.github.com/repos/ManageIQ/manageiq/labels/refactoring","name":"refactoring","color":"fbca04"}],"state":"open","locked":false,"assignee":{"login":"Fryguy","id":52120,"avatar_url":"https://avatars.githubusercontent.com/u/52120?v=3","gravatar_id":"","url":"https://api.github.com/users/Fryguy","html_url":"https://github.com/Fryguy","followers_url":"https://api.github.com/users/Fryguy/followers","following_url":"https://api.github.com/users/Fryguy/following{/other_user}","gists_url":"https://api.github.com/users/Fryguy/gists{/gist_id}","starred_url":"https://api.github.com/users/Fryguy/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Fryguy/subscriptions","organizations_url":"https://api.github.com/users/Fryguy/orgs","repos_url":"https://api.github.com/users/Fryguy/repos","events_url":"https://api.github.com/users/Fryguy/events{/privacy}","received_events_url":"https://api.github.com/users/Fryguy/received_events","type":"User","site_admin":false},"assignees":[{"login":"Fryguy","id":52120,"avatar_url":"https://avatars.githubusercontent.com/u/52120?v=3","gravatar_id":"","url":"https://api.github.com/users/Fryguy","html_url":"https://github.com/Fryguy","followers_url":"https://api.github.com/users/Fryguy/followers","following_url":"https://api.github.com/users/Fryguy/following{/other_user}","gists_url":"https://api.github.com/users/Fryguy/gists{/gist_id}","starred_url":"https://api.github.com/users/Fryguy/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Fryguy/subscriptions","organizations_url":"https://api.github.com/users/Fryguy/orgs","repos_url":"https://api.github.com/users/Fryguy/repos","events_url":"https://api.github.com/users/Fryguy/events{/privacy}","received_events_url":"https://api.github.com/users/Fryguy/received_events","type":"User","site_admin":false}],"milestone":null,"comments":5,"created_at":"2016-06-25T12:33:16Z","updated_at":"2016-07-07T15:26:48Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/ManageIQ/manageiq/pulls/9447","html_url":"https://github.com/ManageIQ/manageiq/pull/9447","diff_url":"https://github.com/ManageIQ/manageiq/pull/9447.diff","patch_url":"https://github.com/ManageIQ/manageiq/pull/9447.patch"},"body":"Purpose or Intent\r\n-----------------\r\nScheduled job `perf_capture_timer` is responsible for kicking off Cap&U for the system.\r\nIt is split up into 2 parts:\r\n\r\n1. Determine which resource (i.e.: `Host`, `Storage`, `Vm`) need Cap & U. **this pr** #9447\r\n2. Submits them into the queue for further processing. ~~#9477~~\r\n\r\nThis PR focuses **part 1** and the themes are based upon 2 pain points from the past year:\r\n\r\na. Ensure the resources have an `Ems` assigned.\r\nb. Make performance measurement easier.\r\n\r\nThis work makes it more explicit which data is loaded and available for part 2, (i.e.: submitting into the queue).\r\nCurrently, part 2 is the main bottleneck but non-trivial to fix.\r\n\r\n\r\nDetails\r\n----------\r\n\r\n- Explicitly use `Zone.ems` to fetch records. This ensures that all records will all have an`Ems` assigned.\r\n- Extract a method for each resource type from `capture_infra_targets`. This simplifies the process and makes it easier to track down performance issues.\r\n- Calculate enabled `hosts` up front and store in it's own variable. This makes it unnecessary to run further host selection logic.\r\n- Extract convenience method to determine supported storage types.\r\n- Merged all preload logic into a single call (for minor savings).\r\n\r\n/cc @jrafanie This is from a conversation we had. I just need this to get in so I know which tags are available for part 2. Ok to pawn off on someone else once you have a one over\r\n\r\nResults\r\n----------\r\n\r\nWhile this is attempting to organize code, it does have performance impact.\r\nFor 1 ems, 100 VMs, 10 hosts, and 13 storages. It ran 50 (6%) fewer queries resulting in 2% fewer objects allocated (~4MB of memory savings).\r\n\r\n|       ms | objects     |queries | query (ms) |     rows |comments\r\n|      ---:|       ---:|     ---:|     ---:|      ---:| ---\r\n|  2,798.3 | 1,997,671 |  961 |   684.9 |   11,202 |before\r\n|  2,546.1 | 1,961,437 |  908 |   556.1 |   11,149 |after\r\n"},"comment":{"url":"https://api.github.com/repos/ManageIQ/manageiq/issues/comments/231113075","html_url":"https://github.com/ManageIQ/manageiq/pull/9447#issuecomment-231113075","issue_url":"https://api.github.com/repos/ManageIQ/manageiq/issues/9447","id":231113075,"user":{"login":"miq-bot","id":7806257,"avatar_url":"https://avatars.githubusercontent.com/u/7806257?v=3","gravatar_id":"","url":"https://api.github.com/users/miq-bot","html_url":"https://github.com/miq-bot","followers_url":"https://api.github.com/users/miq-bot/followers","following_url":"https://api.github.com/users/miq-bot/following{/other_user}","gists_url":"https://api.github.com/users/miq-bot/gists{/gist_id}","starred_url":"https://api.github.com/users/miq-bot/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/miq-bot/subscriptions","organizations_url":"https://api.github.com/users/miq-bot/orgs","repos_url":"https://api.github.com/users/miq-bot/repos","events_url":"https://api.github.com/users/miq-bot/events{/privacy}","received_events_url":"https://api.github.com/users/miq-bot/received_events","type":"User","site_admin":false},"created_at":"2016-07-07T15:26:48Z","updated_at":"2016-07-07T15:26:48Z","body":"<rubocop />Checked commits https://github.com/kbrock/manageiq/compare/bfd4a415e3c5feb3c5ec1002ad6a880a35aad2f3~...23dcbfa95e8f71c965fea7a3ed06e5269a2b5b53 with ruby 2.2.5, rubocop 0.37.2, and haml-lint 0.16.1\n2 files checked, 2 offenses detected\n\n**app/models/metric/targets.rb**\n- [ ] :exclamation: - [Line 97](https://github.com/kbrock/manageiq/blob/23dcbfa95e8f71c965fea7a3ed06e5269a2b5b53/app/models/metric/targets.rb#L97), Col 9 - [Style/Next](http://rubydoc.info/gems/rubocop/0.37.2/RuboCop/Cop/Style/Next) - Use `next` to skip iteration.\n- [ ] :grey_exclamation: - [Line 86](https://github.com/kbrock/manageiq/blob/23dcbfa95e8f71c965fea7a3ed06e5269a2b5b53/app/models/metric/targets.rb#L86), Col 3 - [Metrics/AbcSize](http://rubydoc.info/gems/rubocop/0.37.2/RuboCop/Cop/Metrics/AbcSize) - Assignment Branch Condition size for postload_infra_targets_data is too high. [19.1/15]\n"}}
{"actor":{"display_login":"miq-bot"}}