{"action":"created","issue":{"url":"https://api.github.com/repos/catapult-project/catapult/issues/2238","repository_url":"https://api.github.com/repos/catapult-project/catapult","labels_url":"https://api.github.com/repos/catapult-project/catapult/issues/2238/labels{/name}","comments_url":"https://api.github.com/repos/catapult-project/catapult/issues/2238/comments","events_url":"https://api.github.com/repos/catapult-project/catapult/issues/2238/events","html_url":"https://github.com/catapult-project/catapult/issues/2238","id":147731701,"number":2238,"title":"TBMv2 memory metric specifies some values without improvement direction","user":{"login":"petrcermak","id":2546601,"avatar_url":"https://avatars.githubusercontent.com/u/2546601?v=3","gravatar_id":"","url":"https://api.github.com/users/petrcermak","html_url":"https://github.com/petrcermak","followers_url":"https://api.github.com/users/petrcermak/followers","following_url":"https://api.github.com/users/petrcermak/following{/other_user}","gists_url":"https://api.github.com/users/petrcermak/gists{/gist_id}","starred_url":"https://api.github.com/users/petrcermak/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/petrcermak/subscriptions","organizations_url":"https://api.github.com/users/petrcermak/orgs","repos_url":"https://api.github.com/users/petrcermak/repos","events_url":"https://api.github.com/users/petrcermak/events{/privacy}","received_events_url":"https://api.github.com/users/petrcermak/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/catapult-project/catapult/labels/TBMv2","name":"TBMv2","color":"eb6420"},{"url":"https://api.github.com/repos/catapult-project/catapult/labels/Telemetry","name":"Telemetry","color":"5319e7"},{"url":"https://api.github.com/repos/catapult-project/catapult/labels/Values","name":"Values","color":"207de5"}],"state":"closed","locked":false,"assignee":{"login":"petrcermak","id":2546601,"avatar_url":"https://avatars.githubusercontent.com/u/2546601?v=3","gravatar_id":"","url":"https://api.github.com/users/petrcermak","html_url":"https://github.com/petrcermak","followers_url":"https://api.github.com/users/petrcermak/followers","following_url":"https://api.github.com/users/petrcermak/following{/other_user}","gists_url":"https://api.github.com/users/petrcermak/gists{/gist_id}","starred_url":"https://api.github.com/users/petrcermak/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/petrcermak/subscriptions","organizations_url":"https://api.github.com/users/petrcermak/orgs","repos_url":"https://api.github.com/users/petrcermak/repos","events_url":"https://api.github.com/users/petrcermak/events{/privacy}","received_events_url":"https://api.github.com/users/petrcermak/received_events","type":"User","site_admin":false},"milestone":null,"comments":1,"created_at":"2016-04-12T12:11:19Z","updated_at":"2016-04-12T16:20:06Z","closed_at":"2016-04-12T16:20:06Z","body":"The recently added TBMv2 memory metric (#2195) emits several <tt>unitlessNumber</tt> values (namely process and dump counts) where there's no clear improvement direction. Unfortunately, this causes <tt>run_benchmark</tt> to fail:\r\n\r\n```\r\nTraceback (most recent call last):\r\n  RunBenchmark at /src/third_party/catapult/telemetry/telemetry/internal/story_runner.py:312\r\n    benchmark.ShouldTearDownStateAfterEachStoryRun())\r\n  Run at /src/third_party/catapult/telemetry/telemetry/internal/story_runner.py:225\r\n    _RunStoryAndProcessErrorIfNeeded(story, results, state, test)\r\n  _RunStoryAndProcessErrorIfNeeded at /src/third_party/catapult/telemetry/telemetry/internal/story_runner.py:86\r\n    test.Measure(state.platform, results)\r\n  Measure at /src/third_party/catapult/telemetry/telemetry/web_perf/timeline_based_measurement.py:284\r\n    self._ComputeTimelineBasedMetric(results, trace_value)\r\n  _ComputeTimelineBasedMetric at /src/third_party/catapult/telemetry/telemetry/web_perf/timeline_based_measurement.py:314\r\n    common_value_helpers.TranslateScalarValue(d, page))\r\n  TranslateScalarValue at /src/third_party/catapult/telemetry/telemetry/value/common_value_helpers.py:25\r\n    raise ValueError('Must specify improvement direction for value ' + name)\r\nValueError: Must specify improvement direction for value memory_process_count_browser_sum\r\n```\r\n\r\nIs this intended? I suppose the short-term fix on our side is to add an improvement direction to the units."},"comment":{"url":"https://api.github.com/repos/catapult-project/catapult/issues/comments/208988413","html_url":"https://github.com/catapult-project/catapult/issues/2238#issuecomment-208988413","issue_url":"https://api.github.com/repos/catapult-project/catapult/issues/2238","id":208988413,"user":{"login":"eakuefner","id":75990,"avatar_url":"https://avatars.githubusercontent.com/u/75990?v=3","gravatar_id":"","url":"https://api.github.com/users/eakuefner","html_url":"https://github.com/eakuefner","followers_url":"https://api.github.com/users/eakuefner/followers","following_url":"https://api.github.com/users/eakuefner/following{/other_user}","gists_url":"https://api.github.com/users/eakuefner/gists{/gist_id}","starred_url":"https://api.github.com/users/eakuefner/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/eakuefner/subscriptions","organizations_url":"https://api.github.com/users/eakuefner/orgs","repos_url":"https://api.github.com/users/eakuefner/repos","events_url":"https://api.github.com/users/eakuefner/events{/privacy}","received_events_url":"https://api.github.com/users/eakuefner/received_events","type":"User","site_admin":false},"created_at":"2016-04-12T16:20:06Z","updated_at":"2016-04-12T16:20:06Z","body":"As discussed offline, yes, values requiring improvement directions is WAI. Anyway, I think we can close this bug because of Petr's patch."}}