{"action":"created","issue":{"url":"https://api.github.com/repos/bazelbuild/bazel/issues/5331","repository_url":"https://api.github.com/repos/bazelbuild/bazel","labels_url":"https://api.github.com/repos/bazelbuild/bazel/issues/5331/labels{/name}","comments_url":"https://api.github.com/repos/bazelbuild/bazel/issues/5331/comments","events_url":"https://api.github.com/repos/bazelbuild/bazel/issues/5331/events","html_url":"https://github.com/bazelbuild/bazel/issues/5331","id":329540615,"node_id":"MDU6SXNzdWUzMjk1NDA2MTU=","number":5331,"title":"Java code coverage for sh_tests missing","user":{"login":"dhalperi","id":526415,"node_id":"MDQ6VXNlcjUyNjQxNQ==","avatar_url":"https://avatars0.githubusercontent.com/u/526415?v=4","gravatar_id":"","url":"https://api.github.com/users/dhalperi","html_url":"https://github.com/dhalperi","followers_url":"https://api.github.com/users/dhalperi/followers","following_url":"https://api.github.com/users/dhalperi/following{/other_user}","gists_url":"https://api.github.com/users/dhalperi/gists{/gist_id}","starred_url":"https://api.github.com/users/dhalperi/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dhalperi/subscriptions","organizations_url":"https://api.github.com/users/dhalperi/orgs","repos_url":"https://api.github.com/users/dhalperi/repos","events_url":"https://api.github.com/users/dhalperi/events{/privacy}","received_events_url":"https://api.github.com/users/dhalperi/received_events","type":"User","site_admin":false},"labels":[{"id":920235609,"node_id":"MDU6TGFiZWw5MjAyMzU2MDk=","url":"https://api.github.com/repos/bazelbuild/bazel/labels/coverage","name":"coverage","color":"FFC0CB","default":false},{"id":106503259,"node_id":"MDU6TGFiZWwxMDY1MDMyNTk=","url":"https://api.github.com/repos/bazelbuild/bazel/labels/type:%20bug","name":"type: bug","color":"0052cc","default":false}],"state":"open","locked":false,"assignee":{"login":"iirina","id":1687706,"node_id":"MDQ6VXNlcjE2ODc3MDY=","avatar_url":"https://avatars3.githubusercontent.com/u/1687706?v=4","gravatar_id":"","url":"https://api.github.com/users/iirina","html_url":"https://github.com/iirina","followers_url":"https://api.github.com/users/iirina/followers","following_url":"https://api.github.com/users/iirina/following{/other_user}","gists_url":"https://api.github.com/users/iirina/gists{/gist_id}","starred_url":"https://api.github.com/users/iirina/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/iirina/subscriptions","organizations_url":"https://api.github.com/users/iirina/orgs","repos_url":"https://api.github.com/users/iirina/repos","events_url":"https://api.github.com/users/iirina/events{/privacy}","received_events_url":"https://api.github.com/users/iirina/received_events","type":"User","site_admin":false},"assignees":[{"login":"iirina","id":1687706,"node_id":"MDQ6VXNlcjE2ODc3MDY=","avatar_url":"https://avatars3.githubusercontent.com/u/1687706?v=4","gravatar_id":"","url":"https://api.github.com/users/iirina","html_url":"https://github.com/iirina","followers_url":"https://api.github.com/users/iirina/followers","following_url":"https://api.github.com/users/iirina/following{/other_user}","gists_url":"https://api.github.com/users/iirina/gists{/gist_id}","starred_url":"https://api.github.com/users/iirina/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/iirina/subscriptions","organizations_url":"https://api.github.com/users/iirina/orgs","repos_url":"https://api.github.com/users/iirina/repos","events_url":"https://api.github.com/users/iirina/events{/privacy}","received_events_url":"https://api.github.com/users/iirina/received_events","type":"User","site_admin":false}],"milestone":null,"comments":4,"created_at":"2018-06-05T16:49:44Z","updated_at":"2018-07-11T16:30:03Z","closed_at":null,"author_association":"NONE","body":"### Description of the problem:\r\n\r\nCopying from [bazel-discuss thread](https://groups.google.com/d/msg/bazel-discuss/ScwEnY3Gn-w/fTkyw84YCgAJ) per request:\r\n\r\nWe have a mix of JUnit tests and sh_tests that invoke a binary in a test scenario. `bazel coverage //... --jobs 1` [0] works great for the unit tests, but does not generate a coverage.dat for the `sh_tests`.\r\n\r\nRepo is open source: [https://github.com/batfish/batfish@c6658e88](https://github.com/batfish/batfish/tree/c6658e88c8c9d48e393547281cc119d99b90670d) is base, and [https://github.com/batfish/batfish/blob/c6658e88c8c9d48e393547281cc119d99b90670d/tests/parsing-tests/BUILD](https://github.com/batfish/batfish/blob/c6658e88c8c9d48e393547281cc119d99b90670d/tests/parsing-tests/BUILD#L18) is the integration `sh_test`.\r\n\r\nI tried figuring out how the Java coverage works under the hood, but I got fairly stymied. Any guidance?\r\n\r\n[0] `--jobs 1` workaround for crash: https://github.com/bazelbuild/bazel/issues/4398\r\n\r\n### Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.\r\n\r\nCheck out the linked repository at the linked commit and run `bazel coverage --jobs 1 //...`. Command log below:\r\n\r\n```\r\n➜  batfish git:(master) git checkout c6658e88c8c9d48e393547281cc119d99b90670d \r\nNote: checking out 'c6658e88c8c9d48e393547281cc119d99b90670d'.\r\n\r\nYou are in 'detached HEAD' state. You can look around, make experimental\r\nchanges and commit them, and you can discard any commits you make in this\r\nstate without impacting any branches by performing another checkout.\r\n\r\nIf you want to create a new branch to retain commits you create, you may\r\ndo so (now or later) by using -b with the checkout command again. Example:\r\n\r\n  git checkout -b <new-branch-name>\r\n\r\nHEAD is now at c6658e88c Local interface in bgp session status (#1588)\r\n➜  batfish git:(c6658e88c) bazel clean --expunge                                \r\nINFO: Starting clean.\r\n➜  batfish git:(c6658e88c) bazel coverage --jobs 1 //...\r\nINFO: SHA256 (https://services.gradle.org/distributions/gradle-4.1-bin.zip) = d55dfa9cfb5a3da86a1c9e75bb0b9507f9a8c8c100793ccec7beb6e259f9ed43\r\nINFO: Using default value for --instrumentation_filter: \"//projects,//tests/parsing-tests\".\r\nINFO: Override the above default with --instrumentation_filter\r\nINFO: Analysed 59 targets (190 packages loaded).\r\nINFO: Found 52 targets and 7 test targets...\r\nINFO: Elapsed time: 289.417s, Critical Path: 47.31s\r\nINFO: 324 processes: 261 darwin-sandbox, 13 local, 50 worker.\r\nINFO: Build completed successfully, 500 total actions\r\n//projects/allinone:allinone_tests                                       PASSED in 6.0s\r\n  /private/var/tmp/_bazel_dan/ffd2f268833102995456f3fc3b57cee1/execroot/batfish/bazel-out/darwin-fastbuild/testlogs/projects/allinone/allinone_tests/coverage.dat\r\n//projects/batfish:batfish_tests                                         PASSED in 23.8s\r\n  /private/var/tmp/_bazel_dan/ffd2f268833102995456f3fc3b57cee1/execroot/batfish/bazel-out/darwin-fastbuild/testlogs/projects/batfish/batfish_tests/coverage.dat\r\n//projects/batfish-client:client_tests                                   PASSED in 2.1s\r\n  /private/var/tmp/_bazel_dan/ffd2f268833102995456f3fc3b57cee1/execroot/batfish/bazel-out/darwin-fastbuild/testlogs/projects/batfish-client/client_tests/coverage.dat\r\n//projects/batfish-common-protocol:common_tests                          PASSED in 3.1s\r\n  /private/var/tmp/_bazel_dan/ffd2f268833102995456f3fc3b57cee1/execroot/batfish/bazel-out/darwin-fastbuild/testlogs/projects/batfish-common-protocol/common_tests/coverage.dat\r\n//projects/coordinator:coordinator_tests                                 PASSED in 5.7s\r\n  /private/var/tmp/_bazel_dan/ffd2f268833102995456f3fc3b57cee1/execroot/batfish/bazel-out/darwin-fastbuild/testlogs/projects/coordinator/coordinator_tests/coverage.dat\r\n//projects/question:question_tests                                       PASSED in 1.8s\r\n  /private/var/tmp/_bazel_dan/ffd2f268833102995456f3fc3b57cee1/execroot/batfish/bazel-out/darwin-fastbuild/testlogs/projects/question/question_tests/coverage.dat\r\n//tests/parsing-tests:ref_tests                                          PASSED in 25.0s\r\n\r\nExecuted 7 out of 7 tests: 7 tests pass.\r\nThere were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see wINFO: Build completed successfully, 500 total actions\r\n```\r\n\r\nNote that there were 7 tests run, only 6 coverage.dat files produced. The 6 tests are `junit_tests` and the 7th is a `sh_test`.\r\n\r\nNote that I would actually be (very pleasantly) surprised if this worked as-is... I'd expect I might have to do something in the generated `sh_test` to indicate I wanted Java coverage. I just don't know what that something might be...\r\n\r\n### What operating system are you running Bazel on?\r\n\r\nmacOS High Sierra 10.13.5 (17F77)\r\n\r\n### What's the output of `bazel info release`?\r\n\r\nrelease 0.14.0-homebrew\r\n\r\n### If `bazel info release` returns \"development version\" or \"(@non-git)\", tell us how you built Bazel.\r\n\r\nN/A\r\n\r\n### What's the output of `git remote get-url origin ; git rev-parse master ; git rev-parse HEAD` ?\r\n\r\nN/A I believe.\r\n\r\n###  Have you found anything relevant by searching the web?\r\n\r\n* [bazel-discuss thread](https://groups.google.com/d/msg/bazel-discuss/ScwEnY3Gn-w/fTkyw84YCgAJ) linked above.\r\n\r\n* I was perhaps wondering if a workaround would be a Skylark rule using [`coverage_instrumented`](https://docs.bazel.build/versions/master/skylark/lib/ctx.html#coverage_instrumented) but have not had a chance to try this yet.\r\n\r\n### Any other information, logs, or outputs that you want to share?\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/bazelbuild/bazel/issues/comments/404232059","html_url":"https://github.com/bazelbuild/bazel/issues/5331#issuecomment-404232059","issue_url":"https://api.github.com/repos/bazelbuild/bazel/issues/5331","id":404232059,"node_id":"MDEyOklzc3VlQ29tbWVudDQwNDIzMjA1OQ==","user":{"login":"dhalperi","id":526415,"node_id":"MDQ6VXNlcjUyNjQxNQ==","avatar_url":"https://avatars0.githubusercontent.com/u/526415?v=4","gravatar_id":"","url":"https://api.github.com/users/dhalperi","html_url":"https://github.com/dhalperi","followers_url":"https://api.github.com/users/dhalperi/followers","following_url":"https://api.github.com/users/dhalperi/following{/other_user}","gists_url":"https://api.github.com/users/dhalperi/gists{/gist_id}","starred_url":"https://api.github.com/users/dhalperi/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dhalperi/subscriptions","organizations_url":"https://api.github.com/users/dhalperi/orgs","repos_url":"https://api.github.com/users/dhalperi/repos","events_url":"https://api.github.com/users/dhalperi/events{/privacy}","received_events_url":"https://api.github.com/users/dhalperi/received_events","type":"User","site_admin":false},"created_at":"2018-07-11T16:30:03Z","updated_at":"2018-07-11T16:30:03Z","author_association":"NONE","body":"The current output of `bazel coverage --jobs=1 //...`:\r\n\r\n```\r\n➜  batfish git:(master) ✗ bazel coverage --jobs 1 //...       \r\nStarting local Bazel server and connecting to it...\r\n................\r\nINFO: Using default value for --instrumentation_filter: \"//projects,//tests\".\r\nINFO: Override the above default with --instrumentation_filter\r\nINFO: Analysed 81 targets (202 packages loaded).\r\nINFO: Found 66 targets and 15 test targets...\r\nINFO: Elapsed time: 318.442s, Critical Path: 52.86s\r\nINFO: 113 processes: 57 darwin-sandbox, 20 local, 36 worker.\r\nINFO: Build completed successfully, 246 total actions\r\n//projects/allinone:allinone_tests                                       PASSED in 13.8s\r\n  /private/var/tmp/_bazel_dan/ffd2f268833102995456f3fc3b57cee1/execroot/batfish/bazel-out/darwin-fastbuild/testlogs/projects/allinone/allinone_tests/coverage.dat\r\n//projects/batfish:batfish_tests                                         PASSED in 21.5s\r\n  /private/var/tmp/_bazel_dan/ffd2f268833102995456f3fc3b57cee1/execroot/batfish/bazel-out/darwin-fastbuild/testlogs/projects/batfish/batfish_tests/coverage.dat\r\n//projects/batfish-client:client_tests                                   PASSED in 2.0s\r\n  /private/var/tmp/_bazel_dan/ffd2f268833102995456f3fc3b57cee1/execroot/batfish/bazel-out/darwin-fastbuild/testlogs/projects/batfish-client/client_tests/coverage.dat\r\n//projects/batfish-common-protocol:common_tests                          PASSED in 2.4s\r\n  /private/var/tmp/_bazel_dan/ffd2f268833102995456f3fc3b57cee1/execroot/batfish/bazel-out/darwin-fastbuild/testlogs/projects/batfish-common-protocol/common_tests/coverage.dat\r\n//projects/coordinator:coordinator_tests                                 PASSED in 5.0s\r\n  /private/var/tmp/_bazel_dan/ffd2f268833102995456f3fc3b57cee1/execroot/batfish/bazel-out/darwin-fastbuild/testlogs/projects/coordinator/coordinator_tests/coverage.dat\r\n//projects/question:question_tests                                       PASSED in 2.1s\r\n  /private/var/tmp/_bazel_dan/ffd2f268833102995456f3fc3b57cee1/execroot/batfish/bazel-out/darwin-fastbuild/testlogs/projects/question/question_tests/coverage.dat\r\n//tests/aws:ref_tests                                                    PASSED in 14.1s\r\n//tests/basic:ref_tests                                                  PASSED in 24.2s\r\n//tests/java-smt:ref_tests                                               PASSED in 24.2s\r\n//tests/jsonpath-addons:ref_tests                                        PASSED in 15.2s\r\n//tests/jsonpathtotable:ref_tests                                        PASSED in 15.6s\r\n//tests/parsing-errors-tests:ref_tests                                   PASSED in 14.5s\r\n//tests/parsing-tests:ref_tests                                          PASSED in 27.5s\r\n//tests/roles:ref_tests                                                  PASSED in 16.3s\r\n//tests/ui-focused:ref_tests                                             PASSED in 16.1s\r\n```"}}
{"actor":{"display_login":"dhalperi"}}