{"action":"created","issue":{"url":"https://api.github.com/repos/JMRI/JMRI/issues/2506","repository_url":"https://api.github.com/repos/JMRI/JMRI","labels_url":"https://api.github.com/repos/JMRI/JMRI/issues/2506/labels{/name}","comments_url":"https://api.github.com/repos/JMRI/JMRI/issues/2506/comments","events_url":"https://api.github.com/repos/JMRI/JMRI/issues/2506/events","html_url":"https://github.com/JMRI/JMRI/issues/2506","id":189014667,"number":2506,"title":"Cause CI Tests to fail if Test Coverage drops","user":{"login":"pabender","id":14183416,"avatar_url":"https://avatars.githubusercontent.com/u/14183416?v=3","gravatar_id":"","url":"https://api.github.com/users/pabender","html_url":"https://github.com/pabender","followers_url":"https://api.github.com/users/pabender/followers","following_url":"https://api.github.com/users/pabender/following{/other_user}","gists_url":"https://api.github.com/users/pabender/gists{/gist_id}","starred_url":"https://api.github.com/users/pabender/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pabender/subscriptions","organizations_url":"https://api.github.com/users/pabender/orgs","repos_url":"https://api.github.com/users/pabender/repos","events_url":"https://api.github.com/users/pabender/events{/privacy}","received_events_url":"https://api.github.com/users/pabender/received_events","type":"User","site_admin":false},"labels":[{"id":299186645,"url":"https://api.github.com/repos/JMRI/JMRI/labels/dev/test/publish%20infrastructure","name":"dev/test/publish infrastructure","color":"fef2c0","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":3,"created_at":"2016-11-14T02:15:03Z","updated_at":"2016-11-14T14:33:27Z","closed_at":null,"body":"Hello All,\r\n\r\nWe hit a bit of a milestone with our testing today.  Jacoco on Jenkins is now reporting 50% class coverage.  When we started running the Jacoco builds back in early summer, we were getting somewhere around 25-30% coverage (I can't go back in the history that far) so this is a huge improvement.\r\n\r\n(Build 391 actually shows we've covered 3408 classes and have missed 3439, so this is a rounded 50%, and we need to have another 16 classes covered to hit the actual mark).\r\n\r\nMoving forward, I want to make sure we don't slip on our coverage numbers.  The Jacoco ant plugin actually supports failing a build if the coverage numbers don't hit a certain threshold, but we haven't been using that feature.  I'm in the process of modifying the build scripts to turn that feature on in at least one of the CI tests.  This will let us know before merging code if we slip.\r\n\r\nand just FYI, here's what the failure output looks like (for a check of the class coverage ratio on the whole repository):\r\nalltest-coveragecheck:\r\n[jacoco:report] Loading execution data file /home/Paul/JMRI/git/JMRI/jacoco.exec\r\n[jacoco:report] Writing bundle 'AntTestReporting' with 6847 classes\r\n[jacoco:report] Rule violated for bundle AntTestReporting: classes covered ratio is 0.49, but expected minimum is 0.50\r\n\r\nPaul"},"comment":{"url":"https://api.github.com/repos/JMRI/JMRI/issues/comments/260349931","html_url":"https://github.com/JMRI/JMRI/issues/2506#issuecomment-260349931","issue_url":"https://api.github.com/repos/JMRI/JMRI/issues/2506","id":260349931,"user":{"login":"bobjacobsen","id":2774117,"avatar_url":"https://avatars.githubusercontent.com/u/2774117?v=3","gravatar_id":"","url":"https://api.github.com/users/bobjacobsen","html_url":"https://github.com/bobjacobsen","followers_url":"https://api.github.com/users/bobjacobsen/followers","following_url":"https://api.github.com/users/bobjacobsen/following{/other_user}","gists_url":"https://api.github.com/users/bobjacobsen/gists{/gist_id}","starred_url":"https://api.github.com/users/bobjacobsen/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bobjacobsen/subscriptions","organizations_url":"https://api.github.com/users/bobjacobsen/orgs","repos_url":"https://api.github.com/users/bobjacobsen/repos","events_url":"https://api.github.com/users/bobjacobsen/events{/privacy}","received_events_url":"https://api.github.com/users/bobjacobsen/received_events","type":"User","site_admin":false},"created_at":"2016-11-14T14:33:27Z","updated_at":"2016-11-14T14:33:27Z","body":"I really do value the CI tests and associated infrastructure.  A lot of work has gone into making the code more robust. Multiple people have worked hard on this. Having more robust, maintainable code is important in the long run.\r\n\r\nAt the same time, the rate at which new people contribute has gone way down. Adding new people to the developer team is also important in the long run, probably more important.\r\n\r\nI think it's reasonable to hold seasoned JMRI developers to a high standard on tests and other aspects of code quality.  \r\n\r\nI'm not at all sure it's wise to do that for everybody. In particular, I think it's counterproductive to _require_ somebody new who contributes a small fix or addition (new code) to provide tests for it. Better to get the fix or addition, make that experience rewarding for the new contributor, and work from there. \r\n\r\nSo having CI count the coverage of classes (generally, first contributions aren't entire classes, but it has happened) seems OK to me.  Even raising a fail when it drops below a threshold is fine. I don't think that'll happen very often.  But I'd rather not get a lot tighter than that."}}
{"actor":{"display_login":"bobjacobsen"}}