{"action":"created","issue":{"url":"https://api.github.com/repos/KTH/quixbugs-experiment/issues/1","repository_url":"https://api.github.com/repos/KTH/quixbugs-experiment","labels_url":"https://api.github.com/repos/KTH/quixbugs-experiment/issues/1/labels{/name}","comments_url":"https://api.github.com/repos/KTH/quixbugs-experiment/issues/1/comments","events_url":"https://api.github.com/repos/KTH/quixbugs-experiment/issues/1/events","html_url":"https://github.com/KTH/quixbugs-experiment/issues/1","id":303586783,"number":1,"title":"failing Evosuite test cases","user":{"login":"martinezmatias","id":7804830,"avatar_url":"https://avatars3.githubusercontent.com/u/7804830?v=4","gravatar_id":"","url":"https://api.github.com/users/martinezmatias","html_url":"https://github.com/martinezmatias","followers_url":"https://api.github.com/users/martinezmatias/followers","following_url":"https://api.github.com/users/martinezmatias/following{/other_user}","gists_url":"https://api.github.com/users/martinezmatias/gists{/gist_id}","starred_url":"https://api.github.com/users/martinezmatias/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/martinezmatias/subscriptions","organizations_url":"https://api.github.com/users/martinezmatias/orgs","repos_url":"https://api.github.com/users/martinezmatias/repos","events_url":"https://api.github.com/users/martinezmatias/events{/privacy}","received_events_url":"https://api.github.com/users/martinezmatias/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2018-03-08T18:39:10Z","updated_at":"2018-03-09T10:46:54Z","closed_at":null,"author_association":"COLLABORATOR","body":"@SophieHYe \r\nEvosuite generates test cases that fail on the version used for generating them. \r\nA [test case ](https://github.com/KTH/quixbugs-experiment/blob/master/test_executors_java/src/test/java/qfevotest/generator/EvoTestGeneratorTest.java#L62)reproduces the bug \r\n\r\nI executed Evosuite command over the correct version of repository QuixBug (ie., without our infrastructure) and the problem continues.\r\nThe command used was: java -jar <local_path>/quixbugs-experiment/test_executors_java/./libs/evosuite-master-1.0.6-SNAPSHOT.jar -class correct_java_programs.POSSIBLE_CHANGE -projectCP <local_path_to_quixbug>/target/classes -base_dir <local_path_to_quixbug>/src/main/java/correct_java_programs_test -Dglobal_timeout 120 -seed 1 -Drandom_seed 1 -Dsearch_budget 60 -Dstopping_condition MaxTime -Dsandbox false -Dno_runtime_dependency true -mem 2000 -Djunit_check true\r\n"},"comment":{"url":"https://api.github.com/repos/KTH/quixbugs-experiment/issues/comments/371779069","html_url":"https://github.com/KTH/quixbugs-experiment/issues/1#issuecomment-371779069","issue_url":"https://api.github.com/repos/KTH/quixbugs-experiment/issues/1","id":371779069,"user":{"login":"SophieHYe","id":29943784,"avatar_url":"https://avatars0.githubusercontent.com/u/29943784?v=4","gravatar_id":"","url":"https://api.github.com/users/SophieHYe","html_url":"https://github.com/SophieHYe","followers_url":"https://api.github.com/users/SophieHYe/followers","following_url":"https://api.github.com/users/SophieHYe/following{/other_user}","gists_url":"https://api.github.com/users/SophieHYe/gists{/gist_id}","starred_url":"https://api.github.com/users/SophieHYe/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/SophieHYe/subscriptions","organizations_url":"https://api.github.com/users/SophieHYe/orgs","repos_url":"https://api.github.com/users/SophieHYe/repos","events_url":"https://api.github.com/users/SophieHYe/events{/privacy}","received_events_url":"https://api.github.com/users/SophieHYe/received_events","type":"User","site_admin":false},"created_at":"2018-03-09T10:46:54Z","updated_at":"2018-03-09T10:46:54Z","author_association":"COLLABORATOR","body":"@martinezmatias \r\nHi Matias,\r\n\r\nI looked at the possible_change tests generate by Evosuite.  I would think the problem is because our \"correct\" version is not exactly \"correct\".  \r\n\r\nIn the correct version, two issues are not considered well and EvoSuites expose these two problems:\r\n(1) First, the tests failed because of StackOverFlowError. The correct version missed some conditions to stop the interative method calls.\r\ntest2/test4/test5 from seed 1 expose this problem when the coins is not null and  total number is alway the positive integer, the interation will not stop.\r\n(2) Second, even test7 is passed but it catch NullPointerException which is not handle well in the correct version. (This is my fault in writing the correct version according to their correct Python version)\r\n\r\nThe correct version passed all the tests cases we generated from Json tests because those tests not exposed the problems and make us think it is the correct version.\r\n\r\nTo test it, I translated the Evosuite test2/test4 to Jsons ([[[-2124], 1], 20]; [[[0, 0, 1161, -2685, 639], 639], 20]) to test correct Python programs and they failed as well with similar errors: \r\n\r\n[[[-2124], 1], 20]\r\nCorrect Python: (<class 'RecursionError'>, RecursionError('maximum recursion depth exceeded in comparison',), <traceback object at 0x10163fe08>)\r\n\r\n[[[0, 0, 1161, -2685, 639], 639], 20]\r\nCorrect Python: (<class 'RecursionError'>, RecursionError('maximum recursion depth exceeded in comparison',), <traceback object at 0x10178d888>)\r\n\r\n\r\n\r\n\r\n\r\n"}}
{"actor":{"display_login":"SophieHYe"}}