{"action":"created","comment":{"url":"https://api.github.com/repos/linkedin/photon-ml/pulls/comments/61639672","id":61639672,"diff_hunk":"@@ -87,67 +89,4 @@ class BootstrapTrainingIntegTest extends SparkTestUtils {\n       }\n     })\n   }\n-\n-  /**\n-   * \"Real\" integration test where we hook in all the aggregation operations and sanity check their output\n-   */\n-  @Test\n-  def checkBootstrapHappyPathRealAggregates(): Unit = sparkTest(\"checkBootstrapHappyPathRealAggregates\") {\n-    // Return a different model each time fitFunction is called\n-    var count: Int = -BootstrapTrainingTest.HALF_NUM_SAMPLES\n-    val fitFunction = (x: RDD[LabeledPoint], y: Map[Double, LinearRegressionModel]) => {\n-      val value = count / BootstrapTrainingTest.HALF_NUM_SAMPLES.toDouble\n-      count += 1\n-      val fn = regressionModelFitFunction(value, lambdas)\n-      fn(x, y)\n-    }\n-\n-\n-    val confidenceIntervalsKey = \"confidenceIntervalEstimate\"\n-    val metricsIntervalsKey = \"metricsIntervalEstimate\"\n-    val aggregations: Map[String, Seq[(LinearRegressionModel, Map[String, Double])] => Any] = Map(\n-      confidenceIntervalsKey -> BootstrapTraining.aggregateCoefficientConfidenceIntervals,\n-      metricsIntervalsKey -> BootstrapTraining.aggregateMetricsConfidenceIntervals\n-    )\n-\n-    val validateConfidenceIntervals: Any => Unit = x => {\n-      x match {\n-        case (coeff: Array[CoefficientSummary], intercept: Option[CoefficientSummary]) =>\n-          coeff.foreach(c => {\n-            BootstrapTrainingTest.checkCoefficientSummary(c)\n-          })\n-          intercept match {\n-            case Some(_) => fail(\"Intercept should not have been computed\")\n-            case None =>\n-          }\n-        case _ =>\n-          fail(s\"Aggregate for $confidenceIntervalsKey is of unexpected type\")\n-      }\n-    }: Unit\n-\n-    val validateMetricsIntervals: Any => Unit = x => {\n-      x match {\n-        case m: Map[String, CoefficientSummary] =>\n-        case _ =>\n-          fail(s\"Aggregate for $metricsIntervalsKey is of unexpected type\")\n-      }\n-    }: Unit\n-\n-    val aggregationValidators: Map[String, Any => Unit] = Map(\n-      confidenceIntervalsKey -> validateConfidenceIntervals,\n-      metricsIntervalsKey -> validateMetricsIntervals\n-    )\n-\n-    val data: RDD[LabeledPoint] = sc.parallelize(drawSampleFromNumericallyBenignDenseFeaturesForLinearRegressionLocal(seed.toInt, numSamples, BootstrapTrainingTest.NUM_DIMENSIONS).toSeq).map(x => new LabeledPoint(x._1, x._2)).coalesce(4)\n-\n-    val aggregates: Map[Double, Map[String, Any]] = BootstrapTraining.bootstrap[LinearRegressionModel](\n-      BootstrapTrainingTest.NUM_SAMPLES,\n-      samplePct,\n-      Map[Double, LinearRegressionModel](),\n-      fitFunction,\n-      aggregations,\n-      data)\n-  }\n }","path":"photon-ml/src/integTest/scala/com/linkedin/photon/ml/BootstrapTrainingIntegTest.scala","position":88,"original_position":88,"commit_id":"235cbee342773cb776abe873b22df15416f055f9","original_commit_id":"235cbee342773cb776abe873b22df15416f055f9","user":{"login":"XianXing","id":2010084,"avatar_url":"https://avatars.githubusercontent.com/u/2010084?v=3","gravatar_id":"","url":"https://api.github.com/users/XianXing","html_url":"https://github.com/XianXing","followers_url":"https://api.github.com/users/XianXing/followers","following_url":"https://api.github.com/users/XianXing/following{/other_user}","gists_url":"https://api.github.com/users/XianXing/gists{/gist_id}","starred_url":"https://api.github.com/users/XianXing/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/XianXing/subscriptions","organizations_url":"https://api.github.com/users/XianXing/orgs","repos_url":"https://api.github.com/users/XianXing/repos","events_url":"https://api.github.com/users/XianXing/events{/privacy}","received_events_url":"https://api.github.com/users/XianXing/received_events","type":"User","site_admin":false},"body":"We know that the IntegTest is necessary here for BootstrapTraining class, given the requirement of spark context. But I am asking because I didn't see ```aggregates: Map[Double, Map[String, Any]]``` used anywhere in the test to check whether we got expected results, and in the comment (line 91-93) it's being said that \"and  sanity check their output\", but I didn't see the sanity check. For example, from line 67-89 from the previous tests we know how the testing results are checked. \r\n\r\nAs a result, what we see are a few vals (e.g., line 136-139) hanging there being defined but not used. And the only chance (please correct me if I am wrong) this test would fail is when those sanity checks found in line 156-162 of BootstrapTraining.scala fail, or because of some Spark internal runtime failures.\r\n\r\nHowever, I think you made a good point here. Let me add these lines of code back and let's provide a few more sanity checking for the tests in a separate PR in order to make it even more useful.","created_at":"2016-04-29T20:26:08Z","updated_at":"2016-04-29T20:26:08Z","html_url":"https://github.com/linkedin/photon-ml/pull/48#discussion_r61639672","pull_request_url":"https://api.github.com/repos/linkedin/photon-ml/pulls/48","_links":{"self":{"href":"https://api.github.com/repos/linkedin/photon-ml/pulls/comments/61639672"},"html":{"href":"https://github.com/linkedin/photon-ml/pull/48#discussion_r61639672"},"pull_request":{"href":"https://api.github.com/repos/linkedin/photon-ml/pulls/48"}}},"pull_request":{"url":"https://api.github.com/repos/linkedin/photon-ml/pulls/48","id":68334044,"html_url":"https://github.com/linkedin/photon-ml/pull/48","diff_url":"https://github.com/linkedin/photon-ml/pull/48.diff","patch_url":"https://github.com/linkedin/photon-ml/pull/48.patch","issue_url":"https://api.github.com/repos/linkedin/photon-ml/issues/48","number":48,"state":"open","locked":false,"title":"Simplify redundant integ tests and reduce integTest runtime","user":{"login":"XianXing","id":2010084,"avatar_url":"https://avatars.githubusercontent.com/u/2010084?v=3","gravatar_id":"","url":"https://api.github.com/users/XianXing","html_url":"https://github.com/XianXing","followers_url":"https://api.github.com/users/XianXing/followers","following_url":"https://api.github.com/users/XianXing/following{/other_user}","gists_url":"https://api.github.com/users/XianXing/gists{/gist_id}","starred_url":"https://api.github.com/users/XianXing/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/XianXing/subscriptions","organizations_url":"https://api.github.com/users/XianXing/orgs","repos_url":"https://api.github.com/users/XianXing/repos","events_url":"https://api.github.com/users/XianXing/events{/privacy}","received_events_url":"https://api.github.com/users/XianXing/received_events","type":"User","site_admin":false},"body":"Simplify redundant integ tests and reduce integTest runtime, in order to address issue #19.\r\n\r\nBefore: integTest takes 40 mins - 1 hr\r\nAfter: integTest takes ~20 mins\r\n\r\nAnd the integration tests coverage are not reduced.","created_at":"2016-04-29T07:35:24Z","updated_at":"2016-04-29T20:26:08Z","closed_at":null,"merged_at":null,"merge_commit_sha":"e92ba5287f020c8178c362ce787c7fe5eb25abd3","assignee":null,"milestone":null,"commits_url":"https://api.github.com/repos/linkedin/photon-ml/pulls/48/commits","review_comments_url":"https://api.github.com/repos/linkedin/photon-ml/pulls/48/comments","review_comment_url":"https://api.github.com/repos/linkedin/photon-ml/pulls/comments{/number}","comments_url":"https://api.github.com/repos/linkedin/photon-ml/issues/48/comments","statuses_url":"https://api.github.com/repos/linkedin/photon-ml/statuses/235cbee342773cb776abe873b22df15416f055f9","head":{"label":"XianXing:master","ref":"master","sha":"235cbee342773cb776abe873b22df15416f055f9","user":{"login":"XianXing","id":2010084,"avatar_url":"https://avatars.githubusercontent.com/u/2010084?v=3","gravatar_id":"","url":"https://api.github.com/users/XianXing","html_url":"https://github.com/XianXing","followers_url":"https://api.github.com/users/XianXing/followers","following_url":"https://api.github.com/users/XianXing/following{/other_user}","gists_url":"https://api.github.com/users/XianXing/gists{/gist_id}","starred_url":"https://api.github.com/users/XianXing/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/XianXing/subscriptions","organizations_url":"https://api.github.com/users/XianXing/orgs","repos_url":"https://api.github.com/users/XianXing/repos","events_url":"https://api.github.com/users/XianXing/events{/privacy}","received_events_url":"https://api.github.com/users/XianXing/received_events","type":"User","site_admin":false},"repo":{"id":56644340,"name":"photon-ml","full_name":"XianXing/photon-ml","owner":{"login":"XianXing","id":2010084,"avatar_url":"https://avatars.githubusercontent.com/u/2010084?v=3","gravatar_id":"","url":"https://api.github.com/users/XianXing","html_url":"https://github.com/XianXing","followers_url":"https://api.github.com/users/XianXing/followers","following_url":"https://api.github.com/users/XianXing/following{/other_user}","gists_url":"https://api.github.com/users/XianXing/gists{/gist_id}","starred_url":"https://api.github.com/users/XianXing/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/XianXing/subscriptions","organizations_url":"https://api.github.com/users/XianXing/orgs","repos_url":"https://api.github.com/users/XianXing/repos","events_url":"https://api.github.com/users/XianXing/events{/privacy}","received_events_url":"https://api.github.com/users/XianXing/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/XianXing/photon-ml","description":"A scalable machine learning library on Apache Spark","fork":true,"url":"https://api.github.com/repos/XianXing/photon-ml","forks_url":"https://api.github.com/repos/XianXing/photon-ml/forks","keys_url":"https://api.github.com/repos/XianXing/photon-ml/keys{/key_id}","collaborators_url":"https://api.github.com/repos/XianXing/photon-ml/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/XianXing/photon-ml/teams","hooks_url":"https://api.github.com/repos/XianXing/photon-ml/hooks","issue_events_url":"https://api.github.com/repos/XianXing/photon-ml/issues/events{/number}","events_url":"https://api.github.com/repos/XianXing/photon-ml/events","assignees_url":"https://api.github.com/repos/XianXing/photon-ml/assignees{/user}","branches_url":"https://api.github.com/repos/XianXing/photon-ml/branches{/branch}","tags_url":"https://api.github.com/repos/XianXing/photon-ml/tags","blobs_url":"https://api.github.com/repos/XianXing/photon-ml/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/XianXing/photon-ml/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/XianXing/photon-ml/git/refs{/sha}","trees_url":"https://api.github.com/repos/XianXing/photon-ml/git/trees{/sha}","statuses_url":"https://api.github.com/repos/XianXing/photon-ml/statuses/{sha}","languages_url":"https://api.github.com/repos/XianXing/photon-ml/languages","stargazers_url":"https://api.github.com/repos/XianXing/photon-ml/stargazers","contributors_url":"https://api.github.com/repos/XianXing/photon-ml/contributors","subscribers_url":"https://api.github.com/repos/XianXing/photon-ml/subscribers","subscription_url":"https://api.github.com/repos/XianXing/photon-ml/subscription","commits_url":"https://api.github.com/repos/XianXing/photon-ml/commits{/sha}","git_commits_url":"https://api.github.com/repos/XianXing/photon-ml/git/commits{/sha}","comments_url":"https://api.github.com/repos/XianXing/photon-ml/comments{/number}","issue_comment_url":"https://api.github.com/repos/XianXing/photon-ml/issues/comments{/number}","contents_url":"https://api.github.com/repos/XianXing/photon-ml/contents/{+path}","compare_url":"https://api.github.com/repos/XianXing/photon-ml/compare/{base}...{head}","merges_url":"https://api.github.com/repos/XianXing/photon-ml/merges","archive_url":"https://api.github.com/repos/XianXing/photon-ml/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/XianXing/photon-ml/downloads","issues_url":"https://api.github.com/repos/XianXing/photon-ml/issues{/number}","pulls_url":"https://api.github.com/repos/XianXing/photon-ml/pulls{/number}","milestones_url":"https://api.github.com/repos/XianXing/photon-ml/milestones{/number}","notifications_url":"https://api.github.com/repos/XianXing/photon-ml/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/XianXing/photon-ml/labels{/name}","releases_url":"https://api.github.com/repos/XianXing/photon-ml/releases{/id}","deployments_url":"https://api.github.com/repos/XianXing/photon-ml/deployments","created_at":"2016-04-20T01:12:47Z","updated_at":"2016-04-20T01:12:48Z","pushed_at":"2016-04-29T07:33:08Z","git_url":"git://github.com/XianXing/photon-ml.git","ssh_url":"git@github.com:XianXing/photon-ml.git","clone_url":"https://github.com/XianXing/photon-ml.git","svn_url":"https://github.com/XianXing/photon-ml","homepage":"","size":16455,"stargazers_count":0,"watchers_count":0,"language":"Scala","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"linkedin:master","ref":"master","sha":"4d1e847df3dfc7fb400c1884b0bd96e059fcfc70","user":{"login":"linkedin","id":357098,"avatar_url":"https://avatars.githubusercontent.com/u/357098?v=3","gravatar_id":"","url":"https://api.github.com/users/linkedin","html_url":"https://github.com/linkedin","followers_url":"https://api.github.com/users/linkedin/followers","following_url":"https://api.github.com/users/linkedin/following{/other_user}","gists_url":"https://api.github.com/users/linkedin/gists{/gist_id}","starred_url":"https://api.github.com/users/linkedin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/linkedin/subscriptions","organizations_url":"https://api.github.com/users/linkedin/orgs","repos_url":"https://api.github.com/users/linkedin/repos","events_url":"https://api.github.com/users/linkedin/events{/privacy}","received_events_url":"https://api.github.com/users/linkedin/received_events","type":"Organization","site_admin":false},"repo":{"id":50964104,"name":"photon-ml","full_name":"linkedin/photon-ml","owner":{"login":"linkedin","id":357098,"avatar_url":"https://avatars.githubusercontent.com/u/357098?v=3","gravatar_id":"","url":"https://api.github.com/users/linkedin","html_url":"https://github.com/linkedin","followers_url":"https://api.github.com/users/linkedin/followers","following_url":"https://api.github.com/users/linkedin/following{/other_user}","gists_url":"https://api.github.com/users/linkedin/gists{/gist_id}","starred_url":"https://api.github.com/users/linkedin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/linkedin/subscriptions","organizations_url":"https://api.github.com/users/linkedin/orgs","repos_url":"https://api.github.com/users/linkedin/repos","events_url":"https://api.github.com/users/linkedin/events{/privacy}","received_events_url":"https://api.github.com/users/linkedin/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/linkedin/photon-ml","description":"A scalable machine learning library on Apache Spark","fork":false,"url":"https://api.github.com/repos/linkedin/photon-ml","forks_url":"https://api.github.com/repos/linkedin/photon-ml/forks","keys_url":"https://api.github.com/repos/linkedin/photon-ml/keys{/key_id}","collaborators_url":"https://api.github.com/repos/linkedin/photon-ml/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/linkedin/photon-ml/teams","hooks_url":"https://api.github.com/repos/linkedin/photon-ml/hooks","issue_events_url":"https://api.github.com/repos/linkedin/photon-ml/issues/events{/number}","events_url":"https://api.github.com/repos/linkedin/photon-ml/events","assignees_url":"https://api.github.com/repos/linkedin/photon-ml/assignees{/user}","branches_url":"https://api.github.com/repos/linkedin/photon-ml/branches{/branch}","tags_url":"https://api.github.com/repos/linkedin/photon-ml/tags","blobs_url":"https://api.github.com/repos/linkedin/photon-ml/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/linkedin/photon-ml/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/linkedin/photon-ml/git/refs{/sha}","trees_url":"https://api.github.com/repos/linkedin/photon-ml/git/trees{/sha}","statuses_url":"https://api.github.com/repos/linkedin/photon-ml/statuses/{sha}","languages_url":"https://api.github.com/repos/linkedin/photon-ml/languages","stargazers_url":"https://api.github.com/repos/linkedin/photon-ml/stargazers","contributors_url":"https://api.github.com/repos/linkedin/photon-ml/contributors","subscribers_url":"https://api.github.com/repos/linkedin/photon-ml/subscribers","subscription_url":"https://api.github.com/repos/linkedin/photon-ml/subscription","commits_url":"https://api.github.com/repos/linkedin/photon-ml/commits{/sha}","git_commits_url":"https://api.github.com/repos/linkedin/photon-ml/git/commits{/sha}","comments_url":"https://api.github.com/repos/linkedin/photon-ml/comments{/number}","issue_comment_url":"https://api.github.com/repos/linkedin/photon-ml/issues/comments{/number}","contents_url":"https://api.github.com/repos/linkedin/photon-ml/contents/{+path}","compare_url":"https://api.github.com/repos/linkedin/photon-ml/compare/{base}...{head}","merges_url":"https://api.github.com/repos/linkedin/photon-ml/merges","archive_url":"https://api.github.com/repos/linkedin/photon-ml/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/linkedin/photon-ml/downloads","issues_url":"https://api.github.com/repos/linkedin/photon-ml/issues{/number}","pulls_url":"https://api.github.com/repos/linkedin/photon-ml/pulls{/number}","milestones_url":"https://api.github.com/repos/linkedin/photon-ml/milestones{/number}","notifications_url":"https://api.github.com/repos/linkedin/photon-ml/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/linkedin/photon-ml/labels{/name}","releases_url":"https://api.github.com/repos/linkedin/photon-ml/releases{/id}","deployments_url":"https://api.github.com/repos/linkedin/photon-ml/deployments","created_at":"2016-02-03T01:12:07Z","updated_at":"2016-04-26T09:01:59Z","pushed_at":"2016-04-29T07:35:24Z","git_url":"git://github.com/linkedin/photon-ml.git","ssh_url":"git@github.com:linkedin/photon-ml.git","clone_url":"https://github.com/linkedin/photon-ml.git","svn_url":"https://github.com/linkedin/photon-ml","homepage":"","size":16549,"stargazers_count":10,"watchers_count":10,"language":"Scala","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":4,"mirror_url":null,"open_issues_count":13,"forks":4,"open_issues":13,"watchers":10,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/linkedin/photon-ml/pulls/48"},"html":{"href":"https://github.com/linkedin/photon-ml/pull/48"},"issue":{"href":"https://api.github.com/repos/linkedin/photon-ml/issues/48"},"comments":{"href":"https://api.github.com/repos/linkedin/photon-ml/issues/48/comments"},"review_comments":{"href":"https://api.github.com/repos/linkedin/photon-ml/pulls/48/comments"},"review_comment":{"href":"https://api.github.com/repos/linkedin/photon-ml/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/linkedin/photon-ml/pulls/48/commits"},"statuses":{"href":"https://api.github.com/repos/linkedin/photon-ml/statuses/235cbee342773cb776abe873b22df15416f055f9"}}}}