{"action":"created","issue":{"url":"https://api.github.com/repos/apache/flink/issues/1704","repository_url":"https://api.github.com/repos/apache/flink","labels_url":"https://api.github.com/repos/apache/flink/issues/1704/labels{/name}","comments_url":"https://api.github.com/repos/apache/flink/issues/1704/comments","events_url":"https://api.github.com/repos/apache/flink/issues/1704/events","html_url":"https://github.com/apache/flink/pull/1704","id":136042937,"number":1704,"title":"[FLINK-1159] Case style anonymous functions not supported by Scala API","user":{"login":"stefanobaghino","id":1374489,"avatar_url":"https://avatars.githubusercontent.com/u/1374489?v=3","gravatar_id":"","url":"https://api.github.com/users/stefanobaghino","html_url":"https://github.com/stefanobaghino","followers_url":"https://api.github.com/users/stefanobaghino/followers","following_url":"https://api.github.com/users/stefanobaghino/following{/other_user}","gists_url":"https://api.github.com/users/stefanobaghino/gists{/gist_id}","starred_url":"https://api.github.com/users/stefanobaghino/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/stefanobaghino/subscriptions","organizations_url":"https://api.github.com/users/stefanobaghino/orgs","repos_url":"https://api.github.com/users/stefanobaghino/repos","events_url":"https://api.github.com/users/stefanobaghino/events{/privacy}","received_events_url":"https://api.github.com/users/stefanobaghino/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":18,"created_at":"2016-02-24T12:08:22Z","updated_at":"2016-03-12T22:11:37Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/flink/pulls/1704","html_url":"https://github.com/apache/flink/pull/1704","diff_url":"https://github.com/apache/flink/pull/1704.diff","patch_url":"https://github.com/apache/flink/pull/1704.patch"},"body":"The proposed API extension methods would allow developers to pass a pattern matching anonymous function so that they are applied on a `DataSet` or `DataStream`; many methods defined on the `DataSet` and `DataStream` APIs don't support those functions due to the overloading[[1]][[2]]; pattern matching anonymous functions allow a very idiomatic approach in Scala to *decompose tuples, case classes and collections*.\r\n\r\nThe PR does not pollute the original `DataSet` and `DataStream` APIs but is provided as an optional set of extensions methods, implemented via implicit conversions and made available to the developer by explicitly importing the required package, e.g.:\r\n\r\n```scala\r\nimport org.apache.flink.api.scala.extensions.acceptPartialFunctions\r\nenv.fromElements('a -> 1, 'b -> 2).mapWith {\r\n  case (key, value) =>\r\n    ... // key and value are now available and have a sensible name\r\n}\r\n```\r\n\r\n[1]: https://groups.google.com/d/msg/scala-user/3oHnDEl1UsM/dDNir9BsiG4J\r\n[2]: http://www.scala-lang.org/files/archive/spec/2.11/06-expressions.html#overloading-resolution"},"comment":{"url":"https://api.github.com/repos/apache/flink/issues/comments/195818369","html_url":"https://github.com/apache/flink/pull/1704#issuecomment-195818369","issue_url":"https://api.github.com/repos/apache/flink/issues/1704","id":195818369,"user":{"login":"stefanobaghino","id":1374489,"avatar_url":"https://avatars.githubusercontent.com/u/1374489?v=3","gravatar_id":"","url":"https://api.github.com/users/stefanobaghino","html_url":"https://github.com/stefanobaghino","followers_url":"https://api.github.com/users/stefanobaghino/followers","following_url":"https://api.github.com/users/stefanobaghino/following{/other_user}","gists_url":"https://api.github.com/users/stefanobaghino/gists{/gist_id}","starred_url":"https://api.github.com/users/stefanobaghino/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/stefanobaghino/subscriptions","organizations_url":"https://api.github.com/users/stefanobaghino/orgs","repos_url":"https://api.github.com/users/stefanobaghino/repos","events_url":"https://api.github.com/users/stefanobaghino/events{/privacy}","received_events_url":"https://api.github.com/users/stefanobaghino/received_events","type":"User","site_admin":false},"created_at":"2016-03-12T22:11:37Z","updated_at":"2016-03-12T22:11:37Z","body":"Thanks for the feedback. Basically the tests so far have been a carbon copy of the ones run on the operators I delegate to from the extensions. I agree that this approach is a little bit heavy and would like to make it lighter but perhaps I can use some help on this.\r\n\r\nI've made a few changes that perhaps go in the right direction in my latest commit ([source here](https://github.com/radicalbit/flink/blob/3e6c58c84c977299d05ef9c9675ecda31368731a/flink-tests/src/test/scala/org/apache/flink/api/scala/extensions/AcceptPFMapITCase.scala)).\r\n\r\nHere's a snippet:\r\n\r\n```scala\r\nclass AcceptPFMapITCase extends TestLogger {\r\n\r\n  private val optimizer = new Optimizer(new Configuration)\r\n  private val unusedResultPath = \"UNUSED\"\r\n\r\n  def isTransformation(n: PlanNode): Boolean =\r\n    !n.isInstanceOf[SourcePlanNode] && !n.isInstanceOf[SinkPlanNode]\r\n\r\n  private def getOptimizerTransformationNodes(env: ExecutionEnvironment): Iterable[OptimizerNode] =\r\n    for {\r\n      node <- optimizer.compile(env.createProgramPlan(\"UNUSED\")).getAllNodes\r\n      transformation = node.getOptimizerNode if isTransformation(node)\r\n    } yield transformation\r\n\r\n  @Test\r\n  def testIdentityMapperWithBasicType(): Unit = {\r\n    val env = ExecutionEnvironment.getExecutionEnvironment\r\n    val ds = CollectionDataSets.getStringDataSet(env)\r\n    val identityMapDs = ds.mapWith(identity)\r\n    identityMapDs.writeAsText(unusedResultPath, WriteMode.OVERWRITE)\r\n    val nodes = getOptimizerTransformationNodes(env)\r\n    assertTrue(\"The plan should contain 1 transformation\", nodes.size == 1)\r\n    assertTrue(\"The transformation should be a map\", nodes.forall(_.isInstanceOf[MapNode]))\r\n  }\r\n\r\n  /* [...] */\r\n```\r\n\r\nIf this approach sounds fine, I can apply it to the other tests as well, along with a wider coverage of the extension methods.\r\n\r\nApart from the approach in general, I have a couple of doubts:\r\n* is it safe to share an `Optimizer` among several tests (i.e. no state involved)?\r\n* is it enough to test the transformation or should I test the input and output types as well?"}}