{"action":"created","comment":{"url":"https://api.github.com/repos/adelbertc/frameless/pulls/comments/87703318","pull_request_review_id":8306270,"id":87703318,"diff_hunk":"@@ -0,0 +1,81 @@\n+# Getting started\n+We are going to manipulate some `TypedDataset`s through a small toy example.\n+Before starting the tutorial, let's import/define a couple of things.\n+```tut:book\n+import org.apache.spark.{SparkConf, SparkContext}\n+import org.apache.spark.sql.SparkSession\n+import frameless.functions.aggregate._\n+import frameless.TypedDataset\n+\n+val conf = new SparkConf().setMaster(\"local[*]\").setAppName(\"frameless repl\").set(\"spark.ui.enabled\", \"false\")\n+val spark = SparkSession.builder().config(conf).appName(\"REPL\").getOrCreate()\n+implicit val sqlContext = spark.sqlContext\n+spark.sparkContext.setLogLevel(\"WARN\")\n+\n+import spark.implicits._\n+``` \n+## Creating TypedDataset instances\n+We start by defining a simple case class that will be the basis of our examples.\n+```tut:book\n+case class Appartment(city: String, surface: Int, price: Double)\n+``` \n+And let's define a few `Appartment` instances:\n+```tut:book\n+val appartments = Seq(\n+    Appartment(\"Paris\", 50, 300000.0),\n+    Appartment(\"Paris\", 100, 450000.0),\n+    Appartment(\"Paris\", 25, 250000.0),\n+    Appartment(\"Lyon\", 83, 200000.0),\n+    Appartment(\"Lyon\", 45, 133000.0),\n+    Appartment(\"Nice\", 74, 325000.0)\n+    )\n+```\n+We can create our `TypedDataset` directly :\n+```tut:book\n+val appartmentsTypedDS = TypedDataset.create(appartments)\n+```\n+We can also create it from an existing `Dataset` :\n+```tut:book\n+val appartmentsDS = spark.createDataset(appartments)\n+val appartmentsTypedDS = TypedDataset.create(appartmentsDS)\n+```\n+## Typesafe column referencing\n+Selecting a column from our `TypedDataset` is done in this manner : \n+```tut:book\n+val cities: TypedDataset[String] = appartmentsTypedDS.select(appartmentsTypedDS('city))\n+```\n+This is completely safe, for instance suppose we made a typo :\n+```tut:book:fail\n+appartmentsTypedDS.select(appartmentsTypedDS('citi))                                          ^\n+```\n+This gets caught at compile time.\n+With `Dataset` the error would have been caught at runtime.\n+```tut:book:fail\n+appartmentsDS.select('citi)\n+``` \n+Let us now try to get the price by surface unit : \n+```tut:book:fail\n+val priceBySurfaceUnit = appartmentsTypedDS.select(appartmentsTypedDS('price)/appartmentsTypedDS('surface))                                                                          ^\n+```\n+Argh! Looks like we can't divide a `TypedColumn` of `Double` by `Int`. \n+Well, we can cast our `Int`s to `Double`s explicitely to proceed with the computation.\n+```tut:book\n+val priceBySurfaceUnit = appartmentsTypedDS.select(appartmentsTypedDS('price)/appartmentsTypedDS('surface).cast[Double])\n+priceBySurfaceUnit.collect.run()\n+``` \n+Looks like it worked, but that `cast` looks unsafe right ? Actually it is safe.\n+Let's try to cast `TypedColumn` of `String` to `Double` : \n+```tut:book:fail\n+appartmentsTypedDS('city).cast[Double]                            ^\n+``` \n+The compile-time error tells us that to perform the cast, an evidence (in the form of `CatalystCast[String,Double]`) must be available.\n+## GroupBy and Aggregations\n+Let's suppose we wanted to retrieve the average appartment price in each city\n+```tut:book\n+val priceByCity = appartmentsTypedDS.groupBy(appartmentsTypedDS('city)).agg(avg(appartmentsTypedDS('price)))\n+priceByCity.collect().run()\n+``` \n+Again if we try to aggregate a column that can't be aggregated, we get a compile time error\n+```tut:book:fail\n+appartmentsTypedDS.groupBy(appartmentsTypedDS('city)).agg(avg(appartmentsTypedDS('city)))                                                         ^\n+```","path":"docs/src/main/tut/GettingStarted.md","position":81,"original_position":81,"commit_id":"fe5bde0ec71007db48c01a8dbbf98d9d3096f4b1","original_commit_id":"fe5bde0ec71007db48c01a8dbbf98d9d3096f4b1","user":{"login":"imarios","id":6893839,"avatar_url":"https://avatars.githubusercontent.com/u/6893839?v=3","gravatar_id":"","url":"https://api.github.com/users/imarios","html_url":"https://github.com/imarios","followers_url":"https://api.github.com/users/imarios/followers","following_url":"https://api.github.com/users/imarios/following{/other_user}","gists_url":"https://api.github.com/users/imarios/gists{/gist_id}","starred_url":"https://api.github.com/users/imarios/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/imarios/subscriptions","organizations_url":"https://api.github.com/users/imarios/orgs","repos_url":"https://api.github.com/users/imarios/repos","events_url":"https://api.github.com/users/imarios/events{/privacy}","received_events_url":"https://api.github.com/users/imarios/received_events","type":"User","site_admin":false},"body":"Again, we might want to add the spark.stop() so we are sure things are cleaned up. ","created_at":"2016-11-13T00:49:41Z","updated_at":"2016-11-13T00:50:08Z","html_url":"https://github.com/adelbertc/frameless/pull/66#discussion_r87703318","pull_request_url":"https://api.github.com/repos/adelbertc/frameless/pulls/66","_links":{"self":{"href":"https://api.github.com/repos/adelbertc/frameless/pulls/comments/87703318"},"html":{"href":"https://github.com/adelbertc/frameless/pull/66#discussion_r87703318"},"pull_request":{"href":"https://api.github.com/repos/adelbertc/frameless/pulls/66"}}},"pull_request":{"url":"https://api.github.com/repos/adelbertc/frameless/pulls/66","id":93372289,"html_url":"https://github.com/adelbertc/frameless/pull/66","diff_url":"https://github.com/adelbertc/frameless/pull/66.diff","patch_url":"https://github.com/adelbertc/frameless/pull/66.patch","issue_url":"https://api.github.com/repos/adelbertc/frameless/issues/66","number":66,"state":"open","locked":false,"title":"Getting started tutorial on a toy example","user":{"login":"bamine","id":5724915,"avatar_url":"https://avatars.githubusercontent.com/u/5724915?v=3","gravatar_id":"","url":"https://api.github.com/users/bamine","html_url":"https://github.com/bamine","followers_url":"https://api.github.com/users/bamine/followers","following_url":"https://api.github.com/users/bamine/following{/other_user}","gists_url":"https://api.github.com/users/bamine/gists{/gist_id}","starred_url":"https://api.github.com/users/bamine/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bamine/subscriptions","organizations_url":"https://api.github.com/users/bamine/orgs","repos_url":"https://api.github.com/users/bamine/repos","events_url":"https://api.github.com/users/bamine/events{/privacy}","received_events_url":"https://api.github.com/users/bamine/received_events","type":"User","site_admin":false},"body":"This is just a small getting started tutorial, trying to focus on a some practical aspects by going through an example. I don't if this is still relevant with @imarios PRs , maybe there is something to fuse/change.\r\nAnyway let me know what you think.","created_at":"2016-11-11T17:58:20Z","updated_at":"2016-11-13T00:50:08Z","closed_at":null,"merged_at":null,"merge_commit_sha":"03d73597088e4df3a15f4c326d01e82566f6b0d4","assignee":null,"assignees":[],"milestone":null,"commits_url":"https://api.github.com/repos/adelbertc/frameless/pulls/66/commits","review_comments_url":"https://api.github.com/repos/adelbertc/frameless/pulls/66/comments","review_comment_url":"https://api.github.com/repos/adelbertc/frameless/pulls/comments{/number}","comments_url":"https://api.github.com/repos/adelbertc/frameless/issues/66/comments","statuses_url":"https://api.github.com/repos/adelbertc/frameless/statuses/fe5bde0ec71007db48c01a8dbbf98d9d3096f4b1","head":{"label":"bamine:getting-started-docs","ref":"getting-started-docs","sha":"fe5bde0ec71007db48c01a8dbbf98d9d3096f4b1","user":{"login":"bamine","id":5724915,"avatar_url":"https://avatars.githubusercontent.com/u/5724915?v=3","gravatar_id":"","url":"https://api.github.com/users/bamine","html_url":"https://github.com/bamine","followers_url":"https://api.github.com/users/bamine/followers","following_url":"https://api.github.com/users/bamine/following{/other_user}","gists_url":"https://api.github.com/users/bamine/gists{/gist_id}","starred_url":"https://api.github.com/users/bamine/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bamine/subscriptions","organizations_url":"https://api.github.com/users/bamine/orgs","repos_url":"https://api.github.com/users/bamine/repos","events_url":"https://api.github.com/users/bamine/events{/privacy}","received_events_url":"https://api.github.com/users/bamine/received_events","type":"User","site_admin":false},"repo":{"id":60759596,"name":"frameless","full_name":"bamine/frameless","owner":{"login":"bamine","id":5724915,"avatar_url":"https://avatars.githubusercontent.com/u/5724915?v=3","gravatar_id":"","url":"https://api.github.com/users/bamine","html_url":"https://github.com/bamine","followers_url":"https://api.github.com/users/bamine/followers","following_url":"https://api.github.com/users/bamine/following{/other_user}","gists_url":"https://api.github.com/users/bamine/gists{/gist_id}","starred_url":"https://api.github.com/users/bamine/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bamine/subscriptions","organizations_url":"https://api.github.com/users/bamine/orgs","repos_url":"https://api.github.com/users/bamine/repos","events_url":"https://api.github.com/users/bamine/events{/privacy}","received_events_url":"https://api.github.com/users/bamine/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/bamine/frameless","description":"Expressive types for Spark.","fork":true,"url":"https://api.github.com/repos/bamine/frameless","forks_url":"https://api.github.com/repos/bamine/frameless/forks","keys_url":"https://api.github.com/repos/bamine/frameless/keys{/key_id}","collaborators_url":"https://api.github.com/repos/bamine/frameless/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/bamine/frameless/teams","hooks_url":"https://api.github.com/repos/bamine/frameless/hooks","issue_events_url":"https://api.github.com/repos/bamine/frameless/issues/events{/number}","events_url":"https://api.github.com/repos/bamine/frameless/events","assignees_url":"https://api.github.com/repos/bamine/frameless/assignees{/user}","branches_url":"https://api.github.com/repos/bamine/frameless/branches{/branch}","tags_url":"https://api.github.com/repos/bamine/frameless/tags","blobs_url":"https://api.github.com/repos/bamine/frameless/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/bamine/frameless/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/bamine/frameless/git/refs{/sha}","trees_url":"https://api.github.com/repos/bamine/frameless/git/trees{/sha}","statuses_url":"https://api.github.com/repos/bamine/frameless/statuses/{sha}","languages_url":"https://api.github.com/repos/bamine/frameless/languages","stargazers_url":"https://api.github.com/repos/bamine/frameless/stargazers","contributors_url":"https://api.github.com/repos/bamine/frameless/contributors","subscribers_url":"https://api.github.com/repos/bamine/frameless/subscribers","subscription_url":"https://api.github.com/repos/bamine/frameless/subscription","commits_url":"https://api.github.com/repos/bamine/frameless/commits{/sha}","git_commits_url":"https://api.github.com/repos/bamine/frameless/git/commits{/sha}","comments_url":"https://api.github.com/repos/bamine/frameless/comments{/number}","issue_comment_url":"https://api.github.com/repos/bamine/frameless/issues/comments{/number}","contents_url":"https://api.github.com/repos/bamine/frameless/contents/{+path}","compare_url":"https://api.github.com/repos/bamine/frameless/compare/{base}...{head}","merges_url":"https://api.github.com/repos/bamine/frameless/merges","archive_url":"https://api.github.com/repos/bamine/frameless/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/bamine/frameless/downloads","issues_url":"https://api.github.com/repos/bamine/frameless/issues{/number}","pulls_url":"https://api.github.com/repos/bamine/frameless/pulls{/number}","milestones_url":"https://api.github.com/repos/bamine/frameless/milestones{/number}","notifications_url":"https://api.github.com/repos/bamine/frameless/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/bamine/frameless/labels{/name}","releases_url":"https://api.github.com/repos/bamine/frameless/releases{/id}","deployments_url":"https://api.github.com/repos/bamine/frameless/deployments","created_at":"2016-06-09T08:18:42Z","updated_at":"2016-06-09T08:18:43Z","pushed_at":"2016-11-12T12:25:09Z","git_url":"git://github.com/bamine/frameless.git","ssh_url":"git@github.com:bamine/frameless.git","clone_url":"https://github.com/bamine/frameless.git","svn_url":"https://github.com/bamine/frameless","homepage":"","size":302,"stargazers_count":0,"watchers_count":0,"language":"Scala","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"adelbertc:master","ref":"master","sha":"b49c45a591b2d9772d02f5a582cec3cc3a976783","user":{"login":"adelbertc","id":1332980,"avatar_url":"https://avatars.githubusercontent.com/u/1332980?v=3","gravatar_id":"","url":"https://api.github.com/users/adelbertc","html_url":"https://github.com/adelbertc","followers_url":"https://api.github.com/users/adelbertc/followers","following_url":"https://api.github.com/users/adelbertc/following{/other_user}","gists_url":"https://api.github.com/users/adelbertc/gists{/gist_id}","starred_url":"https://api.github.com/users/adelbertc/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/adelbertc/subscriptions","organizations_url":"https://api.github.com/users/adelbertc/orgs","repos_url":"https://api.github.com/users/adelbertc/repos","events_url":"https://api.github.com/users/adelbertc/events{/privacy}","received_events_url":"https://api.github.com/users/adelbertc/received_events","type":"User","site_admin":false},"repo":{"id":33022955,"name":"frameless","full_name":"adelbertc/frameless","owner":{"login":"adelbertc","id":1332980,"avatar_url":"https://avatars.githubusercontent.com/u/1332980?v=3","gravatar_id":"","url":"https://api.github.com/users/adelbertc","html_url":"https://github.com/adelbertc","followers_url":"https://api.github.com/users/adelbertc/followers","following_url":"https://api.github.com/users/adelbertc/following{/other_user}","gists_url":"https://api.github.com/users/adelbertc/gists{/gist_id}","starred_url":"https://api.github.com/users/adelbertc/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/adelbertc/subscriptions","organizations_url":"https://api.github.com/users/adelbertc/orgs","repos_url":"https://api.github.com/users/adelbertc/repos","events_url":"https://api.github.com/users/adelbertc/events{/privacy}","received_events_url":"https://api.github.com/users/adelbertc/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/adelbertc/frameless","description":"Expressive types for Spark.","fork":false,"url":"https://api.github.com/repos/adelbertc/frameless","forks_url":"https://api.github.com/repos/adelbertc/frameless/forks","keys_url":"https://api.github.com/repos/adelbertc/frameless/keys{/key_id}","collaborators_url":"https://api.github.com/repos/adelbertc/frameless/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/adelbertc/frameless/teams","hooks_url":"https://api.github.com/repos/adelbertc/frameless/hooks","issue_events_url":"https://api.github.com/repos/adelbertc/frameless/issues/events{/number}","events_url":"https://api.github.com/repos/adelbertc/frameless/events","assignees_url":"https://api.github.com/repos/adelbertc/frameless/assignees{/user}","branches_url":"https://api.github.com/repos/adelbertc/frameless/branches{/branch}","tags_url":"https://api.github.com/repos/adelbertc/frameless/tags","blobs_url":"https://api.github.com/repos/adelbertc/frameless/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/adelbertc/frameless/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/adelbertc/frameless/git/refs{/sha}","trees_url":"https://api.github.com/repos/adelbertc/frameless/git/trees{/sha}","statuses_url":"https://api.github.com/repos/adelbertc/frameless/statuses/{sha}","languages_url":"https://api.github.com/repos/adelbertc/frameless/languages","stargazers_url":"https://api.github.com/repos/adelbertc/frameless/stargazers","contributors_url":"https://api.github.com/repos/adelbertc/frameless/contributors","subscribers_url":"https://api.github.com/repos/adelbertc/frameless/subscribers","subscription_url":"https://api.github.com/repos/adelbertc/frameless/subscription","commits_url":"https://api.github.com/repos/adelbertc/frameless/commits{/sha}","git_commits_url":"https://api.github.com/repos/adelbertc/frameless/git/commits{/sha}","comments_url":"https://api.github.com/repos/adelbertc/frameless/comments{/number}","issue_comment_url":"https://api.github.com/repos/adelbertc/frameless/issues/comments{/number}","contents_url":"https://api.github.com/repos/adelbertc/frameless/contents/{+path}","compare_url":"https://api.github.com/repos/adelbertc/frameless/compare/{base}...{head}","merges_url":"https://api.github.com/repos/adelbertc/frameless/merges","archive_url":"https://api.github.com/repos/adelbertc/frameless/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/adelbertc/frameless/downloads","issues_url":"https://api.github.com/repos/adelbertc/frameless/issues{/number}","pulls_url":"https://api.github.com/repos/adelbertc/frameless/pulls{/number}","milestones_url":"https://api.github.com/repos/adelbertc/frameless/milestones{/number}","notifications_url":"https://api.github.com/repos/adelbertc/frameless/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/adelbertc/frameless/labels{/name}","releases_url":"https://api.github.com/repos/adelbertc/frameless/releases{/id}","deployments_url":"https://api.github.com/repos/adelbertc/frameless/deployments","created_at":"2015-03-28T06:03:10Z","updated_at":"2016-11-08T06:16:21Z","pushed_at":"2016-11-12T12:25:10Z","git_url":"git://github.com/adelbertc/frameless.git","ssh_url":"git@github.com:adelbertc/frameless.git","clone_url":"https://github.com/adelbertc/frameless.git","svn_url":"https://github.com/adelbertc/frameless","homepage":"","size":334,"stargazers_count":113,"watchers_count":113,"language":"Scala","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":10,"mirror_url":null,"open_issues_count":14,"forks":10,"open_issues":14,"watchers":113,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/adelbertc/frameless/pulls/66"},"html":{"href":"https://github.com/adelbertc/frameless/pull/66"},"issue":{"href":"https://api.github.com/repos/adelbertc/frameless/issues/66"},"comments":{"href":"https://api.github.com/repos/adelbertc/frameless/issues/66/comments"},"review_comments":{"href":"https://api.github.com/repos/adelbertc/frameless/pulls/66/comments"},"review_comment":{"href":"https://api.github.com/repos/adelbertc/frameless/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/adelbertc/frameless/pulls/66/commits"},"statuses":{"href":"https://api.github.com/repos/adelbertc/frameless/statuses/fe5bde0ec71007db48c01a8dbbf98d9d3096f4b1"}}}}
{"actor":{"display_login":"imarios"}}