{"action":"created","comment":{"url":"https://api.github.com/repos/basho/spark-riak-connector/pulls/comments/61317356","id":61317356,"diff_hunk":"@@ -533,6 +534,160 @@ conf.set(\"spark.riakts.write.bulk-size\", \"500\")\n conf.set(\"spark.riak.connections.min\", \"50\")\n ```\n \n+## Spark Streaming Example\n+\n+The Spark-Riak Connector can be used with Spark Streaming. To demonstrate this usage, we will work through two small Scala examples. These examples are located in the [examples folder](../examples/src/main/scala/com/basho/riak/spark/examples) of the Spark-Riak Connector repo.\n+\n+These examples require the use of Kafka. Please install Kafka and setup a Kafka broker prior to running this example. We will assume that there is a Kafka broker running at `127.0.0.1:9092` with a topic called `streaming`. Instructions on setting up kafka topics can be found in [this guide](https://kafka.apache.org/documentation.html#quickstart). We also assume Riak TS is installed and there is a Riak TS node running at `127.0.0.1:8087`. You will need to build the connector as well. Please follow the instruction on [building the connector](./building-and-testing-connector.md#build). After setting up, there are two examples to run: a KV bucket example and a TS table example.\n+\n+###Spark Streaming KV Buckets Example\n+\n+Now that we are set up, lets look at the KV bucket example [here](../examples/src/main/scala/com/basho/riak/spark/examples/streaming/StreamingKVExample.scala). \n+\n+In the first chunk of code in the main method, we are just setting up our local spark streaming context and setting the name for the KV bucket to `test-data`:\n+\n+```scala\n+val sparkConf = new SparkConf(true)\n+      .setAppName(\"Simple Spark Streaming to Riak KV Demo\")\n+setSparkOpt(sparkConf, \"spark.master\", \"local\")\n+setSparkOpt(sparkConf, \"spark.riak.connection.host\", \"127.0.0.1:8087\")\n+\n+val sc = new SparkContext(sparkConf)\n+val streamCtx = new StreamingContext(sc, Durations.seconds(15))\n+val sqlContext = new org.apache.spark.sql.SQLContext(sc)\n+import sqlContext.implicits._\n+val namespace = new Namespace(\"test-data\")\n+```\n+\n+Next we are setting up kafka properties:","path":"docs/using-connector.md","position":37,"original_position":37,"commit_id":"1dce75c00d10956c317571cb97d62ad4baff33af","original_commit_id":"1dce75c00d10956c317571cb97d62ad4baff33af","user":{"login":"mdigan","id":175687,"avatar_url":"https://avatars.githubusercontent.com/u/175687?v=3","gravatar_id":"","url":"https://api.github.com/users/mdigan","html_url":"https://github.com/mdigan","followers_url":"https://api.github.com/users/mdigan/followers","following_url":"https://api.github.com/users/mdigan/following{/other_user}","gists_url":"https://api.github.com/users/mdigan/gists{/gist_id}","starred_url":"https://api.github.com/users/mdigan/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mdigan/subscriptions","organizations_url":"https://api.github.com/users/mdigan/orgs","repos_url":"https://api.github.com/users/mdigan/repos","events_url":"https://api.github.com/users/mdigan/events{/privacy}","received_events_url":"https://api.github.com/users/mdigan/received_events","type":"User","site_admin":false},"body":"capitalize \"kafka\"","created_at":"2016-04-27T19:04:05Z","updated_at":"2016-04-27T19:04:05Z","html_url":"https://github.com/basho/spark-riak-connector/pull/147#discussion_r61317356","pull_request_url":"https://api.github.com/repos/basho/spark-riak-connector/pulls/147","_links":{"self":{"href":"https://api.github.com/repos/basho/spark-riak-connector/pulls/comments/61317356"},"html":{"href":"https://github.com/basho/spark-riak-connector/pull/147#discussion_r61317356"},"pull_request":{"href":"https://api.github.com/repos/basho/spark-riak-connector/pulls/147"}}},"pull_request":{"url":"https://api.github.com/repos/basho/spark-riak-connector/pulls/147","id":67937997,"html_url":"https://github.com/basho/spark-riak-connector/pull/147","diff_url":"https://github.com/basho/spark-riak-connector/pull/147.diff","patch_url":"https://github.com/basho/spark-riak-connector/pull/147.patch","issue_url":"https://api.github.com/repos/basho/spark-riak-connector/issues/147","number":147,"state":"open","locked":false,"title":"Kclark docs update","user":{"login":"korry8911","id":10292569,"avatar_url":"https://avatars.githubusercontent.com/u/10292569?v=3","gravatar_id":"","url":"https://api.github.com/users/korry8911","html_url":"https://github.com/korry8911","followers_url":"https://api.github.com/users/korry8911/followers","following_url":"https://api.github.com/users/korry8911/following{/other_user}","gists_url":"https://api.github.com/users/korry8911/gists{/gist_id}","starred_url":"https://api.github.com/users/korry8911/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/korry8911/subscriptions","organizations_url":"https://api.github.com/users/korry8911/orgs","repos_url":"https://api.github.com/users/korry8911/repos","events_url":"https://api.github.com/users/korry8911/events{/privacy}","received_events_url":"https://api.github.com/users/korry8911/received_events","type":"User","site_admin":false},"body":"added spark streaming docs and sc.stop to quick start guides","created_at":"2016-04-26T18:48:06Z","updated_at":"2016-04-27T19:04:05Z","closed_at":null,"merged_at":null,"merge_commit_sha":"d62d1d6d72e9edce9709424c7778c099fc8f93ed","assignee":null,"milestone":null,"commits_url":"https://api.github.com/repos/basho/spark-riak-connector/pulls/147/commits","review_comments_url":"https://api.github.com/repos/basho/spark-riak-connector/pulls/147/comments","review_comment_url":"https://api.github.com/repos/basho/spark-riak-connector/pulls/comments{/number}","comments_url":"https://api.github.com/repos/basho/spark-riak-connector/issues/147/comments","statuses_url":"https://api.github.com/repos/basho/spark-riak-connector/statuses/1dce75c00d10956c317571cb97d62ad4baff33af","head":{"label":"basho:kclark-docs-update","ref":"kclark-docs-update","sha":"1dce75c00d10956c317571cb97d62ad4baff33af","user":{"login":"basho","id":176293,"avatar_url":"https://avatars.githubusercontent.com/u/176293?v=3","gravatar_id":"","url":"https://api.github.com/users/basho","html_url":"https://github.com/basho","followers_url":"https://api.github.com/users/basho/followers","following_url":"https://api.github.com/users/basho/following{/other_user}","gists_url":"https://api.github.com/users/basho/gists{/gist_id}","starred_url":"https://api.github.com/users/basho/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/basho/subscriptions","organizations_url":"https://api.github.com/users/basho/orgs","repos_url":"https://api.github.com/users/basho/repos","events_url":"https://api.github.com/users/basho/events{/privacy}","received_events_url":"https://api.github.com/users/basho/received_events","type":"Organization","site_admin":false},"repo":{"id":35239038,"name":"spark-riak-connector","full_name":"basho/spark-riak-connector","owner":{"login":"basho","id":176293,"avatar_url":"https://avatars.githubusercontent.com/u/176293?v=3","gravatar_id":"","url":"https://api.github.com/users/basho","html_url":"https://github.com/basho","followers_url":"https://api.github.com/users/basho/followers","following_url":"https://api.github.com/users/basho/following{/other_user}","gists_url":"https://api.github.com/users/basho/gists{/gist_id}","starred_url":"https://api.github.com/users/basho/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/basho/subscriptions","organizations_url":"https://api.github.com/users/basho/orgs","repos_url":"https://api.github.com/users/basho/repos","events_url":"https://api.github.com/users/basho/events{/privacy}","received_events_url":"https://api.github.com/users/basho/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/basho/spark-riak-connector","description":"The official Riak Spark Connector for Apache Spark with Riak TS and Riak KV","fork":false,"url":"https://api.github.com/repos/basho/spark-riak-connector","forks_url":"https://api.github.com/repos/basho/spark-riak-connector/forks","keys_url":"https://api.github.com/repos/basho/spark-riak-connector/keys{/key_id}","collaborators_url":"https://api.github.com/repos/basho/spark-riak-connector/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/basho/spark-riak-connector/teams","hooks_url":"https://api.github.com/repos/basho/spark-riak-connector/hooks","issue_events_url":"https://api.github.com/repos/basho/spark-riak-connector/issues/events{/number}","events_url":"https://api.github.com/repos/basho/spark-riak-connector/events","assignees_url":"https://api.github.com/repos/basho/spark-riak-connector/assignees{/user}","branches_url":"https://api.github.com/repos/basho/spark-riak-connector/branches{/branch}","tags_url":"https://api.github.com/repos/basho/spark-riak-connector/tags","blobs_url":"https://api.github.com/repos/basho/spark-riak-connector/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/basho/spark-riak-connector/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/basho/spark-riak-connector/git/refs{/sha}","trees_url":"https://api.github.com/repos/basho/spark-riak-connector/git/trees{/sha}","statuses_url":"https://api.github.com/repos/basho/spark-riak-connector/statuses/{sha}","languages_url":"https://api.github.com/repos/basho/spark-riak-connector/languages","stargazers_url":"https://api.github.com/repos/basho/spark-riak-connector/stargazers","contributors_url":"https://api.github.com/repos/basho/spark-riak-connector/contributors","subscribers_url":"https://api.github.com/repos/basho/spark-riak-connector/subscribers","subscription_url":"https://api.github.com/repos/basho/spark-riak-connector/subscription","commits_url":"https://api.github.com/repos/basho/spark-riak-connector/commits{/sha}","git_commits_url":"https://api.github.com/repos/basho/spark-riak-connector/git/commits{/sha}","comments_url":"https://api.github.com/repos/basho/spark-riak-connector/comments{/number}","issue_comment_url":"https://api.github.com/repos/basho/spark-riak-connector/issues/comments{/number}","contents_url":"https://api.github.com/repos/basho/spark-riak-connector/contents/{+path}","compare_url":"https://api.github.com/repos/basho/spark-riak-connector/compare/{base}...{head}","merges_url":"https://api.github.com/repos/basho/spark-riak-connector/merges","archive_url":"https://api.github.com/repos/basho/spark-riak-connector/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/basho/spark-riak-connector/downloads","issues_url":"https://api.github.com/repos/basho/spark-riak-connector/issues{/number}","pulls_url":"https://api.github.com/repos/basho/spark-riak-connector/pulls{/number}","milestones_url":"https://api.github.com/repos/basho/spark-riak-connector/milestones{/number}","notifications_url":"https://api.github.com/repos/basho/spark-riak-connector/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/basho/spark-riak-connector/labels{/name}","releases_url":"https://api.github.com/repos/basho/spark-riak-connector/releases{/id}","deployments_url":"https://api.github.com/repos/basho/spark-riak-connector/deployments","created_at":"2015-05-07T19:22:35Z","updated_at":"2016-04-27T06:03:19Z","pushed_at":"2016-04-27T18:59:57Z","git_url":"git://github.com/basho/spark-riak-connector.git","ssh_url":"git@github.com:basho/spark-riak-connector.git","clone_url":"https://github.com/basho/spark-riak-connector.git","svn_url":"https://github.com/basho/spark-riak-connector","homepage":"http://basho.com/products/spark/","size":17435,"stargazers_count":18,"watchers_count":18,"language":"Scala","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":6,"mirror_url":null,"open_issues_count":7,"forks":6,"open_issues":7,"watchers":18,"default_branch":"master"}},"base":{"label":"basho:master","ref":"master","sha":"f5e4c423a886ee0a62d821a3e1d10eb7e15c051f","user":{"login":"basho","id":176293,"avatar_url":"https://avatars.githubusercontent.com/u/176293?v=3","gravatar_id":"","url":"https://api.github.com/users/basho","html_url":"https://github.com/basho","followers_url":"https://api.github.com/users/basho/followers","following_url":"https://api.github.com/users/basho/following{/other_user}","gists_url":"https://api.github.com/users/basho/gists{/gist_id}","starred_url":"https://api.github.com/users/basho/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/basho/subscriptions","organizations_url":"https://api.github.com/users/basho/orgs","repos_url":"https://api.github.com/users/basho/repos","events_url":"https://api.github.com/users/basho/events{/privacy}","received_events_url":"https://api.github.com/users/basho/received_events","type":"Organization","site_admin":false},"repo":{"id":35239038,"name":"spark-riak-connector","full_name":"basho/spark-riak-connector","owner":{"login":"basho","id":176293,"avatar_url":"https://avatars.githubusercontent.com/u/176293?v=3","gravatar_id":"","url":"https://api.github.com/users/basho","html_url":"https://github.com/basho","followers_url":"https://api.github.com/users/basho/followers","following_url":"https://api.github.com/users/basho/following{/other_user}","gists_url":"https://api.github.com/users/basho/gists{/gist_id}","starred_url":"https://api.github.com/users/basho/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/basho/subscriptions","organizations_url":"https://api.github.com/users/basho/orgs","repos_url":"https://api.github.com/users/basho/repos","events_url":"https://api.github.com/users/basho/events{/privacy}","received_events_url":"https://api.github.com/users/basho/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/basho/spark-riak-connector","description":"The official Riak Spark Connector for Apache Spark with Riak TS and Riak KV","fork":false,"url":"https://api.github.com/repos/basho/spark-riak-connector","forks_url":"https://api.github.com/repos/basho/spark-riak-connector/forks","keys_url":"https://api.github.com/repos/basho/spark-riak-connector/keys{/key_id}","collaborators_url":"https://api.github.com/repos/basho/spark-riak-connector/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/basho/spark-riak-connector/teams","hooks_url":"https://api.github.com/repos/basho/spark-riak-connector/hooks","issue_events_url":"https://api.github.com/repos/basho/spark-riak-connector/issues/events{/number}","events_url":"https://api.github.com/repos/basho/spark-riak-connector/events","assignees_url":"https://api.github.com/repos/basho/spark-riak-connector/assignees{/user}","branches_url":"https://api.github.com/repos/basho/spark-riak-connector/branches{/branch}","tags_url":"https://api.github.com/repos/basho/spark-riak-connector/tags","blobs_url":"https://api.github.com/repos/basho/spark-riak-connector/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/basho/spark-riak-connector/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/basho/spark-riak-connector/git/refs{/sha}","trees_url":"https://api.github.com/repos/basho/spark-riak-connector/git/trees{/sha}","statuses_url":"https://api.github.com/repos/basho/spark-riak-connector/statuses/{sha}","languages_url":"https://api.github.com/repos/basho/spark-riak-connector/languages","stargazers_url":"https://api.github.com/repos/basho/spark-riak-connector/stargazers","contributors_url":"https://api.github.com/repos/basho/spark-riak-connector/contributors","subscribers_url":"https://api.github.com/repos/basho/spark-riak-connector/subscribers","subscription_url":"https://api.github.com/repos/basho/spark-riak-connector/subscription","commits_url":"https://api.github.com/repos/basho/spark-riak-connector/commits{/sha}","git_commits_url":"https://api.github.com/repos/basho/spark-riak-connector/git/commits{/sha}","comments_url":"https://api.github.com/repos/basho/spark-riak-connector/comments{/number}","issue_comment_url":"https://api.github.com/repos/basho/spark-riak-connector/issues/comments{/number}","contents_url":"https://api.github.com/repos/basho/spark-riak-connector/contents/{+path}","compare_url":"https://api.github.com/repos/basho/spark-riak-connector/compare/{base}...{head}","merges_url":"https://api.github.com/repos/basho/spark-riak-connector/merges","archive_url":"https://api.github.com/repos/basho/spark-riak-connector/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/basho/spark-riak-connector/downloads","issues_url":"https://api.github.com/repos/basho/spark-riak-connector/issues{/number}","pulls_url":"https://api.github.com/repos/basho/spark-riak-connector/pulls{/number}","milestones_url":"https://api.github.com/repos/basho/spark-riak-connector/milestones{/number}","notifications_url":"https://api.github.com/repos/basho/spark-riak-connector/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/basho/spark-riak-connector/labels{/name}","releases_url":"https://api.github.com/repos/basho/spark-riak-connector/releases{/id}","deployments_url":"https://api.github.com/repos/basho/spark-riak-connector/deployments","created_at":"2015-05-07T19:22:35Z","updated_at":"2016-04-27T06:03:19Z","pushed_at":"2016-04-27T18:59:57Z","git_url":"git://github.com/basho/spark-riak-connector.git","ssh_url":"git@github.com:basho/spark-riak-connector.git","clone_url":"https://github.com/basho/spark-riak-connector.git","svn_url":"https://github.com/basho/spark-riak-connector","homepage":"http://basho.com/products/spark/","size":17435,"stargazers_count":18,"watchers_count":18,"language":"Scala","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":6,"mirror_url":null,"open_issues_count":7,"forks":6,"open_issues":7,"watchers":18,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/basho/spark-riak-connector/pulls/147"},"html":{"href":"https://github.com/basho/spark-riak-connector/pull/147"},"issue":{"href":"https://api.github.com/repos/basho/spark-riak-connector/issues/147"},"comments":{"href":"https://api.github.com/repos/basho/spark-riak-connector/issues/147/comments"},"review_comments":{"href":"https://api.github.com/repos/basho/spark-riak-connector/pulls/147/comments"},"review_comment":{"href":"https://api.github.com/repos/basho/spark-riak-connector/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/basho/spark-riak-connector/pulls/147/commits"},"statuses":{"href":"https://api.github.com/repos/basho/spark-riak-connector/statuses/1dce75c00d10956c317571cb97d62ad4baff33af"}}}}