{"action":"created","comment":{"url":"https://api.github.com/repos/apache/incubator-fluo-recipes/pulls/comments/79850487","id":79850487,"diff_hunk":"@@ -16,60 +16,58 @@\n package org.apache.fluo.recipes.accumulo.export;\n \n import java.util.ArrayList;\n+import java.util.Collection;\n import java.util.Iterator;\n \n import org.apache.accumulo.core.data.Mutation;\n import org.apache.fluo.api.config.FluoConfiguration;\n-import org.apache.fluo.api.config.SimpleConfiguration;\n import org.apache.fluo.api.observer.Observer.Context;\n+import org.apache.fluo.recipes.core.export.ExportQueue;\n import org.apache.fluo.recipes.core.export.Exporter;\n import org.apache.fluo.recipes.core.export.SequencedExport;\n \n /**\n- * An {@link Exporter} that takes {@link AccumuloExport} objects and writes mutations to Accumulo\n+ * An Accumulo-specific {@link Exporter} that writes mutations to Accumulo using a\n+ * {@link AccumuloWriter}\n  *\n- * @param <K> Export queue key type\n  * @since 1.0.0\n  */\n-public class AccumuloExporter<K> extends Exporter<K, AccumuloExport<K>> {\n+public abstract class AccumuloExporter<K, V> extends Exporter<K, V> {\n \n-  private SharedBatchWriter sbw;\n+  private AccumuloWriter accumuloWriter;\n \n-  @Override\n-  public void init(String queueId, Context context) throws Exception {\n-\n-    SimpleConfiguration appConf = context.getAppConfiguration();\n-\n-    String instanceName = appConf.getString(\"recipes.accumuloExporter.\" + queueId + \".instance\");\n-    String zookeepers = appConf.getString(\"recipes.accumuloExporter.\" + queueId + \".zookeepers\");\n-    String user = appConf.getString(\"recipes.accumuloExporter.\" + queueId + \".user\");\n-    // TODO look into using delegation token\n-    String password = appConf.getString(\"recipes.accumuloExporter.\" + queueId + \".password\");\n-    String table = appConf.getString(\"recipes.accumuloExporter.\" + queueId + \".table\");\n-\n-    sbw = SharedBatchWriter.getInstance(instanceName, zookeepers, user, password, table);\n+  /**\n+   * Configures AccumuloExporter\n+   *\n+   * @param config Fluo configuration\n+   * @param opts Export queue options\n+   * @param writerConf Accumulo writer config\n+   */\n+  public static void configure(FluoConfiguration config, ExportQueue.Options opts,\n+      AccumuloWriter.Config writerConf) {\n+    ExportQueue.configure(config, opts);\n+    AccumuloWriter.setConfig(config.getAppConfiguration(), opts.getQueueId(), writerConf);\n   }\n \n-  public static void setExportTableInfo(FluoConfiguration fconf, String queueId, TableInfo ti) {\n-    SimpleConfiguration appConf = fconf.getAppConfiguration();\n-    appConf.setProperty(\"recipes.accumuloExporter.\" + queueId + \".instance\", ti.instanceName);\n-    appConf.setProperty(\"recipes.accumuloExporter.\" + queueId + \".zookeepers\", ti.zookeepers);\n-    appConf.setProperty(\"recipes.accumuloExporter.\" + queueId + \".user\", ti.user);\n-    appConf.setProperty(\"recipes.accumuloExporter.\" + queueId + \".password\", ti.password);\n-    appConf.setProperty(\"recipes.accumuloExporter.\" + queueId + \".table\", ti.table);\n+  @Override\n+  public void init(String queueId, Context context) throws Exception {\n+    accumuloWriter = AccumuloWriter.getInstance(context.getAppConfiguration(), queueId);\n   }\n \n   @Override\n-  protected void processExports(Iterator<SequencedExport<K, AccumuloExport<K>>> exports) {\n+  protected void processExports(Iterator<SequencedExport<K, V>> exports) {\n+\n     ArrayList<Mutation> buffer = new ArrayList<>();\n \n     while (exports.hasNext()) {\n-      SequencedExport<K, AccumuloExport<K>> export = exports.next();\n-      buffer.addAll(export.getValue().toMutations(export.getKey(), export.getSequence()));\n+      SequencedExport<K, V> export = exports.next();\n+      buffer.addAll(processExport(export));\n     }\n \n     if (buffer.size() > 0) {\n-      sbw.write(buffer);\n+      accumuloWriter.write(buffer);\n     }\n   }\n+\n+  protected abstract Collection<Mutation> processExport(SequencedExport<K, V> export);","path":"modules/accumulo/src/main/java/org/apache/fluo/recipes/accumulo/export/AccumuloExporter.java","position":73,"original_position":86,"commit_id":"c5cee2fa1b402001a432cb6476f59676a0fbb982","original_commit_id":"a1f039d17af202e9660c598265c8f27ffb130852","user":{"login":"mikewalch","id":730698,"avatar_url":"https://avatars.githubusercontent.com/u/730698?v=3","gravatar_id":"","url":"https://api.github.com/users/mikewalch","html_url":"https://github.com/mikewalch","followers_url":"https://api.github.com/users/mikewalch/followers","following_url":"https://api.github.com/users/mikewalch/following{/other_user}","gists_url":"https://api.github.com/users/mikewalch/gists{/gist_id}","starred_url":"https://api.github.com/users/mikewalch/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mikewalch/subscriptions","organizations_url":"https://api.github.com/users/mikewalch/orgs","repos_url":"https://api.github.com/users/mikewalch/repos","events_url":"https://api.github.com/users/mikewalch/events{/privacy}","received_events_url":"https://api.github.com/users/mikewalch/received_events","type":"User","site_admin":false},"body":"I thought about this and thought that we should just extend AccumuloExporter to do this.  I wrote some of the code below but I don't know if it makes sense or makes anything simpler. \r\n```java\r\npublic abstract class AccumuloTranslationExporter<K> extends AccumuloExporter<K, Object> {\r\n\r\n  public interface MutationTranslator<K, V> {\r\n    Collection<Mutation> translate(SequencedExport<K, V> export);\r\n  }\r\n\r\n  public abstract void setup();\r\n\r\n  public void addTranslator(Class clazz, MutationTranslator mt) {\r\n    translatorMap.put(clazz, mt);\r\n  }\r\n\r\n  private Map<Class, MutationTranslator> translatorMap = new HashMap<>();\r\n\r\n  @Override\r\n  protected Collection<Mutation> processExport(SequencedExport<K, Object> export) {\r\n    for (Map.Entry<Class, MutationTranslator> entry : translatorMap.entrySet()) {\r\n      if (entry.getKey().isInstance(export)) {\r\n        return entry.getValue().translate(export);\r\n      }\r\n    }\r\n    return Collections.emptyList();\r\n  }\r\n}\r\n```","created_at":"2016-09-21T14:44:20Z","updated_at":"2016-09-21T14:44:20Z","html_url":"https://github.com/apache/incubator-fluo-recipes/pull/103#discussion_r79850487","pull_request_url":"https://api.github.com/repos/apache/incubator-fluo-recipes/pulls/103","_links":{"self":{"href":"https://api.github.com/repos/apache/incubator-fluo-recipes/pulls/comments/79850487"},"html":{"href":"https://github.com/apache/incubator-fluo-recipes/pull/103#discussion_r79850487"},"pull_request":{"href":"https://api.github.com/repos/apache/incubator-fluo-recipes/pulls/103"}}},"pull_request":{"url":"https://api.github.com/repos/apache/incubator-fluo-recipes/pulls/103","id":84708223,"html_url":"https://github.com/apache/incubator-fluo-recipes/pull/103","diff_url":"https://github.com/apache/incubator-fluo-recipes/pull/103.diff","patch_url":"https://github.com/apache/incubator-fluo-recipes/pull/103.patch","issue_url":"https://api.github.com/repos/apache/incubator-fluo-recipes/issues/103","number":103,"state":"open","locked":false,"title":"Fixes #102 - Simplify Accumulo export queue recipe","user":{"login":"mikewalch","id":730698,"avatar_url":"https://avatars.githubusercontent.com/u/730698?v=3","gravatar_id":"","url":"https://api.github.com/users/mikewalch","html_url":"https://github.com/mikewalch","followers_url":"https://api.github.com/users/mikewalch/followers","following_url":"https://api.github.com/users/mikewalch/following{/other_user}","gists_url":"https://api.github.com/users/mikewalch/gists{/gist_id}","starred_url":"https://api.github.com/users/mikewalch/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mikewalch/subscriptions","organizations_url":"https://api.github.com/users/mikewalch/orgs","repos_url":"https://api.github.com/users/mikewalch/repos","events_url":"https://api.github.com/users/mikewalch/events{/privacy}","received_events_url":"https://api.github.com/users/mikewalch/received_events","type":"User","site_admin":false},"body":"* AccumuloExporter is now abstract class that is implemented by user\r\n  and handles writing mutation given by user to Accumulo.\r\n* AccumuloExport objects are no longer placed on queue\r\n* Renamed SharedBatchWriter to AccumuloWriter\r\n* Code that help users create mutations was moved to AccumuloWriter","created_at":"2016-09-09T19:50:00Z","updated_at":"2016-09-21T14:44:20Z","closed_at":null,"merged_at":null,"merge_commit_sha":"a477d496c764065d5dd7c8774887612d15070dd5","assignee":null,"assignees":[],"milestone":null,"commits_url":"https://api.github.com/repos/apache/incubator-fluo-recipes/pulls/103/commits","review_comments_url":"https://api.github.com/repos/apache/incubator-fluo-recipes/pulls/103/comments","review_comment_url":"https://api.github.com/repos/apache/incubator-fluo-recipes/pulls/comments{/number}","comments_url":"https://api.github.com/repos/apache/incubator-fluo-recipes/issues/103/comments","statuses_url":"https://api.github.com/repos/apache/incubator-fluo-recipes/statuses/c5cee2fa1b402001a432cb6476f59676a0fbb982","head":{"label":"mikewalch:export-rewrite","ref":"export-rewrite","sha":"c5cee2fa1b402001a432cb6476f59676a0fbb982","user":{"login":"mikewalch","id":730698,"avatar_url":"https://avatars.githubusercontent.com/u/730698?v=3","gravatar_id":"","url":"https://api.github.com/users/mikewalch","html_url":"https://github.com/mikewalch","followers_url":"https://api.github.com/users/mikewalch/followers","following_url":"https://api.github.com/users/mikewalch/following{/other_user}","gists_url":"https://api.github.com/users/mikewalch/gists{/gist_id}","starred_url":"https://api.github.com/users/mikewalch/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mikewalch/subscriptions","organizations_url":"https://api.github.com/users/mikewalch/orgs","repos_url":"https://api.github.com/users/mikewalch/repos","events_url":"https://api.github.com/users/mikewalch/events{/privacy}","received_events_url":"https://api.github.com/users/mikewalch/received_events","type":"User","site_admin":false},"repo":{"id":42184370,"name":"incubator-fluo-recipes","full_name":"mikewalch/incubator-fluo-recipes","owner":{"login":"mikewalch","id":730698,"avatar_url":"https://avatars.githubusercontent.com/u/730698?v=3","gravatar_id":"","url":"https://api.github.com/users/mikewalch","html_url":"https://github.com/mikewalch","followers_url":"https://api.github.com/users/mikewalch/followers","following_url":"https://api.github.com/users/mikewalch/following{/other_user}","gists_url":"https://api.github.com/users/mikewalch/gists{/gist_id}","starred_url":"https://api.github.com/users/mikewalch/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mikewalch/subscriptions","organizations_url":"https://api.github.com/users/mikewalch/orgs","repos_url":"https://api.github.com/users/mikewalch/repos","events_url":"https://api.github.com/users/mikewalch/events{/privacy}","received_events_url":"https://api.github.com/users/mikewalch/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/mikewalch/incubator-fluo-recipes","description":"Implementation of common Fluo patterns","fork":true,"url":"https://api.github.com/repos/mikewalch/incubator-fluo-recipes","forks_url":"https://api.github.com/repos/mikewalch/incubator-fluo-recipes/forks","keys_url":"https://api.github.com/repos/mikewalch/incubator-fluo-recipes/keys{/key_id}","collaborators_url":"https://api.github.com/repos/mikewalch/incubator-fluo-recipes/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/mikewalch/incubator-fluo-recipes/teams","hooks_url":"https://api.github.com/repos/mikewalch/incubator-fluo-recipes/hooks","issue_events_url":"https://api.github.com/repos/mikewalch/incubator-fluo-recipes/issues/events{/number}","events_url":"https://api.github.com/repos/mikewalch/incubator-fluo-recipes/events","assignees_url":"https://api.github.com/repos/mikewalch/incubator-fluo-recipes/assignees{/user}","branches_url":"https://api.github.com/repos/mikewalch/incubator-fluo-recipes/branches{/branch}","tags_url":"https://api.github.com/repos/mikewalch/incubator-fluo-recipes/tags","blobs_url":"https://api.github.com/repos/mikewalch/incubator-fluo-recipes/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/mikewalch/incubator-fluo-recipes/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/mikewalch/incubator-fluo-recipes/git/refs{/sha}","trees_url":"https://api.github.com/repos/mikewalch/incubator-fluo-recipes/git/trees{/sha}","statuses_url":"https://api.github.com/repos/mikewalch/incubator-fluo-recipes/statuses/{sha}","languages_url":"https://api.github.com/repos/mikewalch/incubator-fluo-recipes/languages","stargazers_url":"https://api.github.com/repos/mikewalch/incubator-fluo-recipes/stargazers","contributors_url":"https://api.github.com/repos/mikewalch/incubator-fluo-recipes/contributors","subscribers_url":"https://api.github.com/repos/mikewalch/incubator-fluo-recipes/subscribers","subscription_url":"https://api.github.com/repos/mikewalch/incubator-fluo-recipes/subscription","commits_url":"https://api.github.com/repos/mikewalch/incubator-fluo-recipes/commits{/sha}","git_commits_url":"https://api.github.com/repos/mikewalch/incubator-fluo-recipes/git/commits{/sha}","comments_url":"https://api.github.com/repos/mikewalch/incubator-fluo-recipes/comments{/number}","issue_comment_url":"https://api.github.com/repos/mikewalch/incubator-fluo-recipes/issues/comments{/number}","contents_url":"https://api.github.com/repos/mikewalch/incubator-fluo-recipes/contents/{+path}","compare_url":"https://api.github.com/repos/mikewalch/incubator-fluo-recipes/compare/{base}...{head}","merges_url":"https://api.github.com/repos/mikewalch/incubator-fluo-recipes/merges","archive_url":"https://api.github.com/repos/mikewalch/incubator-fluo-recipes/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/mikewalch/incubator-fluo-recipes/downloads","issues_url":"https://api.github.com/repos/mikewalch/incubator-fluo-recipes/issues{/number}","pulls_url":"https://api.github.com/repos/mikewalch/incubator-fluo-recipes/pulls{/number}","milestones_url":"https://api.github.com/repos/mikewalch/incubator-fluo-recipes/milestones{/number}","notifications_url":"https://api.github.com/repos/mikewalch/incubator-fluo-recipes/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/mikewalch/incubator-fluo-recipes/labels{/name}","releases_url":"https://api.github.com/repos/mikewalch/incubator-fluo-recipes/releases{/id}","deployments_url":"https://api.github.com/repos/mikewalch/incubator-fluo-recipes/deployments","created_at":"2015-09-09T14:40:18Z","updated_at":"2016-06-17T17:35:24Z","pushed_at":"2016-09-20T22:04:15Z","git_url":"git://github.com/mikewalch/incubator-fluo-recipes.git","ssh_url":"git@github.com:mikewalch/incubator-fluo-recipes.git","clone_url":"https://github.com/mikewalch/incubator-fluo-recipes.git","svn_url":"https://github.com/mikewalch/incubator-fluo-recipes","homepage":"http://fluo.io","size":510,"stargazers_count":0,"watchers_count":0,"language":"Java","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"apache:master","ref":"master","sha":"d7347ee15555abcc01881d04af1b375eca4018a3","user":{"login":"apache","id":47359,"avatar_url":"https://avatars.githubusercontent.com/u/47359?v=3","gravatar_id":"","url":"https://api.github.com/users/apache","html_url":"https://github.com/apache","followers_url":"https://api.github.com/users/apache/followers","following_url":"https://api.github.com/users/apache/following{/other_user}","gists_url":"https://api.github.com/users/apache/gists{/gist_id}","starred_url":"https://api.github.com/users/apache/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/apache/subscriptions","organizations_url":"https://api.github.com/users/apache/orgs","repos_url":"https://api.github.com/users/apache/repos","events_url":"https://api.github.com/users/apache/events{/privacy}","received_events_url":"https://api.github.com/users/apache/received_events","type":"Organization","site_admin":false},"repo":{"id":41892061,"name":"incubator-fluo-recipes","full_name":"apache/incubator-fluo-recipes","owner":{"login":"apache","id":47359,"avatar_url":"https://avatars.githubusercontent.com/u/47359?v=3","gravatar_id":"","url":"https://api.github.com/users/apache","html_url":"https://github.com/apache","followers_url":"https://api.github.com/users/apache/followers","following_url":"https://api.github.com/users/apache/following{/other_user}","gists_url":"https://api.github.com/users/apache/gists{/gist_id}","starred_url":"https://api.github.com/users/apache/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/apache/subscriptions","organizations_url":"https://api.github.com/users/apache/orgs","repos_url":"https://api.github.com/users/apache/repos","events_url":"https://api.github.com/users/apache/events{/privacy}","received_events_url":"https://api.github.com/users/apache/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/apache/incubator-fluo-recipes","description":"Apache Fluo Recipes (Incubating)","fork":false,"url":"https://api.github.com/repos/apache/incubator-fluo-recipes","forks_url":"https://api.github.com/repos/apache/incubator-fluo-recipes/forks","keys_url":"https://api.github.com/repos/apache/incubator-fluo-recipes/keys{/key_id}","collaborators_url":"https://api.github.com/repos/apache/incubator-fluo-recipes/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/apache/incubator-fluo-recipes/teams","hooks_url":"https://api.github.com/repos/apache/incubator-fluo-recipes/hooks","issue_events_url":"https://api.github.com/repos/apache/incubator-fluo-recipes/issues/events{/number}","events_url":"https://api.github.com/repos/apache/incubator-fluo-recipes/events","assignees_url":"https://api.github.com/repos/apache/incubator-fluo-recipes/assignees{/user}","branches_url":"https://api.github.com/repos/apache/incubator-fluo-recipes/branches{/branch}","tags_url":"https://api.github.com/repos/apache/incubator-fluo-recipes/tags","blobs_url":"https://api.github.com/repos/apache/incubator-fluo-recipes/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/apache/incubator-fluo-recipes/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/apache/incubator-fluo-recipes/git/refs{/sha}","trees_url":"https://api.github.com/repos/apache/incubator-fluo-recipes/git/trees{/sha}","statuses_url":"https://api.github.com/repos/apache/incubator-fluo-recipes/statuses/{sha}","languages_url":"https://api.github.com/repos/apache/incubator-fluo-recipes/languages","stargazers_url":"https://api.github.com/repos/apache/incubator-fluo-recipes/stargazers","contributors_url":"https://api.github.com/repos/apache/incubator-fluo-recipes/contributors","subscribers_url":"https://api.github.com/repos/apache/incubator-fluo-recipes/subscribers","subscription_url":"https://api.github.com/repos/apache/incubator-fluo-recipes/subscription","commits_url":"https://api.github.com/repos/apache/incubator-fluo-recipes/commits{/sha}","git_commits_url":"https://api.github.com/repos/apache/incubator-fluo-recipes/git/commits{/sha}","comments_url":"https://api.github.com/repos/apache/incubator-fluo-recipes/comments{/number}","issue_comment_url":"https://api.github.com/repos/apache/incubator-fluo-recipes/issues/comments{/number}","contents_url":"https://api.github.com/repos/apache/incubator-fluo-recipes/contents/{+path}","compare_url":"https://api.github.com/repos/apache/incubator-fluo-recipes/compare/{base}...{head}","merges_url":"https://api.github.com/repos/apache/incubator-fluo-recipes/merges","archive_url":"https://api.github.com/repos/apache/incubator-fluo-recipes/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/apache/incubator-fluo-recipes/downloads","issues_url":"https://api.github.com/repos/apache/incubator-fluo-recipes/issues{/number}","pulls_url":"https://api.github.com/repos/apache/incubator-fluo-recipes/pulls{/number}","milestones_url":"https://api.github.com/repos/apache/incubator-fluo-recipes/milestones{/number}","notifications_url":"https://api.github.com/repos/apache/incubator-fluo-recipes/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/apache/incubator-fluo-recipes/labels{/name}","releases_url":"https://api.github.com/repos/apache/incubator-fluo-recipes/releases{/id}","deployments_url":"https://api.github.com/repos/apache/incubator-fluo-recipes/deployments","created_at":"2015-09-04T01:24:21Z","updated_at":"2016-07-07T15:32:02Z","pushed_at":"2016-09-20T22:04:16Z","git_url":"git://github.com/apache/incubator-fluo-recipes.git","ssh_url":"git@github.com:apache/incubator-fluo-recipes.git","clone_url":"https://github.com/apache/incubator-fluo-recipes.git","svn_url":"https://github.com/apache/incubator-fluo-recipes","homepage":"https://fluo.apache.org","size":502,"stargazers_count":3,"watchers_count":3,"language":"Java","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":4,"mirror_url":null,"open_issues_count":13,"forks":4,"open_issues":13,"watchers":3,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/apache/incubator-fluo-recipes/pulls/103"},"html":{"href":"https://github.com/apache/incubator-fluo-recipes/pull/103"},"issue":{"href":"https://api.github.com/repos/apache/incubator-fluo-recipes/issues/103"},"comments":{"href":"https://api.github.com/repos/apache/incubator-fluo-recipes/issues/103/comments"},"review_comments":{"href":"https://api.github.com/repos/apache/incubator-fluo-recipes/pulls/103/comments"},"review_comment":{"href":"https://api.github.com/repos/apache/incubator-fluo-recipes/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/apache/incubator-fluo-recipes/pulls/103/commits"},"statuses":{"href":"https://api.github.com/repos/apache/incubator-fluo-recipes/statuses/c5cee2fa1b402001a432cb6476f59676a0fbb982"}}}}
{"actor":{"display_login":"mikewalch"}}