{"action":"created","issue":{"url":"https://api.github.com/repos/marklogic/java-client-api/issues/889","repository_url":"https://api.github.com/repos/marklogic/java-client-api","labels_url":"https://api.github.com/repos/marklogic/java-client-api/issues/889/labels{/name}","comments_url":"https://api.github.com/repos/marklogic/java-client-api/issues/889/comments","events_url":"https://api.github.com/repos/marklogic/java-client-api/issues/889/events","html_url":"https://github.com/marklogic/java-client-api/issues/889","id":296025983,"number":889,"title":"ExtractViaTemplateListener throws MarkLogicIOException when multiple templates, responses ","user":{"login":"georgeajit","id":8634482,"avatar_url":"https://avatars0.githubusercontent.com/u/8634482?v=4","gravatar_id":"","url":"https://api.github.com/users/georgeajit","html_url":"https://github.com/georgeajit","followers_url":"https://api.github.com/users/georgeajit/followers","following_url":"https://api.github.com/users/georgeajit/following{/other_user}","gists_url":"https://api.github.com/users/georgeajit/gists{/gist_id}","starred_url":"https://api.github.com/users/georgeajit/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/georgeajit/subscriptions","organizations_url":"https://api.github.com/users/georgeajit/orgs","repos_url":"https://api.github.com/users/georgeajit/repos","events_url":"https://api.github.com/users/georgeajit/events{/privacy}","received_events_url":"https://api.github.com/users/georgeajit/received_events","type":"User","site_admin":false},"labels":[{"id":95132628,"url":"https://api.github.com/repos/marklogic/java-client-api/labels/Bug","name":"Bug","color":"e11d21","default":false},{"id":95133197,"url":"https://api.github.com/repos/marklogic/java-client-api/labels/major","name":"major","color":"fbca04","default":false},{"id":95133504,"url":"https://api.github.com/repos/marklogic/java-client-api/labels/verify","name":"verify","color":"eb6420","default":false}],"state":"open","locked":false,"assignee":{"login":"vivekmuniyandi","id":14857031,"avatar_url":"https://avatars1.githubusercontent.com/u/14857031?v=4","gravatar_id":"","url":"https://api.github.com/users/vivekmuniyandi","html_url":"https://github.com/vivekmuniyandi","followers_url":"https://api.github.com/users/vivekmuniyandi/followers","following_url":"https://api.github.com/users/vivekmuniyandi/following{/other_user}","gists_url":"https://api.github.com/users/vivekmuniyandi/gists{/gist_id}","starred_url":"https://api.github.com/users/vivekmuniyandi/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/vivekmuniyandi/subscriptions","organizations_url":"https://api.github.com/users/vivekmuniyandi/orgs","repos_url":"https://api.github.com/users/vivekmuniyandi/repos","events_url":"https://api.github.com/users/vivekmuniyandi/events{/privacy}","received_events_url":"https://api.github.com/users/vivekmuniyandi/received_events","type":"User","site_admin":false},"assignees":[{"login":"vivekmuniyandi","id":14857031,"avatar_url":"https://avatars1.githubusercontent.com/u/14857031?v=4","gravatar_id":"","url":"https://api.github.com/users/vivekmuniyandi","html_url":"https://github.com/vivekmuniyandi","followers_url":"https://api.github.com/users/vivekmuniyandi/followers","following_url":"https://api.github.com/users/vivekmuniyandi/following{/other_user}","gists_url":"https://api.github.com/users/vivekmuniyandi/gists{/gist_id}","starred_url":"https://api.github.com/users/vivekmuniyandi/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/vivekmuniyandi/subscriptions","organizations_url":"https://api.github.com/users/vivekmuniyandi/orgs","repos_url":"https://api.github.com/users/vivekmuniyandi/repos","events_url":"https://api.github.com/users/vivekmuniyandi/events{/privacy}","received_events_url":"https://api.github.com/users/vivekmuniyandi/received_events","type":"User","site_admin":false}],"milestone":{"url":"https://api.github.com/repos/marklogic/java-client-api/milestones/30","html_url":"https://github.com/marklogic/java-client-api/milestone/30","labels_url":"https://api.github.com/repos/marklogic/java-client-api/milestones/30/labels","id":2736913,"number":30,"title":"java-client-api-4.0.4","description":"Functional complete: 02 Feb 2018\r\nTest Complete:16 Feb 2018\r\nControlled check-in:02 Mar 2018\r\nTarget: 30 Mar 2018","creator":{"login":"sammefford","id":298950,"avatar_url":"https://avatars0.githubusercontent.com/u/298950?v=4","gravatar_id":"","url":"https://api.github.com/users/sammefford","html_url":"https://github.com/sammefford","followers_url":"https://api.github.com/users/sammefford/followers","following_url":"https://api.github.com/users/sammefford/following{/other_user}","gists_url":"https://api.github.com/users/sammefford/gists{/gist_id}","starred_url":"https://api.github.com/users/sammefford/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sammefford/subscriptions","organizations_url":"https://api.github.com/users/sammefford/orgs","repos_url":"https://api.github.com/users/sammefford/repos","events_url":"https://api.github.com/users/sammefford/events{/privacy}","received_events_url":"https://api.github.com/users/sammefford/received_events","type":"User","site_admin":false},"open_issues":50,"closed_issues":6,"state":"open","created_at":"2017-08-31T20:00:53Z","updated_at":"2018-02-09T22:02:00Z","due_on":"2018-03-30T07:00:00Z","closed_at":null},"comments":2,"created_at":"2018-02-09T22:02:00Z","updated_at":"2018-02-09T22:07:03Z","closed_at":null,"author_association":"CONTRIBUTOR","body":"A test with templates loaded and multiple data files was run, expecting rows of data to be exported as a Tableau data extract file. But the test throws an exception.\r\n\r\nThe data files and templates used in this test are available in the ```test-complete``` project under the ```data\\optics``` folder. Use Java client API's ```DocumentManager.writeAs(...)``` method to write the template and docs into a database or refer to ```TestOpticOnCtsQuery.java``` for the setup.\r\n\r\n* Version of MarkLogic Java Client API issue839 branch\r\n* Version of MarkLogic Server ML server trunk build (as of 02/09/2018)\r\n* Java version : 1.8\r\n* OS and version : Windows\r\n*  Test code\r\n```\r\n\t@Test\r\n\t  public void testJsonPropertyWordQuery() throws KeyManagementException, NoSuchAlgorithmException, IOException, SAXException, ParserConfigurationException\r\n\t  {\r\n\t    System.out.println(\"In testJsonPropertyWordQuery method\");\r\n\t    DatabaseClient client = null;\r\n\t    DataMovementManager dmManager = null;\r\n\t    \r\n\t    String extractedOutFileName = datasource + \"TableauOutput.tde\";\r\n\t    \r\n\t    // Delete any existing extracted file\r\n\t    if (new File(extractedOutFileName).exists()) new File(extractedOutFileName).delete();\r\n\t    \t    \r\n\t    client = getDatabaseClient(\"admin\", \"admin\", Authentication.DIGEST);\r\n\t    dmManager = client.newDataMovementManager();\r\n\r\n\t    QueryManager queryMgr = client.newQueryManager();\r\n\t    TableauDataExtractWriter tableauWriter = new TableauDataExtractWriter(extractedOutFileName)\r\n\t    \t      .withColumn(\"id\", Type.INTEGER)\r\n\t    \t      .withColumn(\"name\", Type.CHAR_STRING);\r\n\t    \t      \t    \r\n\t    ExtractViaTemplateListener extractor = new ExtractViaTemplateListener();\r\n\t    \r\n\t    extractor.withTemplate(\"masterDetail.tde\")\r\n\t             .withTemplate(\"masterDetail2.tde\")\r\n\t             .withTemplate(\"masterDetail3.tde\")\r\n\t             .withTemplate(\"masterDetail4.tde\")\r\n\t             .withTemplateDatabase(dbName);\r\n\t    \r\n\t    StructuredQueryBuilder qb = queryMgr.newStructuredQueryBuilder();\r\n\t    StringBuilder batchResults = new StringBuilder();\r\n\t    StringBuilder batchDetails = new StringBuilder();\r\n\t    StringBuilder forestDetails = new StringBuilder();\r\n\t    \r\n\t    StructuredQueryDefinition querydef = qb.word(qb.element(\"id\"), \"100\");\r\n\t    QueryBatcher queryBatcher = dmManager.newQueryBatcher(querydef)\r\n\t    \t\t.onUrisReady(extractor\r\n\t    \t\t\t\t.onTypedRowReady( row -> {\r\n\t    \t\t\t\t\ttableauWriter.accept((LinkedHashMap<String, XsAnyAtomicTypeVal>) row);\r\n\t    \t\t\t\t})\r\n\t    \t\t\t\t)\r\n\t    \t\t.onUrisReady(batch -> {\r\n\t    \t\t\tfor (String str : batch.getItems()) {\r\n\t    \t\t\t\tbatchResults.append(str)\r\n\t    \t\t\t\t.append('|');\r\n\t    \t\t\t\t// Batch details\r\n\t    \t\t\t\tbatchDetails.append(batch.getJobBatchNumber())\r\n\t    \t\t\t\t.append('|')\r\n\t    \t\t\t\t.append(batch.getJobResultsSoFar())\r\n\t    \t\t\t\t.append('|')\r\n\t    \t\t\t\t.append(batch.getForestBatchNumber());\r\n\r\n\t    \t\t\t\t// Get the Forest details\r\n\t    \t\t\t\tForest forest = batch.getForest();\r\n\t    \t\t\t\tforestDetails.append(forest.getDatabaseName())\r\n\t    \t\t\t\t.append('|')\r\n\t    \t\t\t\t.append(forest.getHost())\r\n\t    \t\t\t\t.append('|')\r\n\t    \t\t\t\t.append(forest.getForestName());\r\n\t    \t\t\t}\r\n\t    \t\t});\r\n\t    \t\t\r\n\t    dmManager.startJob(queryBatcher);\r\n\t    try {\r\n\t\t\tqueryBatcher.awaitCompletion();\r\n\t\t} catch (Exception e) {\r\n\t\t\t// TODO Auto-generated catch block\r\n\t\t\te.printStackTrace();\r\n\t\t}\r\n\t    \r\n\t    // Verify the results\r\n\t    System.out.println(\"batchResults are \" + batchResults.toString());\r\n\t    System.out.println(\"batchDetails are \" + batchDetails.toString());\r\n\t    //dmManager.stopJob(queryBatcher);\r\n\t    \r\n\t    tableauWriter.close();\r\n\r\n\t    // Create a new Plan.\r\n\t    RowManager rowMgr = client.newRowManager();\r\n\t    PlanBuilder p = rowMgr.newPlanBuilder();\r\n\t    // plan1 - fromView\r\n\t    ModifyPlan plan1 = p.fromView(\"opticFunctionalTest4\", \"detail4\", null, null)\r\n\t        .where(p.cts.jsonPropertyWordQuery(\"name\", \"Detail 100\"));\r\n\t    // plan2 - fromView\r\n\t    ModifyPlan plan2 = p.fromView(\"opticFunctionalTest4\", \"master4\");\r\n\r\n\t    ModifyPlan output = plan1.joinInner(plan2, p.on(p.schemaCol(\"opticFunctionalTest4\", \"detail4\", \"masterId\"), p.schemaCol(\"opticFunctionalTest4\", \"master4\", \"id\")))\r\n\t        .orderBy(p.schemaCol(\"opticFunctionalTest4\", \"detail4\", \"id\"));\r\n\r\n\t    JacksonHandle jacksonHandle = new JacksonHandle();\r\n\t    jacksonHandle.setMimetype(\"application/json\");\r\n\t  }\r\n```\r\n* Actual output: Exception on the console log.\r\n```\r\n10:51:44.768 [pool-3-thread-1] WARN  c.m.c.e.c.d.ExtractViaTemplateListener - Error: [com.marklogic.client.MarkLogicIOException: Expected row to end] in batch with urs ([/optic/view/test/masterDetail4.json])\r\n```\r\n* Expected output: Rows of data\r\n* Alternatives:\r\n"},"comment":{"url":"https://api.github.com/repos/marklogic/java-client-api/issues/comments/364583096","html_url":"https://github.com/marklogic/java-client-api/issues/889#issuecomment-364583096","issue_url":"https://api.github.com/repos/marklogic/java-client-api/issues/889","id":364583096,"user":{"login":"georgeajit","id":8634482,"avatar_url":"https://avatars0.githubusercontent.com/u/8634482?v=4","gravatar_id":"","url":"https://api.github.com/users/georgeajit","html_url":"https://github.com/georgeajit","followers_url":"https://api.github.com/users/georgeajit/followers","following_url":"https://api.github.com/users/georgeajit/following{/other_user}","gists_url":"https://api.github.com/users/georgeajit/gists{/gist_id}","starred_url":"https://api.github.com/users/georgeajit/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/georgeajit/subscriptions","organizations_url":"https://api.github.com/users/georgeajit/orgs","repos_url":"https://api.github.com/users/georgeajit/repos","events_url":"https://api.github.com/users/georgeajit/events{/privacy}","received_events_url":"https://api.github.com/users/georgeajit/received_events","type":"User","site_admin":false},"created_at":"2018-02-09T22:07:03Z","updated_at":"2018-02-09T22:07:03Z","author_association":"CONTRIBUTOR","body":"The above two comments contain the output and the JavaScript code (not pretty!) for reference, if there is a need to run them again in the Query Console."}}
{"actor":{"display_login":"georgeajit"}}