title: JDBC feeder for Elasticsearch
body: <p>I am trying to create a JBDC feeder to load data from SQL Server into elasticsearch. I am using the guide here: <a href="https://github.com/jprante/elasticsearch-river-jdbc" rel="nofollow">https://github.com/jprante/elasticsearch-river-jdbc</a> (search for heading 'How to run a standalone JDBC feeder'). </p>
  
  <p>I have successfully downloaded and installed elasticsearch and have it up and running. I have downloaded the JDBC driver for SQL server and moved it into the ./plugins/jdbc folder.</p>
  
  <p>I am up to the part that involves creating a bash script. Before today, I have never even looked at a bash script and I'm having trouble getting it to work since I don't yet know half the syntax.</p>
  
  <p>The elasticsearch directory is c:\elasticsearch-1.4.0</p>
  
  <p>and here is my bash script:</p>
  
  <pre><code>#!/bin/sh
  
  DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" &amp;&amp; pwd )"
  
  # ES_HOME required to detect elasticsearch jars
  export ES_HOME= C:\elasticsearch-1.4.0
  
  echo '
  {
      "elasticsearch" : {
           "cluster" : "elasticsearch",
           "host" : "localhost",
           "port" : 9200
      },
      "type" : "jdbc",
      "jdbc" : {
          "url" : "jdbc:sqlserver://localhost;databaseName=MyDatabase",
          "user" : "MyUser",
          "password" : "MyPassword",
          "sql" :  "select * From MyTable",
          "treat_binary_as_string" : true,
          "index" : "MyFirstESIndex"
        }
  }
  ' | java \
      -cp "${DIR}/*" \
      org.xbib.elasticsearch.plugin.jdbc.feeder.Runner \
      org.xbib.elasticsearch.plugin.jdbc.feeder.JDBCFeeder
  </code></pre>
  
  <p>What do I need to update in this script? Is it something in this line of the script:</p>
  
  <pre><code>DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" &amp;&amp; pwd )"
  </code></pre>
  
  <p>The reason I am doing this is because I'm looking for the best method to insert potentially tens of millions of records into elasticsearch from sql server in one go i.e a bulk insert. </p>
  
  <p>Our first iteration of this involved getting each row of data in a table, converting it to a JSON document, and inserting into ES. This took about 10hrs to get all the data in there.</p>
  
  <p>Thanks in advance for any advice.</p>