title: Indexing XML files in Solr with DataImportHandler
body: <p>I have to index some XML files. The structure of the files is <a href="http://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=pubmed&amp;id=24069587,1568,4069587,2598965,18000000,254875,587895&amp;rettype=fasta&amp;retmode=xml" rel="nofollow">http://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=pubmed&amp;id=24069587,1568,4069587,2598965,18000000,254875,587895&amp;rettype=fasta&amp;retmode=xml</a></p>
  
  <p>I've edited the Solr config files:</p>
  
  <p><strong>data-config.xml</strong></p>
  
  <pre><code>&lt;dataConfig&gt;
      &lt;dataSource type="FileDataSource" encoding="UTF-8" /&gt;
      &lt;document&gt;
      &lt;entity name="PubmedArticle"
          processor="XPathEntityProcessor"
          stream="true"
          pk="pmc"
          forEach="/pmc-articleset/article"
          url="C:\Users\hp\Desktop\idpSOlr - Copy\apache-solr-4.0.0\example\solr\collection1\conf\pmcsampleDownloaded.xml" 
          transformer="RegexTransformer,DateFormatTransformer"
          &gt;
          &lt;field column="journal-title" xpath="/pmc-articleset/article/front/journal-meta/journal-title-group/journal-title"/&gt;
          &lt;filed column="pmc"  xpath="/pmc-articleset/article/front/article-meta/article-id[@pub-id-type='pmc']"/&gt;
          &lt;filed column="pmid"  xpath="/pmc-articleset/article/front/article-meta/article-id[@pub-id-type='pmid']"/&gt;
          &lt;filed column="other"  xpath="/pmc-articleset/article/front/article-meta/article-id[@pub-id-type='other']"/&gt;
          &lt;filed column="doi"  xpath="/pmc-articleset/article/front/article-meta//article-id[@pub-id-type='doi']"/&gt;
          &lt;field column="text" xpath="/pmc-articleset/article/front/article-meta/tittle-group/article-title" /&gt;
  
  
  
      &lt;/entity&gt;
  &lt;/document&gt;
  </code></pre>
  
  <p></p>
  
  <p>and
  <strong>schema.xml</strong></p>
  
  <pre><code>&lt;schema name="pubmed" version="1.1"&gt;
  &lt;types&gt;
      &lt;fieldType name="nametext" class="solr.TextField" sortMissingLast="true" omitNorms="true"/&gt;         
      &lt;fieldType name="integer" class="solr.IntField" omitNorms="true"/&gt;
      &lt;fieldType name="long" class="solr.LongField" omitNorms="true"/&gt;
      &lt;fieldType name="float" class="solr.FloatField" omitNorms="true"/&gt;
      &lt;fieldType name="double" class="solr.DoubleField" omitNorms="true"/&gt;
      &lt;fieldType name="sint" class="solr.SortableIntField" sortMissingLast="true" omitNorms="true"/&gt;
      &lt;fieldType name="slong" class="solr.SortableLongField" sortMissingLast="true" omitNorms="true"/&gt;
      &lt;fieldType name="sfloat" class="solr.SortableFloatField" sortMissingLast="true" omitNorms="true"/&gt;
      &lt;fieldType name="sdouble" class="solr.SortableDoubleField" sortMissingLast="true" omitNorms="true"/&gt;
      &lt;fieldType name="date" class="solr.DateField" sortMissingLast="true" omitNorms="true"/&gt;
      &lt;fieldType name="random" class="solr.RandomSortField" indexed="true" /&gt; 
       &lt;!-- A text field that only splits on whitespace for exact matching of words --&gt;
      &lt;fieldType name="text_ws" class="solr.TextField" positionIncrementGap="100"&gt;
    &lt;analyzer&gt;
      &lt;tokenizer class="solr.WhitespaceTokenizerFactory"/&gt;
    &lt;/analyzer&gt;
  &lt;/fieldType&gt;
      &lt;fieldType name="text" class="solr.TextField" positionIncrementGap="100"&gt;
          &lt;analyzer type="index"&gt;
              &lt;tokenizer class="solr.WhitespaceTokenizerFactory"/&gt;
              &lt;filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt"/&gt;
              &lt;filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="1" catenateNumbers="1" catenateAll="0" splitOnCaseChange="1"/&gt;
              &lt;filter class="solr.LowerCaseFilterFactory"/&gt;
              &lt;filter class="solr.KeywordMarkerFilterFactory" protected="protwords.txt"/&gt;
              &lt;filter class="solr.PorterStemFilterFactory"/&gt;
              &lt;filter class="solr.RemoveDuplicatesTokenFilterFactory"/&gt;
          &lt;/analyzer&gt;
          &lt;analyzer type="query"&gt;
              &lt;tokenizer class="solr.WhitespaceTokenizerFactory"/&gt;
              &lt;filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="true"/&gt;
              &lt;filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt"/&gt;
              &lt;filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="0" catenateNumbers="0" catenateAll="0" splitOnCaseChange="1"/&gt;
              &lt;filter class="solr.LowerCaseFilterFactory"/&gt;
              &lt;filter class="solr.KeywordMarkerFilterFactory" protected="protwords.txt"/&gt;
              &lt;filter class="solr.PorterStemFilterFactory"/&gt;
              &lt;filter class="solr.RemoveDuplicatesTokenFilterFactory"/&gt;
          &lt;/analyzer&gt;
      &lt;/fieldType&gt;
      &lt;fieldType name="text_general" class="solr.TextField" positionIncrementGap="100"&gt;
          &lt;analyzer type="index"&gt;
              &lt;tokenizer class="solr.StandardTokenizerFactory"/&gt;
              &lt;filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt" enablePositionIncrements="true" /&gt;
              &lt;filter class="solr.LowerCaseFilterFactory"/&gt;
          &lt;/analyzer&gt;
          &lt;analyzer type="query"&gt;
              &lt;tokenizer class="solr.StandardTokenizerFactory"/&gt;
              &lt;filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt" enablePositionIncrements="true" /&gt;
              &lt;filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="true"/&gt;
              &lt;filter class="solr.LowerCaseFilterFactory"/&gt;
          &lt;/analyzer&gt;
      &lt;/fieldType&gt;
      &lt;fieldType name="alphaOnlySort" class="solr.TextField" sortMissingLast="true" omitNorms="true"&gt;
      &lt;analyzer&gt;   
          &lt;tokenizer class="solr.KeywordTokenizerFactory"/&gt;
              &lt;filter class="solr.LowerCaseFilterFactory" /&gt;
          &lt;filter class="solr.TrimFilterFactory" /&gt;
          &lt;filter class="solr.PatternReplaceFilterFactory"
              pattern="([^a-z])" replacement="" replace="all"
          /&gt;
    &lt;/analyzer&gt;
  &lt;/fieldType&gt;
      &lt;fieldtype name="ignored" stored="false" indexed="false" class="solr.StrField" /&gt; 
      &lt;fieldType name="string" class="solr.StrField" sortMissingLast="true" /&gt;    
  &lt;/types&gt;
  &lt;fields&gt;
      &lt;field name="journal-title" type="text" indexed="true" stored="true" required="true" multiValued="false" /&gt; 
      &lt;field name="pmc" type="string" indexed="true" stored="true" required="false" multiValued="false" /&gt; 
      &lt;field name="pmid" type="string" indexed="true" stored="true" required="false" multiValued="false" /&gt;
      &lt;field name="other"  type="string" indexed="true" stored="true" required="false" multiValued="false" /&gt;     
      &lt;field name="doi" type="string" indexed="true" stored="true" required="false" multiValued="false" /&gt;        
      &lt;field name="article-title" type="text" indexed="true" stored="true" required="false" multiValued="true" /&gt; 
      &lt;!--&lt;field name="surname" type="string" indexed="true" stored="true" required="true" multiValued="true" /&gt; 
      &lt;field name="given-name" type="string" indexed="true" stored="true" required="true" multiValued="true" /&gt;       
      &lt;field name="pub-date_day" type="long" indexed="true" stored="true" required="true" multiValued="true" /&gt;--&gt;
      &lt;field name="text" type="text_general" indexed="true" stored="true" required="true" multiValued="true"/&gt;
      &lt;field name="_version_" type="long" indexed="true" stored="true"/&gt;
  &lt;/fields&gt;
  &lt;uniqueKey&gt;pmc&lt;/uniqueKey&gt;
  &lt;solrQueryParser defaultOperator="OR"/&gt;
  &lt;/schema&gt;
  </code></pre>
  
  <p>And I have configured a request handler as:</p>
  
  <pre><code>&lt;requestHandler name="/dataimport" class="org.apache.solr.handler.dataimport.DataImportHandler"&gt;
  &lt;lst name="defaults"&gt;
  &lt;str name="config"&gt;data-config.xml&lt;/str&gt;
  &lt;/lst&gt;
  </code></pre>
  
  <p></p>
  
  <p>When I issue a full-import request to Solr 
  (<code>http://localhost:8983/solr/dataimport?command=full-import</code>) 
  for the first time, I get this response:</p>
  
  <pre><code>&lt;response&gt;
      &lt;lst name="responseHeader"&gt;
          &lt;int name="status"&gt;0&lt;/int&gt;
          &lt;int name="QTime"&gt;18&lt;/int&gt;
      &lt;/lst&gt;
      &lt;lst name="initArgs"&gt;
          &lt;lst name="defaults"&gt;
      &lt;str name="config"&gt;data-config.xml&lt;/str&gt;
      &lt;/lst&gt;
      &lt;/lst&gt;
      &lt;str name="command"&gt;full-import&lt;/str&gt;
      &lt;str name="status"&gt;idle&lt;/str&gt;
      &lt;str name="importResponse"/&gt;
      &lt;lst name="statusMessages"/&gt;
      &lt;str name="WARNING"&gt;
      This response format is experimental. It is likely to change in the future.
      &lt;/str&gt;
      &lt;/response&gt;
  </code></pre>
  
  <p>And for the second time, I get this response:</p>
  
  <pre><code>&lt;response&gt;
      &lt;lst name="responseHeader"&gt;
          &lt;int name="status"&gt;0&lt;/int&gt;
          &lt;int name="QTime"&gt;6&lt;/int&gt;
      &lt;/lst&gt;
      &lt;lst name="initArgs"&gt;
      &lt;lst name="defaults"&gt;
          &lt;str name="config"&gt;data-config.xml&lt;/str&gt;
      &lt;/lst&gt;
      &lt;/lst&gt;
      &lt;str name="command"&gt;full-import&lt;/str&gt;
      &lt;str name="status"&gt;idle&lt;/str&gt;
      &lt;str name="importResponse"/&gt;
      &lt;lst name="statusMessages"&gt;
          &lt;str name="Total Requests made to DataSource"&gt;0&lt;/str&gt;
          &lt;str name="Total Rows Fetched"&gt;0&lt;/str&gt;
          &lt;str name="Total Documents Skipped"&gt;0&lt;/str&gt;
          &lt;str name="Full Dump Started"&gt;2013-10-12 20:45:12&lt;/str&gt;
          &lt;str name=""&gt;
          Indexing completed. Added/Updated: 0 documents. Deleted 0 documents.
          &lt;/str&gt;
          &lt;str name="Committed"&gt;2013-10-12 20:45:12&lt;/str&gt;
          &lt;str name="Total Documents Processed"&gt;0&lt;/str&gt;
          &lt;str name="Time taken"&gt;0:0:0.270&lt;/str&gt;
      &lt;/lst&gt;
      &lt;str name="WARNING"&gt;
          This response format is experimental. It is likely to change in the future.
      &lt;/str&gt;
  &lt;/response&gt;
  </code></pre>
  
  <p>When I query Solr, I find that no documents are indexed. Could anyone help me out with this? Are the schema and data config right?</p>