title: Reading XML with nested tags into a Spark RDD, and transforming to JSON
body: <p>I have following xml structure:</p>
  
  <pre><code>&lt;root&gt;
    &lt;items&gt;
      &lt;item&gt;
        &lt;property&gt;1&lt;/property&gt;
        &lt;property2&gt;2&lt;/property2&gt; 
        &lt;nested-property&gt;
          &lt;property&gt;1&lt;/property&gt;
        &lt;/nested-property&gt;   
      &lt;/item&gt;
    &lt;/items&gt;
  &lt;/root&gt;
  </code></pre>
  
  <p>I am using Spark 2.2, in its Scala API, and I have pretty big XML Files in a Local File System (10GB). </p>
  
  <p>My goal is to read each element and, do some transformation and convert it to <strong>Json</strong>, but the result should be similar to the XML I have provided.</p>
  
  <blockquote>
    <ul>
    <li><p>The key point is I need to iterate of the file not Line By Line, but "Tag by Tag", in this case, from tag  to the next </p></li>
    <li><p>Another point is the fact that I need to read nested xml tags, so I am not sure if <a href="https://github.com/databricks/spark-xml" rel="nofollow noreferrer">this library</a> can help me. My understanding is it's
    not reading XML nested tags, and also it is just working with
    DataFrames, and I would also prefer RDD.  Is there a way of achieving
    the functionality?</p></li>
    </ul>
  </blockquote>
  
  <p>I tried just reading as a text file and using the tag item as delimiter, but it did not work. It still considers line breakers as delimiters:</p>
  
  <pre><code>  val spark = SparkSession
      .builder()
      .config("textinputformat.record.delimiter", "&lt;/item&gt;")
      .master("local[*]")
      .getOrCreate()
  
  
    val documents = spark.sparkContext.textFile("/home/myuser/test-data/Records.xml")
  </code></pre>
  
  <blockquote>
    <p>How are the nested XML elements treated? How would I access them? </p>
    
    <p><strong>I don't want to flatten the nested structures, 
    I want transform each XML Row I iterate through into a separate JSON file - with the same structure.</strong></p>
  </blockquote>