title: Map a JSON dictionary to case class with Spark
body: <p>For the last hours, I was trying to convert a JSON file to a Scala <code>case class</code> with Apache Spark.</p>
  
  <p>The JSON has the following structure:</p>
  
  <pre><code>{
    "12": {
      "wordA": 1,
      "wordB": 2,
      "wordC": 3
    },
    "13": {
      "wordX": 10,
      "wordY": 12,
      "wordZ": 15
    }
  }
  </code></pre>
  
  <p><strong>First try: Set an build-up schema</strong></p>
  
  <p>I have tried to build artificially my schema:</p>
  
  <pre><code>val schema = new StructType()
     .add("",MapType(StringType, new StructType()
            .add("", StringType)
            .add("", IntegerType)))
  val df = session.read
    .option("multiline",true)
    .option("mode", "PERMISSIVE")
    .schema(schema)
    .json(filePath)
  df.show()
  </code></pre>
  
  <p>But this is obviously not right since I have to give the field name.</p>
  
  <p><strong>Second try: map to a case class</strong></p>
  
  <p>I have also tried to create <code>case class</code>es, which is a bit more elegant:</p>
  
  <pre><code>case class KeywordData (keywordsByCode: Map[String, WordAndWeight])
  
  case class WordAndWeight (word: String, weight: Int)
  </code></pre>
  
  <p><strong>Problem:</strong></p>
  
  <p>But in any case, df.show() displays: </p>
  
  <pre><code>+----+
  |    |
  +----+
  |null|
  +----+
  </code></pre>
  
  <p>The JSON structure is not easy to manipulate since my columns don't have a fix name. Any idea?</p>
  
  <p><strong>Expected result</strong></p>
  
  <p>A map with 12 and 13 as key and List[wordA,...wordC] respectively List[wordX, ..., wordZ] as values</p>
  
  <p><strong>Edit: Map of Map</strong>
  With the case class</p>
  
  <pre><code>case class WordAndWeight(code: Map[String, Map[String, Integer]])
  </code></pre>
  
  <p>It gives me the following error: </p>
  
  <pre><code>+-------+----------+
  |     12|        13|
  +-------+----------+
  |[1,2,3]|[10,12,15]|
  +-------+----------+
  
  
  cannot resolve '`code`' given input columns: [12, 13];
  org.apache.spark.sql.AnalysisException: cannot resolve '`code`' given input columns: [12, 13];
      at org.apache.spark.sql.catalyst.analysis.package$AnalysisErrorAt.failAnalysis(package.scala:42)
  </code></pre>