title: How is Spark reading my image using the image format?
body: <p>It might be a silly question but I can't figure out how Spark read my image using the <code>spark.read.format("image").load(....)</code> argument.    </p>
  
  <p>After importing my image which gives me the following: </p>
  
  <pre><code>&gt;&gt;&gt; image_df.select("image.height","image.width","image.nChannels", "image.mode", "image.data").show()
  +------+-----+---------+----+--------------------+
  |height|width|nChannels|mode|                data|
  +------+-----+---------+----+--------------------+
  |   430|  470|        3|  16|[4D 55 4E 4C 54 4...|
  +------+-----+---------+----+--------------------+
  </code></pre>
  
  <p>I arrive to the conclusion that:  </p>
  
  <ul>
  <li>my image is 430x470 pixels, </li>
  <li>my image is colored (RGB due to nChannels = 3) which is an openCV compatible-type,</li>
  <li>my image mode is 16 which corresponds to a particular openCV byte-order.
  
  <ul>
  <li>Does someone knows which website/documentation I could browse to know more about it?</li>
  </ul></li>
  <li>the data in the data column is of type <code>Binary</code> but:
  
  <ul>
  <li>when I run <code>image_df.select("image.data").take(1)</code> I got an output which seems to be only one array (see below).</li>
  </ul></li>
  </ul>
  
  <pre><code>&gt;&gt;&gt; image_df.select("image.data").take(1)
  
  # **1/** Here are the last elements of the result
  ....&lt;&lt;One Eternity Later&gt;&gt;....x92\x89\x8a\x8d\x84\x86\x89\x80\x84\x87~'))]
  
  # 2/ I got also several part of the result which looks like:
  .....\x89\x80\x80\x83z|\x7fvz}tpsjqtkrulsvmsvmsvmrulrulrulqtkpsjnqhnqhmpgmpgmpgnqhnqhn
  qhnqhnqhnqhnqhnqhmpgmpgmpgmpgmpgmpgmpgmpgnqhnqhnqhnqhnqhnqhnqhnqhknejmdilcilchkbh
  kbilcilckneloflofmpgnqhorioripsjsvmsvmtwnvypx{ry|sz}t{~ux{ry|sy|sy|sy|sz}tz}tz}tz}
  ty|sy|sy|sy|sz}t{~u|\x7fv|\x7fv}.....
  
  </code></pre>
  
  <p>What come next are linked to the results displayed above. Those might be due to my lack of knowledge concerning openCV (or else). Nonetheless:</p>
  
  <ul>
  <li><strong>1/</strong> I don't understand the fact that if I got an RGB image, I should have 3 matrix but the output finishes by <code>.......\x84\x87~'))]</code>. I was more thinking on obtaining something like <code>[(...),(...),(...\x87~')]</code>.</li>
  <li><strong>2/</strong> Is this part has a special meaning? Like those are the separator between each matrix or something?</li>
  </ul>
  
  <p>To be more clear about what I'm trying to achieve, I want to process images to do pixel comparison between each images. Therefore, I want to know the pixel values for a given position in my image (I assume that if I have an RGB image, I shall have 3 pixel values for a given position). </p>
  
  <p>Example: let's say that I have a webcam pointing to the sky only during the day and I want to know the values of a pixel at a position corresponding to the top left sky part, I found out that the concatenation of those values gives the colour <em>Light Blue</em> which says that the photo was taken on a sunny day. Let's say that the only possibility is that a sunny day takes the colour <code>Light Blue</code>.<br>
  Next I want to compare the previous concatenation with another concat of pixel values at the exact same position but from a picture taken the next day. If I found out that they are not equal then I conclude that the given picture was taken on a cloudy/rainy day. If equal then sunny day.</p>
  
  <p>Any help on that would be highly appreciated. I have vulgarized my example for a better understanding but my goal is pretty much the same. I know that ML model can exist to achieve those stuff but I would be happy to try this first. My first goal is to split this column into 3 columns corresponding to each color code: a red matrix, a green matrix, a blue matrix</p>