title: Hive Hadoop : Need to LOAD data into a table based on conditions on the input file
body: <p>I am new to Hadoop Hive and have just started to do basic querying in hive.
  My intention is I have an input text file (which has large number of records per line). The format of the file is something like this:</p>
  
  <pre><code>1;23;0;;;;1;3;2;1;1;4;5;6;;;;
  
  1;43;6;;;;1;3;2;1;1;4;5;5;;;;
  
  1;53;7;;;;1;3;2;1;1;4;5;2;;;;
  </code></pre>
  
  <p>(Each integer before a "<code>;</code>" has a meaning which I am intending to put it in Hive table as column names - and each line contains about 400 fields)</p>
  
  <p>So for inserting this I have created a table "test" - using the following query:</p>
  
  <pre><code>CREATE TABLE test (field1 INT, field2 INT, field3 INT, field4 INT, ... field390 INT)
  ROW FORMAT DELIMITED
  FIELDS TERMINATED BY "\073";
  </code></pre>
  
  <p>And I load my text file with the records using the LOAD query as below:</p>
  
  <pre><code>LOAD DATA LOCAL INPATH '/tmp/test.txt'
  OVERWRITE INTO TABLE test;
  </code></pre>
  
  <p>For now all the fields are getting inserted into the table upto 50 fields accurately. Later I have mismatches.</p>
  
  <p>What I have in my format of input is - at 50th field in the test.txt I have a INT number which decides the number of fields to take following the field.</p>
  
  <p><strong>Example:</strong></p>
  
  <p>50th field: <strong>2</strong>   -> Hive has to take the next <strong>2*10</strong> field INT values and insert in the table.</p>
  
  <p>50th field: <strong>1</strong>   -> Hive has to take the next <strong>1*10</strong> field INT values and insert in the table. And the rest 10 fields can be set NULL.</p>
  
  <p>(The maximum value of 50th field is <strong>2</strong> - so I have reserved 2*10 fields for this in the table)</p>
  
  <p>After 50th+(2*10) fields , the data should be read normally in the sequence as it did before the 50th field.</p>
  
  <p>Do we have a way in which we can have a condition on the input so that the data gets inserted accordingly in Hive.</p>
  
  <p>A help may be appreciated. Need a solution which will not guide me to pre-process the test.txt and then supply to the table.</p>