title: hive understanding table creation
body: <p>I am taking a <a href="https://courses.edx.org/courses/course-v1:Microsoft+DAT202.1x+3T2015/courseware/f0b45801068c404bb90b49853fe55ccd/1aaca55dcff640b99bc9ba93ac5fa4af/" rel="nofollow">mooc</a>.</p>
  
  <p>it told us to upload a few files from our PC to hdfs using below commands</p>
  
  <pre><code>azure storage blob upload local_path container data/logs/2008-01.txt.gz
  </code></pre>
  
  <p>I did the same.
  later on when I typed below command in PUTTY secure shell  I was able to see that file</p>
  
  <pre><code>hdfs dfs -ls /data/logs
  Found 6 items
  -rwxrwxrwx   1     331941 2016-03-03 15:56 /data/logs/2008-01.txt.gz
  -rwxrwxrwx   1     331941 2016-03-03 15:58 /data/logs/2008-02.txt.gz
  -rwxrwxrwx   1     331941 2016-03-03 15:58 /data/logs/2008-03.txt.gz
  -rwxrwxrwx   1     331941 2016-03-03 15:58 /data/logs/2008-04.txt.gz
  -rwxrwxrwx   1     331941 2016-03-03 15:58 /data/logs/2008-05.txt.gz
  -rwxrwxrwx   1     331941 2016-03-03 15:58 /data/logs/2008-06.txt.gz
  </code></pre>
  
  <p>then we started a <code>hive</code> terminal and first created a table and then inserted data into that table using </p>
  
  <pre><code>load data inpath '/data/logs' into TABLE rawlog;
  </code></pre>
  
  <p>Then we created an external table using below command</p>
  
  <pre><code>CREATE EXTERNAL TABLE cleanlog
  (log_date DATE,
  log_time STRING,
  c_ip STRING,
  cs_username STRING,
  s_ip STRING,
  s_port STRING,
  cs_method STRING,
  cs_uri_stem STRING,
  cs_uri_query STRING,
  sc_status STRING,
  sc_bytes INT,
  cs_bytes INT,
  time_taken INT,
  cs_user_agent STRING,
  cs_referrer STRING)
  ROW FORMAT DELIMITED FIELDS TERMINATED BY ' '
  STORED AS TEXTFILE LOCATION '/data/cleanlog';
  </code></pre>
  
  <p>we inserted data into the table using</p>
  
  <pre><code>INSERT INTO TABLE cleanlog
  SELECT *
  FROM rawlog
  WHERE SUBSTR(log_date, 1, 1) &lt;&gt; '#';
  </code></pre>
  
  <p>I exited out of hive and typed in below command</p>
  
  <pre><code>hdfs dfs -ls /data/logs
  </code></pre>
  
  <ol>
  <li>I dont see anything in that folder, why? where did uploaded log
  files go? </li>
  <li>Where is the rawlog table? does it exist in the same folder? Why dont i see it?</li>
  <li><p>Why do i see file 00000_0 in my cleanlog folder? is it the new
  table? If i type command</p>
  
  <p>hdfs dfs -ls /data/cleanlog</p></li>
  </ol>
  
  <p>The output that i get is </p>
  
  <pre><code>Found 1 items
  -rwxr-xr-x   1 sshuser supergroup   71323206 2016-03-03 16:11 /data/cleanlog/000000_0
  </code></pre>
  
  ################----------------------------------update 1
  
  <ol>
  <li>What would happen if load one more data file at <code>/data/logs/</code> and
  then run <code>select * from rawlog</code>? would it automatically pull data
  from the new file?</li>
  </ol>