title: Pig: Filtering out the last tuple in a relation
body: <p>I have below data in hdfs and I want to remove the last row.</p>
  
  <h2>/user/cloudera/test/testfile.csv</h2>
  
  <pre><code>Day,TimeCST,Conditions
  1,12:53 AM,Clear
  1,1:53 AM,Clear
  1,2:53 AM,Clear
  1,3:53 AM,Clear
  1,4:53 AM,Clear
  1,5:53 AM,Clear
  1,6:53 AM,Clear
  1,7:53 AM,Clear
  1,8:53 AM,Clear
  1,9:53 AM,Clear
  1,10:53 AM,Clear
  1,11:53 AM,Clear
  1,12:53 PM,Clear
  1,1:53 PM,Clear
  1,2:53 PM,Clear
  1,3:53 PM,Clear
  1,4:53 PM,Clear
  1,5:53 PM,Clear
  </code></pre>
  
  <p>First, I <code>load</code> the data, remove the header, and get the number of rows/tuples:</p>
  
  <pre><code>rawdata = LOAD 'hdfs:/user/cloudera/test/testfile.csv' using PigStorage(',') AS (day:int, timecst:chararray, condition:chararray);
  filtereddata = FILTER rawdata BY day &gt; 0; --filters out header
  rowcount = FOREACH (GROUP filtereddata ALL) GENERATE COUNT_STAR(filtereddata);
  dump rowcount; --Prints (18)
  </code></pre>
  
  <p>Next, I <code>rank</code> the data and then try to use the generated row number to <code>filter</code> out the last row/tuple:</p>
  
  <pre><code>ranked = RANK filtereddata;
  weatherdata = FILTER ranked BY $0 != rowcount.$0;
  </code></pre>
  
  <p>Above <code>filter</code> operation fails with the following error:</p>
  
  <pre><code>ERROR 2017: Internal error creating job configuration.
  org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to open iterator for alias weatherdata.....
  </code></pre>
  
  <p>However, if I hardcode the rowcount into my script as below, the job runs fine: </p>
  
  <pre><code>weatherdata = FILTER ranked BY $0 != 18;
  </code></pre>
  
  <p>I want to avoid hardcoding the rowcount. Did you detect where I might be going astray? Thanks.</p>
  
  <p><strong>Apache Pig version 0.12.0-cdh5.5.0 (rexported)
  compiled Nov 09 2015, 12:41:48</strong></p>