title: Ifelse behavior within data.table (R)
body: <p>I have a data.table full of some consumer products. I've created some distinction for the products as <code>'low'</code>, <code>'high'</code>, or <code>'unknown'</code> quality. The data are time series, and I'm interested in smoothing out some seasonality in the data. If a product's raw classification (the classification churned out by the algorithm I used to determine quality) is <code>'low'</code> quality in period X, but its raw classification was <code>'high'</code> quality in period X-1, I'm reclassifying that product as <code>'high'</code> quality for period X. This process is done within some sort of product group distinction.</p>
  
  <p>To accomplish this, I've got something like the following:</p>
  
  <pre><code>require(data.table)
  
  # lag takes a column and lags it by one period,
  # padding with NA
  
  lag &lt;- function(var) {
      lagged &lt;- c(NA, 
                  var[1:(length(var)-1)])
      return(lagged)
  }
  
  set.seed(120)
  
  foo &lt;- data.table(group = c('A', rep(c('B', 'C', 'D'), 5)),
                    period = c(1:16),
                    quality = c('unknown', sample(c('high', 'low', 'unknown'), 15, replace = TRUE)))
  
  foo[, quality_lag := lag(quality), by = group]
  
  foo[, quality_1 := ifelse(quality == 'low' &amp; quality_lag == 'high',
                            'high',
                            quality)]
  </code></pre>
  
  <p>Taking a look at <code>foo</code>:</p>
  
  <pre><code>    group period quality quality_lag quality_1
   1:     A      1 unknown          NA   unknown
   2:     B      2     low          NA        NA
   3:     C      3    high          NA      high
   4:     D      4     low          NA        NA
   5:     B      5 unknown         low   unknown
   6:     C      6    high        high      high
   7:     D      7     low         low       low
   8:     B      8 unknown     unknown   unknown
   9:     C      9    high        high      high
  10:     D     10 unknown         low   unknown
  11:     B     11 unknown     unknown   unknown
  12:     C     12     low        high      high
  13:     D     13 unknown     unknown   unknown
  14:     B     14    high     unknown      high
  15:     C     15    high         low      high
  16:     D     16 unknown     unknown   unknown
  </code></pre>
  
  <p>So, <code>quality_1</code> is mostly what I want. If period X is <code>'low'</code> and period X-1 is <code>'high'</code>, we see the reclassification to <code>'high'</code> occurs and everything is left mostly intact from <code>quality</code>. However, when <code>quality_lag</code> is NA, <code>'low'</code> gets reclassified to <code>NA</code> in <code>quality_1</code>. This is not an issue with <code>'high'</code> or <code>'unknown'</code>. </p>
  
  <p>That is, the first four rows of <code>foo</code> <em>should</em> look like this:</p>
  
  <pre><code>   group period quality quality_lag quality_1
   1:     A      1 unknown          NA   unknown
   2:     B      2     low          NA       low
   3:     C      3    high          NA      high
   4:     D      4     low          NA       low
  </code></pre>
  
  <p>Any thoughts on what is causing this? </p>