Operations using ffdfwith in R <p>I am using ff and R because I have a huge dataset (around 16 GB) to work with. As a test case, I got the file to read around 1M records and wrote it out as a ff database.</p> <pre><code>system.time(te3 &lt;- read.csv.ffdf(file="testdata.csv", sep = ",", header=TRUE, first.rows=10000, next.rows=50000, colClasses=c("numeric","numeric","numeric","numeric"))) </code></pre> <p>I have uploaded the resulting file (te3) here: <a href="http://bit.ly/1c8pXqt" rel="nofollow">http://bit.ly/1c8pXqt</a></p> <p>I tried to do a simple calculation to create a new variable</p> <pre><code>ffdfwith(te3, {odfips &lt;- ofips*100000 + dfips}) </code></pre> <p>I get the following error (there are no missing records) which has flummoxed me:</p> <pre><code>Error in if (by &lt; 1) stop("'by' must be &gt; 0") : missing value where TRUE/FALSE needed In addition: Warning message: In chunk.default(from = 1L, to = 1000000L, by = 2293760000, maxindex = 1000000L) : NAs introduced by coercion </code></pre> <p>Any insights will be appreciated. Also, related to FF, is it possible to use standard R packages such as MCMC (I need to use the inverse gamma function) with FF databases?</p> <p>TIA,</p> <p>Krishnan</p>