title: Bypassing "ghost" line break or end of file (EOF) in data.table::fread
body: <p>I'm loading several large, tab-delimited text files exported from an (accessible to me) database into R using <code>data.table::fread</code>.  <code>fread</code> handles most of the files with great ease and speed, but one of the files is generating a regularly-reported <code>fread</code> error:</p>
  
  <pre><code>Error in fread(read_problem, encoding = "UTF-8", na.strings = "", header = TRUE,  : 
                     Expected sep ('  ') but new line or EOF ends field ...
  </code></pre>
  
  <p>A smaller (2000 rows) version of the file containing the offending line is available here (<a href="https://drive.google.com/file/d/0B0srJaKm5SW_UVBBdkJjZHFoa1U/view">RDS file</a>).</p>
  
  <p>Here's how I've tried to diagnose the problem to this point:</p>
  
  <pre><code>library(data.table) # I'm using 1.9.7 development (same error with 1.9.6)
  read_problem &lt;- readRDS("read_problem.rds")
  error &lt;- fread(read_problem, encoding = "UTF-8", na.strings = "",
                 header = TRUE, sep = "\t", 
                 colClasses = rep("character", 44),  # For simplicity
                 verbose = TRUE)
  </code></pre>
  
  <p>If I excise the offending line, the problem disappears:</p>
  
  <pre><code>cat(read_problem, file = "temp")   
  string_vec &lt;- readLines("temp")
  clipped_vec &lt;- string_vec[-1027] # Get rid of problem line 1027
  restored &lt;- paste(clipped_vec, collapse = "\n")
  noerror &lt;- fread(restored, encoding = "UTF-8", na.strings = "",
                   header = TRUE, sep = "\t", 
                   colClasses = rep("character", 44)) # For simplicity
  class(noerror)
  [1] "data.table" "data.frame"
  
  dim(noerror) 
  [1] 1999   44
  </code></pre>
  
  <p>The error message seems clear enough: <code>fread</code> is looking for a "\t" but is finding something else in its place.</p>
  
  <p>But I find nothing obvious from a closer look at the offending line relative to those around it.</p>
  
  <p>The number of tab characters is the same</p>
  
  <pre><code>sapply(gregexpr("\t", string_vec[1026:1028]), length)
  [1] 43 43 43
  </code></pre>
  
  <p>Line break information seems identical</p>
  
  <pre><code>unlist(gregexpr("\n", string_vec[1026:1028]))
  [1] -1 -1 -1
  </code></pre>
  
  <p>Here's a look at the offending line itself as a string:</p>
  
  <pre><code>string_vec[1027]
  [1] "URN:CornellLabOfOrnithology:EBIRD:OBS132960387\t29816\tspecies\tNelson's Sparrow\tAmmodramus nelsoni\t\t\t1\t\t\tUnited States\tUS\tGeorgia\tUS-GA\tGlynn\tUS-GA-127\tUS-GA_3181\t\t\tJekyll Island\tL140461\tH\t31.0464993\t-81.4113007\t1990-11-03\t13:15:00\t\"Jekyll Island and Causeway. Partly cloudy, mild, NE wind 8-15 mph. Note: Did very little birding in upland habitats as time available was rather brief.\"  Data entered on behalf of Paul Sykes by Alison Huff (arhuff@uga.edu) on 12-15-11.\tListed on old Georgia Field Checklist as \"Sparrow, Sharp-tailed.\"\tobsr289931\tPaul\tSykes\tS9336358\teBird - Traveling Count\tEBIRD\t270\t8.047\t\t1\t1\t\t1\t0\t\t"
  </code></pre>
  
  <p>Any advice to get around this problem without the manual extraction of offending lines?</p>