title: how to efficiently extract values from (many) large raster(s) for many polygones
body: <p>I have a set of points and <strong>I want to extract values from several large rasters</strong> for a buffer around those points. <em>The rasters are too large to be held in memory</em> (> 1e10 cells). I illustrate my current approach below but would be interested if there is a faster approach.</p>
  
  <pre><code>library(maps)
  library(sf)
  library(raster)
  library(dplyr)
  library(parallel)
  
  # sf object with polygones for which we want values
  crs &lt;- "+proj=utm +zone=33 +ellps=GRS80 +towgs84=0,0,0,0,0,0,0 +units=m +no_defs"
  map &lt;- sf::st_as_sf( maps::map(regions = "Sweden", plot = FALSE, fill = TRUE))
  map &lt;- st_transform(map, crs = crs)
  sf_points &lt;- st_sfc(st_sample(map, 100))
  
  sf_points &lt;- 
    data.frame(A = 1:length(sf_points)) %&gt;% 
    st_set_geometry(sf_points) 
  
  # raster too large to fit in memory
  # the raster(s) I am working on has 10m resolution
  
  r &lt;- raster(extent(map), nrow = 15000, ncol = 7000,
         crs = crs)
  
  values(r) &lt;- rep(sample(1:10, 77, replace = T),  length.out = ncell(r))
  
  #use the parallel package for parallel processing
  cluster &lt;- makeCluster(4)
  clusterExport(cluster, c("r","sf_points", "as_Spatial"))
  
  List_points &lt;- 
    sf_points %&gt;% 
    mutate(split = rep(1:ceiling(n()/4), each=4, length.out=n())) %&gt;% # 4 cores
    split(f = .$split) %&gt;% 
    parLapply(cl = cluster, X = ., function(x) raster::extract(r, y = as_Spatial(x), buffer = 5000)) %&gt;% 
    unlist(recursive = F)
  </code></pre>
  
  <p>I repeat the extraction for each raster. As the values are ordered I can then summarize the pixel values across lists. I cannot (easily) create a raster stack as the raster have different extent. </p>
  
  <p>using the <code>velox</code> package has sugested <a href="https://stackoverflow.com/questions/54565713/extracting-values-from-inside-polygons-raster-r">here</a> doesn't seem to work as it tries to load the raster into memory which fails. I could try to load it chunk-wise but then I would need to figure out which points are on which chunk...</p>