title: Very Fast string fuzzy matching in R
body: <p>I have a set of 40.000 rows x 4 columns and I need to compare each column to itself in order to find the most closest result or the minimum levenshtein distance. The idea is to get an &quot;almost duplicate&quot; for every row. I have calculated with &quot;adist&quot; but seems too slow. For example, for only one column, 5.000 rows compared to all column dataset, 40.000 rows, takes almost 2 hours. This is, for 4 columns, 8 hours, and for the entire dataset, 32 hours. Is there any faster way to achieve the same? I need it to be in 1 or 2 hours if possible. This is an example of what have I done so far:</p>
  <pre><code>
      #vector example
      a&lt;-as.character(c(&quot;hello&quot;,&quot;allo&quot;,&quot;hola&quot;))
      b&lt;-as.character(c(&quot;hello&quot;,&quot;allo&quot;,&quot;hola&quot;))
      
      #execution time
      start_time &lt;- Sys.time()
      
      #Matrix with distance
      dist.name&lt;-adist(a,b, partial = TRUE, ignore.case = TRUE)
      
      #time elapsed
      end_time &lt;- Sys.time()
      end_time - start_time
      
      Output:
      Time difference of 5.873202 secs
      
      #result
      dist.name
            [,1] [,2] [,3]
      [1,]    0    4    5
      [2,]    2    0    2
      [3,]    5    4    0
  
  </code></pre>
  <h1>desired output (minimum distance for every row, but no for the same row), but I need it faster.</h1>
  <pre><code>[1,] 4
  [2,] 2
  [3,] 4
  </code></pre>