title: Retrieving sentence score based on values of words in a dictionary
body: <p><strong>Edited</strong> <code>df</code> and <code>dict</code> </p>
  
  <p>I have a data frame containing sentences:</p>
  
  <pre><code>df &lt;- data_frame(text = c("I love pandas", "I hate monkeys", "pandas pandas pandas", "monkeys monkeys"))
  </code></pre>
  
  <p>And a dictionary containing words and their corresponding scores:</p>
  
  <pre><code>dict &lt;- data_frame(word = c("love", "hate", "pandas", "monkeys"),
                     score = c(1,-1,1,-1))
  </code></pre>
  
  <p>I want to append a column "score" to <code>df</code> that would sum the score for each sentence:</p>
  
  <p><strong>Expected results</strong></p>
  
  <pre><code>                  text score
  1        I love pandas     2
  2       I hate monkeys    -2
  3 pandas pandas pandas     3
  4      monkeys monkeys    -2
  </code></pre>
  
  <p><strong>Update</strong></p>
  
  <p>Here are the results so far:</p>
  
  <p><em>Akrun's methods</em></p>
  
  <p><em>Suggestion 1</em></p>
  
  <pre><code>df %&gt;% mutate(score = sapply(strsplit(text, ' '), function(x) with(dict, sum(score[word %in% x]))))
  </code></pre>
  
  <p>Note that for this method to work, I had to use <code>data_frame()</code> to create <code>df</code> and <code>dict</code> instead of <code>data.frame()</code> otherwise I get: <code>Error in strsplit(text, " ") : non-character argument</code></p>
  
  <pre><code>Source: local data frame [4 x 2]
  
                    text score
  1        I love pandas     2
  2       I hate monkeys    -2
  3 pandas pandas pandas     1
  4      monkeys monkeys    -1
  </code></pre>
  
  <p>This does not accounts for multiple matches in a single string. Close to expected result, but not quite there yet.</p>
  
  <p><em>Suggestion 2</em></p>
  
  <p>I tweaked a bit one of akrun's suggestion in the comments to apply it to the edited post</p>
  
  <pre><code>cbind(df, unnest(stri_split_fixed(df$text, ' '), group) %&gt;% 
          group_by(group) %&gt;% 
          summarise(score = sum(dict$score[dict$word %in% x])) %&gt;% 
          ungroup() %&gt;% select(-group) %&gt;% data.frame())
  </code></pre>
  
  <p>This does not account for multiple matches in a string:</p>
  
  <pre><code>                  text score
  1        I love pandas     2
  2       I hate monkeys    -2
  3 pandas pandas pandas     1
  4      monkeys monkeys    -1
  </code></pre>
  
  <p><em>Richard Scriven's methods</em></p>
  
  <p><em>Suggestion 1</em></p>
  
  <pre><code>group_by(df, text) %&gt;%
  mutate(score = sum(dict$score[stri_detect_fixed(text, dict$word)]))
  </code></pre>
  
  <p>After updating all packages, this now works (although it does not account for multiple matches)</p>
  
  <pre><code>Source: local data frame [4 x 2]
  Groups: text
  
                    text score
  1        I love pandas     2
  2       I hate monkeys    -2
  3 pandas pandas pandas     1
  4      monkeys monkeys    -1
  </code></pre>
  
  <p><em>Suggestion 2</em></p>
  
  <pre><code>total &lt;- with(dict, {
    vapply(df$text, function(X) {
      sum(score[vapply(word, grepl, logical(1L), x = X, fixed = TRUE)])
    }, 1)
  })
  
  cbind(df, total)
  </code></pre>
  
  <p>This give the same results:</p>
  
  <pre><code>                  text total
  1        I love pandas     2
  2       I hate monkeys    -2
  3 pandas pandas pandas     1
  4      monkeys monkeys    -1
  </code></pre>
  
  <p><em>Suggestion 3</em></p>
  
  <pre><code>s &lt;- strsplit(df$text, " ")
  total &lt;- vapply(s, function(x) sum(with(dict, score[match(x, word, 0L)])), 1)
  cbind(df, total)
  </code></pre>
  
  <p>This actually works:</p>
  
  <pre><code>                  text total
  1        I love pandas     2
  2       I hate monkeys    -2
  3 pandas pandas pandas     3
  4      monkeys monkeys    -2
  </code></pre>
  
  <p><em>Thelatemail's method</em></p>
  
  <pre><code>res &lt;- sapply(dict$word, function(x) {
    sapply(gregexpr(x,df$text),function(y) length(y[y!=-1]) )
  })
  
  cbind(df, score = rowSums(res * dict$score))
  </code></pre>
  
  <p>Note that I added the <code>cbind()</code> part. This actually match the expected result.</p>
  
  <pre><code>                  text score
  1        I love pandas     2
  2       I hate monkeys    -2
  3 pandas pandas pandas     3
  4      monkeys monkeys    -2
  </code></pre>
  
  <p><strong>Final answer</strong></p>
  
  <p>Inspired by akrun's suggestion, here is what I ended up writing as the most <code>dplyr</code>-esque solution:</p>
  
  <pre><code>library(dplyr)
  library(tidyr)
  library(stringi)
  
  bind_cols(df, unnest(stri_split_fixed(df$text, ' '), group) %&gt;% 
              group_by(x) %&gt;% mutate(score = sum(dict$score[dict$word %in% x])) %&gt;% 
              group_by(group) %&gt;% 
              summarise(score = sum(score)) %&gt;% 
              select(-group))
  </code></pre>
  
  <p>Although I will implement Richard Scriven's suggestion #3 since it's the most efficient.</p>
  
  <p><strong>Benchmark</strong></p>
  
  <p>Here are the suggestions applied to much larger datasets (<code>df</code> of 93 sentences and <code>dict</code> of 14K words) using <code>microbenchmark()</code>:</p>
  
  <pre><code>mbm = microbenchmark(
    akrun = df %&gt;% mutate(score = sapply(stri_detect_fixed(text, ' '), function(x) with(dict, sum(score[word %in% x])))),
    akrun2 = cbind(df, unnest(stri_split_fixed(df$text, ' '), group) %&gt;% group_by(group) %&gt;% summarise(score = sum(dict$score[dict$word %in% x])) %&gt;% ungroup() %&gt;% select(-group) %&gt;% data.frame()),
    rscriven1 = group_by(df, text) %&gt;% mutate(score = sum(dict$score[stri_detect_fixed(text, dict$word)])),
    rscriven2 = cbind(df, score = with(dict, { vapply(df$text, function(X) { sum(score[vapply(word, grepl, logical(1L), x = X, fixed = TRUE)])}, 1)})),
    rscriven3 = cbind(df, score = vapply(strsplit(df$text, " "), function(x) sum(with(dict, score[match(x, word, 0L)])), 1)),
    thelatemail = cbind(df, score = rowSums(sapply(dict$word, function(x) { sapply(gregexpr(x,df$text),function(y) length(y[y!=-1]) ) }) * dict$score)),
    sbeaupre = bind_cols(df, unnest(stri_split_fixed(df$text, ' '), group) %&gt;% group_by(x) %&gt;% mutate(score = sum(dict$score[dict$word %in% x])) %&gt;% group_by(group) %&gt;% summarise(score = sum(score)) %&gt;% select(-group)),
    times = 10
  )
  </code></pre>
  
  <p>And the results:</p>
  
  <p><img src="https://i.stack.imgur.com/ffXBD.png" alt="enter image description here"></p>