title: Using grep() with Unicode characters in R
body: <p>(strap in!)</p>
  
  <p>Hi, I'm running into issues involving Unicode encoding in R. </p>
  
  <p>Basically, I'm importing data sets that contain Unicode (UTF-8) characters, and then running grep() searches to match values. For example, say I have:</p>
  
  <pre><code>bigData &lt;- c("foo","αβγ","bar","αβγγ (abgg)", ...)
  smallData &lt;- c("αβγ","foo", ...)
  </code></pre>
  
  <p>What I'm trying to do is take the entries in smallData and match them to entries in bigData. (The actual sets are matrixes with columns of values, so what I'm trying to do is find the indexes of the matches, so I can tell what row to add the values to.) I've been using</p>
  
  <pre><code>matches &lt;- grepl(smallData[i], bigData, fixed=T)
  </code></pre>
  
  <p>which usually results in a vector of matches. For i=2, it would return 1, since "foo" is element 1 of bigData. This is peachy and all is well. But RStudio seems to not be dealing with unicode characters properly. When I import the sets and view them, they use the character IDs.</p>
  
  <pre><code>dataset &lt;- read_csv("[file].csv", col_names = FALSE, locale = locale())
  </code></pre>
  
  <p>Using View(dataset) shows "aß&lt;U+03B3>" instead of "αβγ." The same goes for </p>
  
  <pre><code>dataset[1]
  A tibble: 1x1 &lt;chr&gt;
  [1] aß&lt;U+03B3&gt;
  
  print(dataset[1])
  A tibble: 1x1 &lt;chr&gt;
  [1] aß&lt;U+03B3&gt;
  </code></pre>
  
  <p>However, and this is why I'm stuck rather than just adjusting the encoding:</p>
  
  <pre><code>paste(dataset[1])
  [1] "αβγ"
  Encoding(toString(dataset[1]))
  [1] "UTF-8"
  </code></pre>
  
  <p>So it appears that R is recognizing in certain contexts that it should display Unicode characters, while in others it just sticks to--ASCII? I'm not entirely sure, but certainly a more limited set.</p>
  
  <p>In any case, regardless of how it displays, what I want to do is be able to get</p>
  
  <pre><code>grep("αβγ", bigData)
  [1] 2 4
  </code></pre>
  
  <p>However, none of the following work:</p>
  
  <pre><code>grep("αβ", bigData)               #(Searching the two letters that do appear to convert)
  grep("&lt;U+03B3&gt;",bigData,fixed=T)  #(Searching the code ID itself)
  grep("αβ", toString(bigData))     #(converts the whole thing to one string)
  grep("\\β", bigData)              #(only mentioning because it matches, bizarrely, to ß)
  </code></pre>
  
  <p>The only solution I've found is:</p>
  
  <pre><code>grep("\u03B3", bigData)
  [1] 2 4
  </code></pre>
  
  <p>Which is not ideal for a couple reasons, most jarringly that it doesn't look like it's possible to just take every &lt;U+####> and replace it with \u####, since not every Unicode character is converted to the &lt;U+####> format, but none of them can be searched. (i.e., α and ß didn't turn into their unicode keys, but they're also not searchable by themselves. So I'd have to turn them into their keys, then alter their keys to a form that grep() can use, <em>then</em> search.) </p>
  
  <p>That means I can't just regex the keys into a searchable format--and even if I could, I have a lot of entries including characters that'd need to be escaped (e.g., () or ), so having to remove the fixed=T term would be its own headache involving nested escapes.</p>
  
  <p>Anyway...I realize that a significant part of the problem is that my set apparently involves every sort of character under the sun, and it seems I have thoroughly entrapped myself in a net of regular expressions. </p>
  
  <p>Is there any way of forcing a search with (arbitrary) unicode characters? Or do I have to find a way of using regular expressions to escape every ( and α in my data set? (coordinate to that second question: is there a method to convert a unicode character to its key? I can't seem to find anything that does that specific function.)</p>