title: Swift 4 base64 String to Data not working due to String containing "incomplete" emoji
body: <p>I am coming from this post <a href="https://stackoverflow.com/questions/52521480/swift-4-json-string-with-unknown-utf8-character-is-not-convertible-to-data">Swift 4 JSON String with unknown UTF8 &quot;�&quot; character is not convertible to Data/ Dictionary</a> but meanwhile I was able to isolate the issue to a 10-character-string.</p>
  
  <p>Short intro: one user's app did not show any content. Looking at his 6kb of data in plain text with TextWrangler I found 2 red question marks</p>
  
  <p><a href="https://i.stack.imgur.com/uTAIb.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/uTAIb.png" alt="enter image description here"></a>. </p>
  
  <p>I tried to cut some chunks of the base64-encoded data around the question marks and convert them to Data which didn't work. As soon as I removed the bits from the red question mark from the chunks it seemed to work again. Please take a look at my following Playground example:</p>
  
  <pre><code>//those do NOT work
  let toEndBracket = "ACAAKgBVAFMAQQAqACAnlgAg2DwAIgB9AF0A" // *USA* ' &lt;"}]//
  let toMidBracket = "ACAAKgBVAFMAQQAqACAnlgAg2DwAIgB9"     // *USA* ' &lt;"}//
  let toCarrot =     "ACAAKgBVAFMAQQAqACAnlgAg2DwA"         // *USA* ' &lt;//
  let toSpace =      "ACAAKgBVAFMAQQAqACAnlgAg"             // *USA* ' //
  
  //but this one WORKS
  let toApostrophe = "ACAAKgBVAFMAQQAqACAn"                 // *USA* '//
  //(basically the last one is without the space before the carrot, I've added the slashes after it to emphasize that)
  //clear strings taken from https://www.base64decode.org/ using the UTF-8 setting WITHOUT "Live mode".
  
  if let textData = Data(base64Encoded: toApostrophe) {
      print("Data created")   //works for all of them
      print(textData)
      if let decodedString = String(data: textData, encoding: .utf8) {
          print("WORKED!!!")  //only happens for the toApostrophe
          print(decodedString)
      } else {
          print("DID NOT WORK")
      }
  }
  </code></pre>
  
  <p>So it basically fails as soon as soon as it contains <code>lgAg</code>. Replacing this with something like <code>U29t</code> does make the small strings work again but I can't do this in production code as I am sure my examples aren't the only occurrences of this issue. I don't care what happens with the original characters/ symbols/ emojis that are causing this, if there was a way to just "ignore" them that would be more than helpful already!</p>
  
  <p>Here is another example of where this occurs:</p>
  
  <pre><code>//OTHER SYMBOL WITH SAME BEHAVIOR
  //not working
  let secondFromSpace =  "ACDYPAAiACwA"       // &lt;",//
  
  //WORKING
  let secondFromCarrot = "PAAiACwA"           //&lt;",//
  </code></pre>
  
  <p>Here is the original text in its habitat, a messenger message saying "USA" with an emoji hence the "USA" in my examples texts and my suspicion it's the emojis that make it break:</p>
  
  <p><a href="https://i.stack.imgur.com/pqwCm.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/pqwCm.png" alt="enter image description here"></a></p>
  
  <p>I'd be grateful if someone can tell me how I can "clean up" the base64 string so it's convertible to data again. It might also be due to some weird encoding with some of the emojis but for the very most cases, the app receives and displays content with emojis just fine.</p>
  
  <hr>
  
  <p>I have finally figured out why this is happening. It's not a swift-side solution to my problem but now it makes at least some sense. For previews of new content I cut off strings to match the viewport of the browser. This particular unlucky user has had the USA flag emoji on the edge of the display bezel. Never would I have thought of emojis consisting of multiple letters and JavaScript's <code>substring()</code> decapitating them. Take a look at the picture, this explains where the character comes from etc.</p>
  
  <p>I would still appreciate an answer as to how to avoid/ignore/catch that in Swift but to every poor soul running into this issue I hope you will stumble across this thread.</p>
  
  <p><a href="https://i.stack.imgur.com/COFpt.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/COFpt.png" alt="enter image description here"></a></p>