title: Handling bad UTF-8 from json,in ruby
body: <p>I'm pulling data from remote json at <a href="http://hndroidapi.appspot.com/news/format/json/page/?appid=test" rel="noreferrer">http://hndroidapi.appspot.com/news/format/json/page/?appid=test</a> .  The problem I'm running into is that this API appears to be building the JSON without correctly handling UTF-8 encoding (correct me if I'm wrong here).  For example, part of the result that gets passed right now is </p>
  
  <pre><code>{
  "title":"IPad - please don&amp;euro;&amp;trade;t ding while you and I are asleep  ",
  "url":"http://modern-products.tumblr.com/post/25384729998/ipad-please-dont-ding-while-you-and-i-are-asleep",
  "score":"10 points",
  "user":"roee",
  "comments":"18 comments",
  "time":"1 hour ago",
  "item_id":"4128497",
  "description":"10 points by roee 1 hour ago  | 18 comments"
  }
  </code></pre>
  
  <p>Notice the <code>don&amp;euro;&amp;trade;t</code>.  And that isn't the only type of character it is choking on.  Is there anything I can do to convert the data into something clean, given that I don't control the API?</p>
  
  <p>Edit:</p>
  
  <p>Here is how I'm pulling down the JSON:</p>
  
  <pre><code>hn_url = "http://hndroidapi.appspot.com/news/format/json/page/?appid=test"
    url = URI.parse(hn_url)
  
    # Attempt to get the json
    req = Net::HTTP::Get.new(hn_url)
    req.add_field('User-Agent', 'Test')
    res = Net::HTTP.start(url.host, url.port) {|http| http.request(req) }
    response = res.body
    if response.nil?
      puts "Bad response when fetching HN json"
      return
    end
  
    # Attempt to parse the json
    result = JSON.parse(response)
    if result.nil?
      puts "Error parsing HN json"
      return
    end
  </code></pre>
  
  <p>Edit 2:</p>
  
  <p><strong>Just found the API's GitHub page.  Looks like this is an outstanding issue.  Still not sure if there's any workarounds that I can do from my end:</strong>
  <a href="https://github.com/glebpopov/Hacker-News-Droid-API/issues/4" rel="noreferrer">https://github.com/glebpopov/Hacker-News-Droid-API/issues/4</a></p>