app/jobs/github_repo_scraper.rb
# TODO: we should cache all the users for a repo when a repo is requested, so
# we don't have to hit the DB as often, because I'll have to get the name
# of the user from the comment, search if it exist, then create it.
# If we had them cached I could search those, if not found, create it.
# Basically let's make that query when we get the repo.
we should cache all the users for a repo when a repo is requested, so
we don't have to hit the DB as often, because I'll have to get the name
of the user from the comment, search if it exist, then create it.
If we had them cached I could search those, if not found, create it.
Basically let's make that query when we get the repo.
we should cache all the users for a repo when a repo is requested so we don ' t have to hit the DB as often because I ' ll have to get the name of the user from the comment search if it exist then create it . If we had them cached I could search those if not found create it . Basically let ' s make that query when we get the repo .
def build_comment(raw_comment)
      user_name = raw_comment.css("a.author").text
      user = User.find_by(github_username: user_name) 
      unless user
        puts "Creating new user: #{user_name}"
        user = User.create(github_username: user_name)
      end
      comment_json = {}
      comment_json['user_id'] = user.id
      comment_json['github_created_at'] = raw_comment.css("a relative-time").attribute("datetime").value
      comment_json['body'] = raw_comment.css("td.comment-body").text
      comment_json
    end
def build_comment ( raw_comment ) user_name = raw_comment . css ( "a.author" ) . text user = User . find_by ( github_username : user_name ) unless user puts "Creating new user: #{user_name}" user = User . create ( github_username : user_name ) end comment_json = { } comment_json [ 'user_id' ] = user . id comment_json [ 'github_created_at' ] = raw_comment . css ( "a relative-time" ) . attribute ( "datetime" ) . value comment_json [ 'body' ] = raw_comment . css ( "td.comment-body" ) . text comment_json end
TODO: we should cache all the users for a repo when a repo is requested, so
we don't have to hit the DB as often, because I'll have to get the name
of the user from the comment, search if it exist, then create it.
TODO : we should cache all the users for a repo when a repo is requested so we don ' t have to hit the DB as often because I ' ll have to get the name of the user from the comment search if it exist then create it .
# TODO: use create! and if it fails, a rescue and find ?
raw_comment
<not_specific>