{"action":"created","issue":{"url":"https://api.github.com/repos/chanks/que/issues/166","repository_url":"https://api.github.com/repos/chanks/que","labels_url":"https://api.github.com/repos/chanks/que/issues/166/labels{/name}","comments_url":"https://api.github.com/repos/chanks/que/issues/166/comments","events_url":"https://api.github.com/repos/chanks/que/issues/166/events","html_url":"https://github.com/chanks/que/issues/166","id":167416476,"number":166,"title":"Que jobs frequently repeat themselves","user":{"login":"bramski","id":421781,"avatar_url":"https://avatars.githubusercontent.com/u/421781?v=3","gravatar_id":"","url":"https://api.github.com/users/bramski","html_url":"https://github.com/bramski","followers_url":"https://api.github.com/users/bramski/followers","following_url":"https://api.github.com/users/bramski/following{/other_user}","gists_url":"https://api.github.com/users/bramski/gists{/gist_id}","starred_url":"https://api.github.com/users/bramski/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bramski/subscriptions","organizations_url":"https://api.github.com/users/bramski/orgs","repos_url":"https://api.github.com/users/bramski/repos","events_url":"https://api.github.com/users/bramski/events{/privacy}","received_events_url":"https://api.github.com/users/bramski/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":13,"created_at":"2016-07-25T17:05:42Z","updated_at":"2016-07-25T19:03:44Z","closed_at":null,"body":"Hello,\r\n\r\nWe're having a consistently repeatable problem over here of what we think is a fundamental problem of the MVCC advisory lock model in use here.\r\n\r\nI see this code, and it doesn't seem to be effective:\r\nhttps://github.com/chanks/que/blob/master/lib/que/job.rb#L88\r\n\r\nScreenshot of our logs:\r\n![as_a_user_i_want_lucy_to_deal_with_facebook_sending_the_same_message_twice_so_that_i_don_t_have_a_confused_conversation__-_lucy_-_confluence](https://cloud.githubusercontent.com/assets/421781/17109919/efd08cb6-524e-11e6-91eb-86a60ec528d0.png)\r\n\r\nOur Job Runner Looks like:\r\n```\r\nrequire 'que'\r\n\r\nmodule ActiveJob\r\n  module QueueAdapters\r\n    class ImhrJobRunner < ::Que::Job\r\n      # Places the que job in job_data[\"que_ref\"]\r\n      # which can be picked up by ApplicationJob\r\n      def run(job_data)\r\n        @job_data = job_data\r\n\r\n        if que_job_done? # we only need to check cause Que picks up destroyed jobs.\r\n          log_rerun_issue_info\r\n        else\r\n          job_data[\"que_ref\"] = self\r\n          ApplicationJob.execute job_data\r\n        end\r\n\r\n        mark_que_job_as_done # if you reach this point it's done\r\n      end\r\n\r\n      public :destroy\r\n\r\n      private\r\n\r\n      attr_reader :job_data\r\n\r\n      def log_rerun_issue_info\r\n        Rails.logger.warn \"#{self.class.name}: Que tried to rerun #{job_data[\"job_class\"]}: \" \\\r\n                          \"#{job_data[\"job_id\"]}\"\r\n      end\r\n\r\n      def que_job_done?\r\n        Rails.cache.exist?(que_job_cache_key)\r\n      end\r\n\r\n      def mark_que_job_as_done\r\n        Rails.cache.write(que_job_cache_key, true, expires_in: 10.seconds)\r\n      end\r\n\r\n      def que_job_cache_key\r\n        [\"que-job-done\", job_data[\"job_id\"]].join(\"-\")\r\n      end\r\n    end\r\n  end\r\nend\r\n```\r\n\r\nWe've added Rails cache which uses redis to ensure that the jobs aren't being run twice."},"comment":{"url":"https://api.github.com/repos/chanks/que/issues/comments/235051666","html_url":"https://github.com/chanks/que/issues/166#issuecomment-235051666","issue_url":"https://api.github.com/repos/chanks/que/issues/166","id":235051666,"user":{"login":"joevandyk","id":1179,"avatar_url":"https://avatars.githubusercontent.com/u/1179?v=3","gravatar_id":"","url":"https://api.github.com/users/joevandyk","html_url":"https://github.com/joevandyk","followers_url":"https://api.github.com/users/joevandyk/followers","following_url":"https://api.github.com/users/joevandyk/following{/other_user}","gists_url":"https://api.github.com/users/joevandyk/gists{/gist_id}","starred_url":"https://api.github.com/users/joevandyk/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/joevandyk/subscriptions","organizations_url":"https://api.github.com/users/joevandyk/orgs","repos_url":"https://api.github.com/users/joevandyk/repos","events_url":"https://api.github.com/users/joevandyk/events{/privacy}","received_events_url":"https://api.github.com/users/joevandyk/received_events","type":"User","site_admin":false},"created_at":"2016-07-25T19:03:44Z","updated_at":"2016-07-25T19:03:44Z","body":"Is there anything in your code/job that could be starting new database connections? Forking processes, etc? Or stopping processes?\r\n\r\nThat warning indicates that the advisory lock is being dropped and so it would make sense that Que would re-run the job. "}}
{"actor":{"display_login":"joevandyk"}}