{"action":"created","issue":{"url":"https://api.github.com/repos/nervous-systems/hildebrand/issues/12","repository_url":"https://api.github.com/repos/nervous-systems/hildebrand","labels_url":"https://api.github.com/repos/nervous-systems/hildebrand/issues/12/labels{/name}","comments_url":"https://api.github.com/repos/nervous-systems/hildebrand/issues/12/comments","events_url":"https://api.github.com/repos/nervous-systems/hildebrand/issues/12/events","html_url":"https://github.com/nervous-systems/hildebrand/issues/12","id":164792502,"number":12,"title":"crc32-mismatch for big requests","user":{"login":"robertvojta","id":1084172,"avatar_url":"https://avatars.githubusercontent.com/u/1084172?v=3","gravatar_id":"","url":"https://api.github.com/users/robertvojta","html_url":"https://github.com/robertvojta","followers_url":"https://api.github.com/users/robertvojta/followers","following_url":"https://api.github.com/users/robertvojta/following{/other_user}","gists_url":"https://api.github.com/users/robertvojta/gists{/gist_id}","starred_url":"https://api.github.com/users/robertvojta/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/robertvojta/subscriptions","organizations_url":"https://api.github.com/users/robertvojta/orgs","repos_url":"https://api.github.com/users/robertvojta/repos","events_url":"https://api.github.com/users/robertvojta/events{/privacy}","received_events_url":"https://api.github.com/users/robertvojta/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":16,"created_at":"2016-07-11T09:14:06Z","updated_at":"2016-07-12T09:12:53Z","closed_at":null,"body":"Just started tracking this issue, not sure where the problem is, but maybe anyone already saw it.\r\n\r\nI've got simple lambda function ...\r\n\r\n```\r\n(defn- get-participants [org survey]\r\n  (query!\r\n    (lambda/credentials)\r\n    :frank-participant\r\n    {:nyx-organization [:= org]\r\n     :id               [:begins-with (str survey \":\")]}\r\n    {:project [:nyx-organization :nyx-user :id :timestamp :employee :participant :answers]}\r\n    {:chan (async/chan 1024)}))\r\n\r\n(def ^:export list\r\n  (async-lambda-fn\r\n    (fn [ev ctx]\r\n      (go\r\n        (try\r\n          (if-let [_ (schema/check ParticipantListEventSchema ev)]\r\n            (lambda/bad-request! ctx)\r\n            (let [org (:nyx-organization ev)\r\n                  id (:survey ev)\r\n                  participants (get-participants org id)\r\n                  participants (<! (async/into [] participants))]\r\n              (if (some #(instance? js/Error %) participants)\r\n                (lambda/internal-server-error! ctx)\r\n                (lambda/succeed! ctx {:participants (participants-for-user participants \"admin\")}))))\r\n          (catch :default _\r\n            (lambda/internal-server-error! ctx)))))))\r\n```\r\n\r\n... this function does work perfectly for ~100 participants. But when I try to fetch 500 participants (roughly > 100), I've got crc32-mismatch for almost all requests. When I try this in Python, it does work like a charm even for 1000 participants.\r\n\r\nAn idea what can be wrong?"},"comment":{"url":"https://api.github.com/repos/nervous-systems/hildebrand/issues/comments/231982816","html_url":"https://github.com/nervous-systems/hildebrand/issues/12#issuecomment-231982816","issue_url":"https://api.github.com/repos/nervous-systems/hildebrand/issues/12","id":231982816,"user":{"login":"robertvojta","id":1084172,"avatar_url":"https://avatars.githubusercontent.com/u/1084172?v=3","gravatar_id":"","url":"https://api.github.com/users/robertvojta","html_url":"https://github.com/robertvojta","followers_url":"https://api.github.com/users/robertvojta/followers","following_url":"https://api.github.com/users/robertvojta/following{/other_user}","gists_url":"https://api.github.com/users/robertvojta/gists{/gist_id}","starred_url":"https://api.github.com/users/robertvojta/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/robertvojta/subscriptions","organizations_url":"https://api.github.com/users/robertvojta/orgs","repos_url":"https://api.github.com/users/robertvojta/repos","events_url":"https://api.github.com/users/robertvojta/events{/privacy}","received_events_url":"https://api.github.com/users/robertvojta/received_events","type":"User","site_admin":false},"created_at":"2016-07-12T09:12:53Z","updated_at":"2016-07-12T09:12:53Z","body":"Hmm, interesting ... There's undocummented (?) `gzip` option for node request. I tried to add it:\r\n\r\n```\r\n(defn req->node [{{:keys [query host port path]} :endpoint :keys [headers method] :as req}]\r\n  (cond->\r\n      {:host   host\r\n       :path   (cond-> path query (str \"?\" (url/map->query query)))\r\n       :method method\r\n       :headers headers\r\n       :gzip false}\r\n    port (assoc :port port)))\r\n```\r\n\r\nAnd good & bad news. With `:limit 256` (previously failing in 100% cases) and `:gzip false`, it fails in 50% of cases now and it succeeds in another 50% of cases. Which is interesting, some kind of progress, but not 100% reproducible, which is worse. Or it can be some DynamoDB noise today ..."}}
{"actor":{"display_login":"robertvojta"}}