{"action":"closed","issue":{"url":"https://api.github.com/repos/CartoDB/cartodb/issues/1691","repository_url":"https://api.github.com/repos/CartoDB/cartodb","labels_url":"https://api.github.com/repos/CartoDB/cartodb/issues/1691/labels{/name}","comments_url":"https://api.github.com/repos/CartoDB/cartodb/issues/1691/comments","events_url":"https://api.github.com/repos/CartoDB/cartodb/issues/1691/events","html_url":"https://github.com/CartoDB/cartodb/issues/1691","id":53859813,"number":1691,"title":"Transactions taking too long upon import","user":{"login":"Kartones","id":2085449,"avatar_url":"https://avatars.githubusercontent.com/u/2085449?v=3","gravatar_id":"","url":"https://api.github.com/users/Kartones","html_url":"https://github.com/Kartones","followers_url":"https://api.github.com/users/Kartones/followers","following_url":"https://api.github.com/users/Kartones/following{/other_user}","gists_url":"https://api.github.com/users/Kartones/gists{/gist_id}","starred_url":"https://api.github.com/users/Kartones/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Kartones/subscriptions","organizations_url":"https://api.github.com/users/Kartones/orgs","repos_url":"https://api.github.com/users/Kartones/repos","events_url":"https://api.github.com/users/Kartones/events{/privacy}","received_events_url":"https://api.github.com/users/Kartones/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/CartoDB/cartodb/labels/0%20-%20Backlog","name":"0 - Backlog","color":"CCCCCC"},{"url":"https://api.github.com/repos/CartoDB/cartodb/labels/importer","name":"importer","color":"eb6420"},{"url":"https://api.github.com/repos/CartoDB/cartodb/labels/robustness","name":"robustness","color":"000000"}],"state":"closed","locked":false,"assignee":{"login":"luisbosque","id":93268,"avatar_url":"https://avatars.githubusercontent.com/u/93268?v=3","gravatar_id":"","url":"https://api.github.com/users/luisbosque","html_url":"https://github.com/luisbosque","followers_url":"https://api.github.com/users/luisbosque/followers","following_url":"https://api.github.com/users/luisbosque/following{/other_user}","gists_url":"https://api.github.com/users/luisbosque/gists{/gist_id}","starred_url":"https://api.github.com/users/luisbosque/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/luisbosque/subscriptions","organizations_url":"https://api.github.com/users/luisbosque/orgs","repos_url":"https://api.github.com/users/luisbosque/repos","events_url":"https://api.github.com/users/luisbosque/events{/privacy}","received_events_url":"https://api.github.com/users/luisbosque/received_events","type":"User","site_admin":false},"milestone":null,"comments":0,"created_at":"2015-01-09T11:44:02Z","updated_at":"2016-05-25T14:54:18Z","closed_at":"2016-05-25T14:54:18Z","body":"Big twitter imports were giving timeouts. Upon closer inspection, a search retrieving ~365k rows was making transactions take more than 600 secs with quite small chunks of 1K rows:\r\n![](http://comomonos.com/up/long_transactions.jpg)\r\n\r\n(this kept happening until we killed the process)\r\n\r\nThe full process itself is heavy, as it:\r\n- creates a temporal geometry column\r\n- \"moves\" the_geom to it, cleans the_geom\r\n- \"moves back\" to the_geom data if they either are points (setting SRID to them) or calculating a random point inside bounding box if is a \"not so big\" polygon\r\n- deletes temp column\r\n\r\nAlso some operations batched so they add a temporal column to work in batches and not timeout by using full table data.\r\n\r\nThis didn't happened before, so either something changed in the underlying code (and at least I didn't touched anything) or something is causing that huge retention of transactions.\r\n"}}
{"actor":{"display_login":"saleiva"}}