{"action":"created","issue":{"url":"https://api.github.com/repos/OpenEnergyPlatform/oeplatform/issues/360","repository_url":"https://api.github.com/repos/OpenEnergyPlatform/oeplatform","labels_url":"https://api.github.com/repos/OpenEnergyPlatform/oeplatform/issues/360/labels{/name}","comments_url":"https://api.github.com/repos/OpenEnergyPlatform/oeplatform/issues/360/comments","events_url":"https://api.github.com/repos/OpenEnergyPlatform/oeplatform/issues/360/events","html_url":"https://github.com/OpenEnergyPlatform/oeplatform/issues/360","id":391817510,"node_id":"MDU6SXNzdWUzOTE4MTc1MTA=","number":360,"title":"Long upload times","user":{"login":"jnnr","id":32454596,"node_id":"MDQ6VXNlcjMyNDU0NTk2","avatar_url":"https://avatars1.githubusercontent.com/u/32454596?v=4","gravatar_id":"","url":"https://api.github.com/users/jnnr","html_url":"https://github.com/jnnr","followers_url":"https://api.github.com/users/jnnr/followers","following_url":"https://api.github.com/users/jnnr/following{/other_user}","gists_url":"https://api.github.com/users/jnnr/gists{/gist_id}","starred_url":"https://api.github.com/users/jnnr/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jnnr/subscriptions","organizations_url":"https://api.github.com/users/jnnr/orgs","repos_url":"https://api.github.com/users/jnnr/repos","events_url":"https://api.github.com/users/jnnr/events{/privacy}","received_events_url":"https://api.github.com/users/jnnr/received_events","type":"User","site_admin":false},"labels":[{"id":441551520,"node_id":"MDU6TGFiZWw0NDE1NTE1MjA=","url":"https://api.github.com/repos/OpenEnergyPlatform/oeplatform/labels/Feedback","name":"Feedback","color":"bfdadc","default":false},{"id":289428779,"node_id":"MDU6TGFiZWwyODk0Mjg3Nzk=","url":"https://api.github.com/repos/OpenEnergyPlatform/oeplatform/labels/help%20wanted","name":"help wanted","color":"159818","default":true}],"state":"open","locked":false,"assignee":{"login":"MGlauer","id":7835111,"node_id":"MDQ6VXNlcjc4MzUxMTE=","avatar_url":"https://avatars0.githubusercontent.com/u/7835111?v=4","gravatar_id":"","url":"https://api.github.com/users/MGlauer","html_url":"https://github.com/MGlauer","followers_url":"https://api.github.com/users/MGlauer/followers","following_url":"https://api.github.com/users/MGlauer/following{/other_user}","gists_url":"https://api.github.com/users/MGlauer/gists{/gist_id}","starred_url":"https://api.github.com/users/MGlauer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/MGlauer/subscriptions","organizations_url":"https://api.github.com/users/MGlauer/orgs","repos_url":"https://api.github.com/users/MGlauer/repos","events_url":"https://api.github.com/users/MGlauer/events{/privacy}","received_events_url":"https://api.github.com/users/MGlauer/received_events","type":"User","site_admin":false},"assignees":[{"login":"MGlauer","id":7835111,"node_id":"MDQ6VXNlcjc4MzUxMTE=","avatar_url":"https://avatars0.githubusercontent.com/u/7835111?v=4","gravatar_id":"","url":"https://api.github.com/users/MGlauer","html_url":"https://github.com/MGlauer","followers_url":"https://api.github.com/users/MGlauer/followers","following_url":"https://api.github.com/users/MGlauer/following{/other_user}","gists_url":"https://api.github.com/users/MGlauer/gists{/gist_id}","starred_url":"https://api.github.com/users/MGlauer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/MGlauer/subscriptions","organizations_url":"https://api.github.com/users/MGlauer/orgs","repos_url":"https://api.github.com/users/MGlauer/repos","events_url":"https://api.github.com/users/MGlauer/events{/privacy}","received_events_url":"https://api.github.com/users/MGlauer/received_events","type":"User","site_admin":false},{"login":"oakca","id":17292790,"node_id":"MDQ6VXNlcjE3MjkyNzkw","avatar_url":"https://avatars1.githubusercontent.com/u/17292790?v=4","gravatar_id":"","url":"https://api.github.com/users/oakca","html_url":"https://github.com/oakca","followers_url":"https://api.github.com/users/oakca/followers","following_url":"https://api.github.com/users/oakca/following{/other_user}","gists_url":"https://api.github.com/users/oakca/gists{/gist_id}","starred_url":"https://api.github.com/users/oakca/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/oakca/subscriptions","organizations_url":"https://api.github.com/users/oakca/orgs","repos_url":"https://api.github.com/users/oakca/repos","events_url":"https://api.github.com/users/oakca/events{/privacy}","received_events_url":"https://api.github.com/users/oakca/received_events","type":"User","site_admin":false},{"login":"christian-rli","id":42000429,"node_id":"MDQ6VXNlcjQyMDAwNDI5","avatar_url":"https://avatars3.githubusercontent.com/u/42000429?v=4","gravatar_id":"","url":"https://api.github.com/users/christian-rli","html_url":"https://github.com/christian-rli","followers_url":"https://api.github.com/users/christian-rli/followers","following_url":"https://api.github.com/users/christian-rli/following{/other_user}","gists_url":"https://api.github.com/users/christian-rli/gists{/gist_id}","starred_url":"https://api.github.com/users/christian-rli/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/christian-rli/subscriptions","organizations_url":"https://api.github.com/users/christian-rli/orgs","repos_url":"https://api.github.com/users/christian-rli/repos","events_url":"https://api.github.com/users/christian-rli/events{/privacy}","received_events_url":"https://api.github.com/users/christian-rli/received_events","type":"User","site_admin":false}],"milestone":null,"comments":0,"created_at":"2018-12-17T17:34:03Z","updated_at":"2018-12-17T17:53:07Z","closed_at":null,"author_association":"NONE","body":"Uploading a timeseries of 8760 entries (380kB) takes 4049.0355615615845 sec. which is more than an hour. \r\nDownload, on the other side, is done within milliseconds.\r\n* is this normal?\r\n* can it be accelerated?\r\n\r\nThis is the function I execute, if that helps. \r\n\r\n> \r\n    def upload_to_oep(df, Table, engine, metadata):\r\n        table_name = Table.name\r\n        schema_name = Table.schema\r\n    \r\n        if not engine.dialect.has_table(engine, table_name, schema_name):\r\n            Table.create()\r\n            print('Created table')\r\n        else:\r\n            print('Table already exists')\r\n    \r\n        Session = sessionmaker(bind=engine)\r\n        session = Session()\r\n        # insert data\r\n        try:\r\n            dtype = {key: Table.columns[key].type for key in Table.columns.keys()}\r\n            df.to_sql(table_name, engine,\r\n                              schema='sandbox',\r\n                              if_exists='replace',\r\n                              dtype=dtype)\r\n            print('Inserted to ' + table_name)\r\n        except Exception as e:\r\n            session.rollback()\r\n            session.close()\r\n            raise\r\n            print('Insert incomplete!')\r\n        finally:\r\n            session.close()\r\n    \r\n        return Table"},"comment":{"url":"https://api.github.com/repos/OpenEnergyPlatform/oeplatform/issues/comments/447936491","html_url":"https://github.com/OpenEnergyPlatform/oeplatform/issues/360#issuecomment-447936491","issue_url":"https://api.github.com/repos/OpenEnergyPlatform/oeplatform/issues/360","id":447936491,"node_id":"MDEyOklzc3VlQ29tbWVudDQ0NzkzNjQ5MQ==","user":{"login":"oakca","id":17292790,"node_id":"MDQ6VXNlcjE3MjkyNzkw","avatar_url":"https://avatars1.githubusercontent.com/u/17292790?v=4","gravatar_id":"","url":"https://api.github.com/users/oakca","html_url":"https://github.com/oakca","followers_url":"https://api.github.com/users/oakca/followers","following_url":"https://api.github.com/users/oakca/following{/other_user}","gists_url":"https://api.github.com/users/oakca/gists{/gist_id}","starred_url":"https://api.github.com/users/oakca/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/oakca/subscriptions","organizations_url":"https://api.github.com/users/oakca/orgs","repos_url":"https://api.github.com/users/oakca/repos","events_url":"https://api.github.com/users/oakca/events{/privacy}","received_events_url":"https://api.github.com/users/oakca/received_events","type":"User","site_admin":false},"created_at":"2018-12-17T17:53:07Z","updated_at":"2018-12-17T17:53:07Z","author_association":"NONE","body":"I would first check, how `to_sql` works. But even it functions with a for-loop (setting values 1-by-1), it would be 8760*2(entries and primary key) entries, and still no reason to take 4k+ seconds."}}
{"actor":{"display_login":"oakca"}}