{"action":"created","issue":{"url":"https://api.github.com/repos/Pylons/pyramid_tm/issues/53","repository_url":"https://api.github.com/repos/Pylons/pyramid_tm","labels_url":"https://api.github.com/repos/Pylons/pyramid_tm/issues/53/labels{/name}","comments_url":"https://api.github.com/repos/Pylons/pyramid_tm/issues/53/comments","events_url":"https://api.github.com/repos/Pylons/pyramid_tm/issues/53/events","html_url":"https://github.com/Pylons/pyramid_tm/issues/53","id":192288186,"number":53,"title":"For no obvious reason pyramid_tm rolls back the transaction. ","user":{"login":"bemineni","id":1664905,"avatar_url":"https://avatars.githubusercontent.com/u/1664905?v=3","gravatar_id":"","url":"https://api.github.com/users/bemineni","html_url":"https://github.com/bemineni","followers_url":"https://api.github.com/users/bemineni/followers","following_url":"https://api.github.com/users/bemineni/following{/other_user}","gists_url":"https://api.github.com/users/bemineni/gists{/gist_id}","starred_url":"https://api.github.com/users/bemineni/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bemineni/subscriptions","organizations_url":"https://api.github.com/users/bemineni/orgs","repos_url":"https://api.github.com/users/bemineni/repos","events_url":"https://api.github.com/users/bemineni/events{/privacy}","received_events_url":"https://api.github.com/users/bemineni/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2016-11-29T13:56:30Z","updated_at":"2016-11-29T16:50:42Z","closed_at":null,"body":"I am using a recipe as suggested by Mike bayer https://bitbucket.org/zzzeek/sqlalchemy/wiki/UsageRecipes/SessionModifiedSQL. To  create and update sharded tables in the same database. His recipe works absolutely fine with sessions, when the same is added to a scoped session managed by pyramid_tm transactin manager, then it just rolls back the transaction at the end of request for no reason.\r\n\r\n```\r\n*view.py*\r\n\r\nwith session_shardid(DBSession,table_hash):\t\r\n\t\t\tBase.metadata.create_all(DBSession.connection())\r\n                        self.request.tm.commit()\r\n\r\n*model.py*\r\nDBSession = scoped_session(sessionmaker(extension=ZopeTransactionExtension()))\r\n\r\n@listens_for(DBSession, \"after_begin\")\r\ndef _connection_for_session(session, trans, connection):\r\n    \"\"\"Share the 'info' dictionary of Session with Connection\r\n    objects.\r\n    This occurs as new Connection objects are associated with the\r\n    Session.   The .info dictionary on Connection is local to the\r\n    DBAPI connection.\r\n    \"\"\"\r\n    connection.info['session_info'] = session.info\r\n\r\n@contextmanager\r\ndef session_shardid(session, shardid):\r\n    \"\"\"Apply the \"shard\" id to all SQL emitted by the given Session.\r\n    \"\"\"\r\n    session.info[\"shard\"] = shardid\r\n    yield\r\n    del session.info[\"shard\"]\r\n\r\n@listens_for(Engine, \"before_cursor_execute\", retval=True)\r\ndef _apply_shard_id(connection, cursor, statement, parameters,\r\n                                        context, executemany):\r\n    \"\"\"Apply a \"shard id\" to statements.\r\n\r\n    Similar to the comment listener, we alter the statement on the\r\n    fly replacing occurrences of \"_shard_\" with the current \"shard id\".\r\n\r\n    \"\"\"\r\n    session_info = connection.info.get('session_info', {})\r\n    if \"shard\" in session_info:\r\n        statement = statement.replace(\"_shardid_\", session_info[\"shard\"])\r\n        for param,value in parameters.items():\r\n        \tif isinstance(value, str):\r\n        \t\tparameters[param] = value.replace(\"_shardid_\",session_info[\"shard\"])\r\n\r\n    return statement, parameters\r\n\r\n\r\n\r\n2016-11-29 07:40:08,279 INFO  [sqlalchemy.engine.base.Engine:1145][b'uWSGIWorker2Core0'] {}\r\nInside _apply_shard_id\r\n{'shard': 'ba6bd213'}\r\n2016-11-29 07:40:08,298 INFO  [sqlalchemy.engine.base.Engine:1142][b'uWSGIWorker2Core0'] CREATE INDEX group_topic_ba6bd213_index1 ON group_topic_ba6bd213 (group_id, user_id)\r\n2016-11-29 07:40:08,298 INFO  [sqlalchemy.engine.base.Engine:1145][b'uWSGIWorker2Core0'] {}\r\n2016-11-29 07:40:08,300 INFO  [sqlalchemy.engine.base.Engine:701][b'uWSGIWorker2Core0'] ROLLBACK\r\n[pid: 42827|app: 0|req: 1/1] 127.0.0.1 () {40 vars in 696 bytes} [Tue Nov 29 07:40:08 2016] GET /group_debug => generated 14 bytes in 99 msecs (HTTP/1.1 200) 2 headers in 79 bytes (1 switches on core 0)\r\n```\r\n\r\nThe stack trace\r\n\r\n````\r\nTraceback (most recent call last):\r\n\r\n  File \"/home/izero/devel/medics_city/medics_city_env/lib/python3.4/site-packages/sqlalchemy/pool.py\", line 687, in _finalize_fairy\r\n    fairy._reset(pool)\r\n  File \"/home/izero/devel/medics_city/medics_city_env/lib/python3.4/site-packages/sqlalchemy/pool.py\", line 827, in _reset\r\n    self._reset_agent.rollback()\r\n  File \"/home/izero/devel/medics_city/medics_city_env/lib/python3.4/site-packages/sqlalchemy/engine/base.py\", line 1614, in rollback\r\n    self._do_rollback()\r\n  File \"/home/izero/devel/medics_city/medics_city_env/lib/python3.4/site-packages/sqlalchemy/engine/base.py\", line 1652, in _do_rollback\r\n    self.connection._rollback_impl()\r\n  File \"/home/izero/devel/medics_city/medics_city_env/lib/python3.4/site-packages/sqlalchemy/engine/base.py\", line 694, in _rollback_impl\r\n    tb = traceback.format_exc()\r\n\r\nsqlalchemy/pool.py\r\ndef _rollback_impl(self):\r\n        assert not self.__branch_from\r\n        import traceback\r\n        tb = traceback.format_exc()\r\n\r\n\r\n        if self._has_events or self.engine._has_events:\r\n            self.dispatch.rollback(self)\r\n\r\n\r\n        **if self._still_open_and_connection_is_valid:**\r\n            if self._echo:\r\n                self.engine.logger.info(\"ROLLBACK\")\r\n            try:\r\n                self.engine.dialect.do_rollback(self.connection)\r\n            except BaseException as e:\r\n                self._handle_dbapi_exception(e, None, None, None, None)\r\n            finally:\r\n                if not self.__invalid and \\\r\n                        self.connection._reset_agent is self.__transaction:\r\n                    self.connection._reset_agent = None\r\n                self.__transaction = None\r\n        else:\r\n            self.__transaction = None\r\n```"},"comment":{"url":"https://api.github.com/repos/Pylons/pyramid_tm/issues/comments/263627419","html_url":"https://github.com/Pylons/pyramid_tm/issues/53#issuecomment-263627419","issue_url":"https://api.github.com/repos/Pylons/pyramid_tm/issues/53","id":263627419,"user":{"login":"mmerickel","id":487237,"avatar_url":"https://avatars.githubusercontent.com/u/487237?v=3","gravatar_id":"","url":"https://api.github.com/users/mmerickel","html_url":"https://github.com/mmerickel","followers_url":"https://api.github.com/users/mmerickel/followers","following_url":"https://api.github.com/users/mmerickel/following{/other_user}","gists_url":"https://api.github.com/users/mmerickel/gists{/gist_id}","starred_url":"https://api.github.com/users/mmerickel/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mmerickel/subscriptions","organizations_url":"https://api.github.com/users/mmerickel/orgs","repos_url":"https://api.github.com/users/mmerickel/repos","events_url":"https://api.github.com/users/mmerickel/events{/privacy}","received_events_url":"https://api.github.com/users/mmerickel/received_events","type":"User","site_admin":false},"created_at":"2016-11-29T16:50:42Z","updated_at":"2016-11-29T16:50:42Z","body":"The ZTE only tracks changes done by the ORM itself. If you are doing changes via `session.execute`, etc then the ZTE will not detect them. You need to use `zope.sqlalchemy.mark_changed(session)` to tell it that it changed. Alternatively you can initialize the ZTE into the `changed` state by default so it will always try to do a commit."}}
{"actor":{"display_login":"mmerickel"}}