{"action":"created","issue":{"url":"https://api.github.com/repos/python-pillow/Pillow/issues/1934","repository_url":"https://api.github.com/repos/python-pillow/Pillow","labels_url":"https://api.github.com/repos/python-pillow/Pillow/issues/1934/labels{/name}","comments_url":"https://api.github.com/repos/python-pillow/Pillow/issues/1934/comments","events_url":"https://api.github.com/repos/python-pillow/Pillow/issues/1934/events","html_url":"https://github.com/python-pillow/Pillow/pull/1934","id":157408252,"number":1934,"title":"Rewrite of Jpeg2k data handling","user":{"login":"wiredfool","id":19792,"avatar_url":"https://avatars.githubusercontent.com/u/19792?v=3","gravatar_id":"","url":"https://api.github.com/users/wiredfool","html_url":"https://github.com/wiredfool","followers_url":"https://api.github.com/users/wiredfool/followers","following_url":"https://api.github.com/users/wiredfool/following{/other_user}","gists_url":"https://api.github.com/users/wiredfool/gists{/gist_id}","starred_url":"https://api.github.com/users/wiredfool/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/wiredfool/subscriptions","organizations_url":"https://api.github.com/users/wiredfool/orgs","repos_url":"https://api.github.com/users/wiredfool/repos","events_url":"https://api.github.com/users/wiredfool/events{/privacy}","received_events_url":"https://api.github.com/users/wiredfool/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/python-pillow/Pillow/labels/Do%20Not%20Merge","name":"Do Not Merge","color":"ff0000"},{"url":"https://api.github.com/repos/python-pillow/Pillow/labels/Needs%20Documentation","name":"Needs Documentation","color":"fad8c7"},{"url":"https://api.github.com/repos/python-pillow/Pillow/labels/Needs%20Review","name":"Needs Review","color":"eb6420"}],"state":"open","locked":false,"assignee":null,"milestone":{"url":"https://api.github.com/repos/python-pillow/Pillow/milestones/18","html_url":"https://github.com/python-pillow/Pillow/milestones/3.3.0","labels_url":"https://api.github.com/repos/python-pillow/Pillow/milestones/18/labels","id":1645347,"number":18,"title":"3.3.0","description":"","creator":{"login":"aclark4life","id":72164,"avatar_url":"https://avatars.githubusercontent.com/u/72164?v=3","gravatar_id":"","url":"https://api.github.com/users/aclark4life","html_url":"https://github.com/aclark4life","followers_url":"https://api.github.com/users/aclark4life/followers","following_url":"https://api.github.com/users/aclark4life/following{/other_user}","gists_url":"https://api.github.com/users/aclark4life/gists{/gist_id}","starred_url":"https://api.github.com/users/aclark4life/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/aclark4life/subscriptions","organizations_url":"https://api.github.com/users/aclark4life/orgs","repos_url":"https://api.github.com/users/aclark4life/repos","events_url":"https://api.github.com/users/aclark4life/events{/privacy}","received_events_url":"https://api.github.com/users/aclark4life/received_events","type":"User","site_admin":false},"open_issues":10,"closed_issues":4,"state":"open","created_at":"2016-03-15T11:22:27Z","updated_at":"2016-06-17T09:42:09Z","due_on":"2016-07-01T04:00:00Z","closed_at":null},"comments":2,"created_at":"2016-05-29T21:53:19Z","updated_at":"2016-06-21T11:07:20Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/python-pillow/Pillow/pulls/1934","html_url":"https://github.com/python-pillow/Pillow/pull/1934","diff_url":"https://github.com/python-pillow/Pillow/pull/1934.diff","patch_url":"https://github.com/python-pillow/Pillow/pull/1934.patch"},"body":"Fixes the intermittent test failures and occasional dos/thread deadlock in the J2k stack by rewriting the data handling to not use threading. \r\n\r\nChanges proposed in this pull request:\r\n\r\n * Adds a new interface/mode to the ImageFile where the C layer can write to the python file like object. \r\n * Rips out the threading in Incremental.c and converts J2k to use the new layer. \r\n\r\nRationale:\r\n\r\nIt is a relatively common pattern in image libraries to provide a set of read/seek/write functions to the underlying library so that the library can handle pulling/pushing data to a file like object. OpenJpeg does this for the J2k file format, and I worked up a way to do this in the tiff support as well. This additional interface in the C layer allows for a consistent implementation of this core bit of code. \r\n\r\nThe de/encoder needs to set pushes/pulls_fd =1 from within its initialization routine. If that flag is set, ImageFile.py will call `setfd(self.fd)` and then call the de/encoder routine. At that point, the de/encoder can point (or wrap) any read/seek/write functions at the `_imaging_*_pyfd` family of functions which call out to the python file like object. There will be one call to the de/encoder per tile, and the coder will handle any buffer sizes or shuffling within that call. This reduces the complexity of the decoder function, as there is no longer a requirement for multiple calls with additional data to be handled sequentially. \r\n\r\nToDo:\r\n- [x]  There's still cleanup of warnings and such. \r\n- [ ]  Make sure we like the interface and that we're returning the right results from the write/seek/read functions.  "},"comment":{"url":"https://api.github.com/repos/python-pillow/Pillow/issues/comments/227408685","html_url":"https://github.com/python-pillow/Pillow/pull/1934#issuecomment-227408685","issue_url":"https://api.github.com/repos/python-pillow/Pillow/issues/1934","id":227408685,"user":{"login":"wiredfool","id":19792,"avatar_url":"https://avatars.githubusercontent.com/u/19792?v=3","gravatar_id":"","url":"https://api.github.com/users/wiredfool","html_url":"https://github.com/wiredfool","followers_url":"https://api.github.com/users/wiredfool/followers","following_url":"https://api.github.com/users/wiredfool/following{/other_user}","gists_url":"https://api.github.com/users/wiredfool/gists{/gist_id}","starred_url":"https://api.github.com/users/wiredfool/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/wiredfool/subscriptions","organizations_url":"https://api.github.com/users/wiredfool/orgs","repos_url":"https://api.github.com/users/wiredfool/repos","events_url":"https://api.github.com/users/wiredfool/events{/privacy}","received_events_url":"https://api.github.com/users/wiredfool/received_events","type":"User","site_admin":false},"created_at":"2016-06-21T11:07:19Z","updated_at":"2016-06-21T11:07:19Z","body":"The api is tricky. I'm basing this on:\r\n\r\n- The proper place to define and set this up is in the PyImaging_*De/EncoderNew setup functions. \r\n- There's a different set of parameters that are required for the pyfd approach than the existing feeder (and the encode_to_file call)\r\n- The different parameters span both the ImageFile python layer and the lower decoder/encoder layer. \r\n- I don't really want to change the method signature of all of the PyImaging_*De/EncoderNew functions, as that would break anyone who has a non-contributed image de/encoder.\r\n\r\nThe encoder is simpler, as it's already got the `encode_to_file` method. Here I've added `encode_to_pyfd`.  To be cleaner, we could include the pyfd in as a parameter there rather than the flag + the `setfd` call. But we'd still effectively need a flag somewhere, either try the method and throw an exception if it's not appropriate or check the flag and use the method. \r\n\r\nI suppose we could split the `decode(bytes)` into `decode_from_pyfd(self.fp)`, and eliminate the `decoder.setfd` call and eliminate the empty bytestring that we're passing in.  But again, that doesn't eliminate the flag, it just streamlines the implementation. \r\n\r\n\r\n\r\n\r\n \r\n"}}
{"actor":{"display_login":"wiredfool"}}