{"action":"created","issue":{"url":"https://api.github.com/repos/openmicroscopy/bioformats/issues/2227","repository_url":"https://api.github.com/repos/openmicroscopy/bioformats","labels_url":"https://api.github.com/repos/openmicroscopy/bioformats/issues/2227/labels{/name}","comments_url":"https://api.github.com/repos/openmicroscopy/bioformats/issues/2227/comments","events_url":"https://api.github.com/repos/openmicroscopy/bioformats/issues/2227/events","html_url":"https://github.com/openmicroscopy/bioformats/pull/2227","id":131087564,"number":2227,"title":"getUsedFiles performance fix","user":{"login":"simleo","id":1183171,"avatar_url":"https://avatars.githubusercontent.com/u/1183171?v=3","gravatar_id":"","url":"https://api.github.com/users/simleo","html_url":"https://github.com/simleo","followers_url":"https://api.github.com/users/simleo/followers","following_url":"https://api.github.com/users/simleo/following{/other_user}","gists_url":"https://api.github.com/users/simleo/gists{/gist_id}","starred_url":"https://api.github.com/users/simleo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/simleo/subscriptions","organizations_url":"https://api.github.com/users/simleo/orgs","repos_url":"https://api.github.com/users/simleo/repos","events_url":"https://api.github.com/users/simleo/events{/privacy}","received_events_url":"https://api.github.com/users/simleo/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/openmicroscopy/bioformats/labels/metadata","name":"metadata","color":"069856"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2016-02-03T16:59:45Z","updated_at":"2016-04-25T20:32:48Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/openmicroscopy/bioformats/pulls/2227","html_url":"https://github.com/openmicroscopy/bioformats/pull/2227","diff_url":"https://github.com/openmicroscopy/bioformats/pull/2227.diff","patch_url":"https://github.com/openmicroscopy/bioformats/pull/2227.patch"},"body":"This PR addresses a performance issue in `FormatReader.getUsedFiles` that can get very serious for formats with a large number of files in multiple series (e.g., `ScreenReader`).\r\n\r\nThe current implementation iteratively updates a `Vector` with the contents of arrays returned from `getSeriesUsedFiles`. To avoid duplicates, each filename from `getSeriesUsedFiles` is checked to see if it already belongs to the vector before adding. The problem is that `Vector.contains` is O(n), which makes the whole update operation **quadratic**.\r\n\r\nA previous attempt at fixing the problem (see #1028) only made things better for single-series images: updates for subsequent series are still performed in the same way.\r\n\r\nOn my laptop, `ScreenReader.getUsedFiles` on [a TARA screen file](https://github.com/IDR/idr-metadata/blob/master/idr0015-UNKNOWN-taraoceans/screenA/patterns/TARA_HCS1_H5_G100001472_G100001473--2013_09_28_19_45_25_chamber--U01--V01.screen) takes ~60 seconds without the patch and ~3 seconds with the patch. Note that I've replaced `.ome.tif` entries with `.fake` ones to avoid copying real data.\r\n\r\nThis problem is also present in develop. I'm opening the PR vs metadata first because `ScreenReader`  allows to better evaluate the performance improvement.\r\n\r\nNOTE: I have used `LinkedHashSet` instead of `HashSet` for predictable iteration order."},"comment":{"url":"https://api.github.com/repos/openmicroscopy/bioformats/issues/comments/214511825","html_url":"https://github.com/openmicroscopy/bioformats/pull/2227#issuecomment-214511825","issue_url":"https://api.github.com/repos/openmicroscopy/bioformats/issues/2227","id":214511825,"user":{"login":"joshmoore","id":88113,"avatar_url":"https://avatars.githubusercontent.com/u/88113?v=3","gravatar_id":"","url":"https://api.github.com/users/joshmoore","html_url":"https://github.com/joshmoore","followers_url":"https://api.github.com/users/joshmoore/followers","following_url":"https://api.github.com/users/joshmoore/following{/other_user}","gists_url":"https://api.github.com/users/joshmoore/gists{/gist_id}","starred_url":"https://api.github.com/users/joshmoore/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/joshmoore/subscriptions","organizations_url":"https://api.github.com/users/joshmoore/orgs","repos_url":"https://api.github.com/users/joshmoore/repos","events_url":"https://api.github.com/users/joshmoore/events{/privacy}","received_events_url":"https://api.github.com/users/joshmoore/received_events","type":"User","site_admin":false},"created_at":"2016-04-25T20:32:48Z","updated_at":"2016-04-25T20:32:48Z","body":"Deployed to the idrstaging server as build 66. Merging to be tagged for 0.0.4."}}