{"action":"created","issue":{"url":"https://api.github.com/repos/openmicroscopy/bioformats/issues/2308","repository_url":"https://api.github.com/repos/openmicroscopy/bioformats","labels_url":"https://api.github.com/repos/openmicroscopy/bioformats/issues/2308/labels{/name}","comments_url":"https://api.github.com/repos/openmicroscopy/bioformats/issues/2308/comments","events_url":"https://api.github.com/repos/openmicroscopy/bioformats/issues/2308/events","html_url":"https://github.com/openmicroscopy/bioformats/issues/2308","id":142793355,"number":2308,"title":"[bug] Micromanager: reader crashes with stack format and large datasets","user":{"login":"julou","id":615747,"avatar_url":"https://avatars.githubusercontent.com/u/615747?v=3","gravatar_id":"","url":"https://api.github.com/users/julou","html_url":"https://github.com/julou","followers_url":"https://api.github.com/users/julou/followers","following_url":"https://api.github.com/users/julou/following{/other_user}","gists_url":"https://api.github.com/users/julou/gists{/gist_id}","starred_url":"https://api.github.com/users/julou/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/julou/subscriptions","organizations_url":"https://api.github.com/users/julou/orgs","repos_url":"https://api.github.com/users/julou/repos","events_url":"https://api.github.com/users/julou/events{/privacy}","received_events_url":"https://api.github.com/users/julou/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":9,"created_at":"2016-03-22T21:59:46Z","updated_at":"2016-04-06T14:44:58Z","closed_at":null,"body":"Hello,\r\n\r\nFollowing [your fixes to parse config-specific metadata in MM datasets](https://github.com/openmicroscopy/bioformats/pull/2226), I realised that I focused on providing small datasets (for obvious practical reasons). Hence the current MM reader (5.1.8) crashes when the dataset is split over several tiff files.\r\n\r\nIn more details:\r\n- when MM datasets are stored in stack format and the dataset size exceeds 4.7gb, it's split over several files (e.g. prefix_1_MMStack.ome.tif, prefix_1_MMStack_1.ome.tif, etc)\r\n- because MM can save one tif file (or set of tif files if > 4.7gb) per position, it is sometimes useful to use BF to open PosXX (in fact it would be my primary use case!)\r\n- with BF 5.1.7, the following readers were used depending on the input path:\r\n    - first tif file of the dataset: OME_TIF reader\r\n    - any other tif file: TIF reader\r\n- with current BF 5.1.8, the following readers are used depending on the input path:\r\n    - first tif file of the dataset: OME_TIF reader\r\n    - any other tif file with associated *_metadata.txt file: MM reader (OK provided the position dataset isn't >4.7gb)\r\n    - any other tif file (without *_metadata.txt file): TIF reader\r\n\r\nI would be happy to provide you with a demo large dataset, or you can easily create one using MM's demo config and the Multi Dimensional Acquisition window (select e.g. enough time points so that the dataset size is >4.7gb).\r\n\r\nAlso, as far as I understand it, it means that there is no easy way to open the first position of a dataset (because when this path is given as input, the OME-TIF reader is triggered which will load the full dataset). Although I don't understand how the reader selection works (but that you want to keep a way to trigger the OME-TIF reader for such datasets), I find this very cumbersome! What would be the other possible designs?\r\n\r\nThank you in advance for your attention. Best,\r\nThomas"},"comment":{"url":"https://api.github.com/repos/openmicroscopy/bioformats/issues/comments/206410214","html_url":"https://github.com/openmicroscopy/bioformats/issues/2308#issuecomment-206410214","issue_url":"https://api.github.com/repos/openmicroscopy/bioformats/issues/2308","id":206410214,"user":{"login":"sbesson","id":1355463,"avatar_url":"https://avatars.githubusercontent.com/u/1355463?v=3","gravatar_id":"","url":"https://api.github.com/users/sbesson","html_url":"https://github.com/sbesson","followers_url":"https://api.github.com/users/sbesson/followers","following_url":"https://api.github.com/users/sbesson/following{/other_user}","gists_url":"https://api.github.com/users/sbesson/gists{/gist_id}","starred_url":"https://api.github.com/users/sbesson/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sbesson/subscriptions","organizations_url":"https://api.github.com/users/sbesson/orgs","repos_url":"https://api.github.com/users/sbesson/repos","events_url":"https://api.github.com/users/sbesson/events{/privacy}","received_events_url":"https://api.github.com/users/sbesson/received_events","type":"User","site_admin":false},"created_at":"2016-04-06T14:44:58Z","updated_at":"2016-04-06T14:44:58Z","body":"Hi,\r\n\r\nanswering some questions unrelated to the discussion opened in https://github.com/openmicroscopy/bioformats/issues/2309.\r\n\r\nIt looks that the MicromanagerReader will  initialize the files in one position only in the case of multiple positions, independently of the input file:\r\n\r\n```\r\n$  showinf -nopix -nometa Test_Dataset_MultiPosition_Stack_2_MMStack_Pos0_metadata.txt \r\nChecking file format [Micro-Manager]\r\nInitializing reader\r\nMicromanagerReader initializing Test_Dataset_MultiPosition_Stack_2_MMStack_Pos0_metadata.txt\r\nReading metadata file\r\nPopulating metadata\r\nFinding image file names\r\nBuilding list of TIFFs\r\nInitialization took 0.394s\r\n\r\nReading core metadata\r\nfilename = Test_Dataset_MultiPosition_Stack_2_MMStack_Pos0_metadata.txt\r\nUsed files:\r\n\t/Users/sbesson/Desktop/MM1_4_22/Test_Dataset_MultiPosition_Stack_2/Test_Dataset_MultiPosition_Stack_2_MMStack_Pos0_metadata.txt\r\n\t/Users/sbesson/Desktop/MM1_4_22/Test_Dataset_MultiPosition_Stack_2/Test_Dataset_MultiPosition_Stack_2_MMStack_Pos0.ome.tif\r\nSeries count = 1\r\nSeries #0 :\r\n\tImage count = 126\r\n\tRGB = false (1) \r\n\tInterleaved = false\r\n\tIndexed = false (true color)\r\n\tWidth = 512\r\n\tHeight = 512\r\n\tSizeZ = 21\r\n\tSizeT = 2\r\n\tSizeC = 3\r\n\tThumbnail size = 128 x 128\r\n\tEndianness = intel (little)\r\n\tDimension order = XYZCT (uncertain)\r\n\tPixel type = uint16\r\n\tValid bits per pixel = 16\r\n\tMetadata complete = true\r\n\tThumbnail series = false\r\nsbesson@ls30630:Test_Dataset_MultiPosition_Stack_2 $  showinf -nopix -nometa Test_Dataset_MultiPosition_Stack_2_MMStack_Pos1_metadata.txt \r\nChecking file format [Micro-Manager]\r\nInitializing reader\r\nMicromanagerReader initializing Test_Dataset_MultiPosition_Stack_2_MMStack_Pos1_metadata.txt\r\nReading metadata file\r\nPopulating metadata\r\nFinding image file names\r\nBuilding list of TIFFs\r\nInitialization took 0.387s\r\n\r\nReading core metadata\r\nfilename = Test_Dataset_MultiPosition_Stack_2_MMStack_Pos1_metadata.txt\r\nUsed files:\r\n\t/Users/sbesson/Desktop/MM1_4_22/Test_Dataset_MultiPosition_Stack_2/Test_Dataset_MultiPosition_Stack_2_MMStack_Pos1_metadata.txt\r\n\t/Users/sbesson/Desktop/MM1_4_22/Test_Dataset_MultiPosition_Stack_2/Test_Dataset_MultiPosition_Stack_2_MMStack_Pos1.ome.tif\r\nSeries count = 1\r\nSeries #0 :\r\n\tImage count = 126\r\n\tRGB = false (1) \r\n\tInterleaved = false\r\n\tIndexed = false (true color)\r\n\tWidth = 512\r\n\tHeight = 512\r\n\tSizeZ = 21\r\n\tSizeT = 2\r\n\tSizeC = 3\r\n\tThumbnail size = 128 x 128\r\n\tEndianness = intel (little)\r\n\tDimension order = XYZCT (uncertain)\r\n\tPixel type = uint16\r\n\tValid bits per pixel = 16\r\n\tMetadata complete = true\r\n\tThumbnail series = false\r\nsbesson@ls30630:Test_Dataset_MultiPosition_Stack_2 $ showinf -nopix -nometa  -format Micromanager Test_Dataset_MultiPosition_Stack_2_MMStack_Pos1.ome.tif \r\nChecking Micro-Manager format [yes]\r\nInitializing reader\r\nReading metadata file\r\nPopulating metadata\r\nFinding image file names\r\nBuilding list of TIFFs\r\nInitialization took 0.402s\r\n\r\nReading core metadata\r\nfilename = Test_Dataset_MultiPosition_Stack_2_MMStack_Pos1.ome.tif\r\nUsed files:\r\n\t/Users/sbesson/Desktop/MM1_4_22/Test_Dataset_MultiPosition_Stack_2/Test_Dataset_MultiPosition_Stack_2_MMStack_Pos1_metadata.txt\r\n\t/Users/sbesson/Desktop/MM1_4_22/Test_Dataset_MultiPosition_Stack_2/Test_Dataset_MultiPosition_Stack_2_MMStack_Pos1.ome.tif\r\nSeries count = 1\r\nSeries #0 :\r\n\tImage count = 126\r\n\tRGB = false (1) \r\n\tInterleaved = false\r\n\tIndexed = false (true color)\r\n\tWidth = 512\r\n\tHeight = 512\r\n\tSizeZ = 21\r\n\tSizeT = 2\r\n\tSizeC = 3\r\n\tThumbnail size = 128 x 128\r\n\tEndianness = intel (little)\r\n\tDimension order = XYZCT (uncertain)\r\n\tPixel type = uint16\r\n\tValid bits per pixel = 16\r\n\tMetadata complete = true\r\n\tThumbnail series = false\r\n```\r\n\r\nAs mentioned above, the bug for the large files split into several TIFF files has been captured and is definitely on our roadmap for 5.2.0. As explained in our [Bio-Formats status blog post](http://blog.openmicroscopy.org/file-formats/data-model/future-plans/2016/01/26/bf-model-status/), most of our resources are currently focused on the model changes with bug fixing being currently lower priority.\r\n\r\nRegarding your error when forcing the usage of the OMETiffReader, do you have a stacktrace? It might well be fixed by https://github.com/openmicroscopy/bioformats/pull/2320 which will be released as part of the upcoming Bio-Formats 5.1.9.\r\n\r\nBest,\r\nSebastien"}}