{"action":"created","issue":{"url":"https://api.github.com/repos/hydroshare/hydroshare/issues/1034","repository_url":"https://api.github.com/repos/hydroshare/hydroshare","labels_url":"https://api.github.com/repos/hydroshare/hydroshare/issues/1034/labels{/name}","comments_url":"https://api.github.com/repos/hydroshare/hydroshare/issues/1034/comments","events_url":"https://api.github.com/repos/hydroshare/hydroshare/issues/1034/events","html_url":"https://github.com/hydroshare/hydroshare/issues/1034","id":139133523,"number":1034,"title":"File metadata for resources","user":{"login":"alvacouch","id":3342292,"avatar_url":"https://avatars.githubusercontent.com/u/3342292?v=3","gravatar_id":"","url":"https://api.github.com/users/alvacouch","html_url":"https://github.com/alvacouch","followers_url":"https://api.github.com/users/alvacouch/followers","following_url":"https://api.github.com/users/alvacouch/following{/other_user}","gists_url":"https://api.github.com/users/alvacouch/gists{/gist_id}","starred_url":"https://api.github.com/users/alvacouch/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/alvacouch/subscriptions","organizations_url":"https://api.github.com/users/alvacouch/orgs","repos_url":"https://api.github.com/users/alvacouch/repos","events_url":"https://api.github.com/users/alvacouch/events{/privacy}","received_events_url":"https://api.github.com/users/alvacouch/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":{"login":"pkdash","id":2523434,"avatar_url":"https://avatars.githubusercontent.com/u/2523434?v=3","gravatar_id":"","url":"https://api.github.com/users/pkdash","html_url":"https://github.com/pkdash","followers_url":"https://api.github.com/users/pkdash/followers","following_url":"https://api.github.com/users/pkdash/following{/other_user}","gists_url":"https://api.github.com/users/pkdash/gists{/gist_id}","starred_url":"https://api.github.com/users/pkdash/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pkdash/subscriptions","organizations_url":"https://api.github.com/users/pkdash/orgs","repos_url":"https://api.github.com/users/pkdash/repos","events_url":"https://api.github.com/users/pkdash/events{/privacy}","received_events_url":"https://api.github.com/users/pkdash/received_events","type":"User","site_admin":false},"assignees":[{"login":"pkdash","id":2523434,"avatar_url":"https://avatars.githubusercontent.com/u/2523434?v=3","gravatar_id":"","url":"https://api.github.com/users/pkdash","html_url":"https://github.com/pkdash","followers_url":"https://api.github.com/users/pkdash/followers","following_url":"https://api.github.com/users/pkdash/following{/other_user}","gists_url":"https://api.github.com/users/pkdash/gists{/gist_id}","starred_url":"https://api.github.com/users/pkdash/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pkdash/subscriptions","organizations_url":"https://api.github.com/users/pkdash/orgs","repos_url":"https://api.github.com/users/pkdash/repos","events_url":"https://api.github.com/users/pkdash/events{/privacy}","received_events_url":"https://api.github.com/users/pkdash/received_events","type":"User","site_admin":false}],"milestone":{"url":"https://api.github.com/repos/hydroshare/hydroshare/milestones/13","html_url":"https://github.com/hydroshare/hydroshare/milestones/Release%201.8.0","labels_url":"https://api.github.com/repos/hydroshare/hydroshare/milestones/13/labels","id":1193860,"number":13,"title":"Release 1.8.0","description":"This release will focus on enhancements to the ability to organize content in HydroShare.  \r\n* Composite resources.  These are collections of other resources. More elaborate resource types, such as RiverML describing river geometry, model instances and model packages will be composite resources.\r\n* Organization of my resources into folders\r\n* Tagging of resources\r\n* Hierarchical directories within resources\r\n* iRODS Workflow Structured Objects (WSO) resource type. \r\n","creator":{"login":"dtarb","id":3456833,"avatar_url":"https://avatars.githubusercontent.com/u/3456833?v=3","gravatar_id":"","url":"https://api.github.com/users/dtarb","html_url":"https://github.com/dtarb","followers_url":"https://api.github.com/users/dtarb/followers","following_url":"https://api.github.com/users/dtarb/following{/other_user}","gists_url":"https://api.github.com/users/dtarb/gists{/gist_id}","starred_url":"https://api.github.com/users/dtarb/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dtarb/subscriptions","organizations_url":"https://api.github.com/users/dtarb/orgs","repos_url":"https://api.github.com/users/dtarb/repos","events_url":"https://api.github.com/users/dtarb/events{/privacy}","received_events_url":"https://api.github.com/users/dtarb/received_events","type":"User","site_admin":false},"open_issues":91,"closed_issues":6,"state":"open","created_at":"2015-07-05T20:52:16Z","updated_at":"2016-06-28T18:45:41Z","due_on":null,"closed_at":null},"comments":9,"created_at":"2016-03-08T00:09:51Z","updated_at":"2016-07-11T21:29:49Z","closed_at":null,"body":"See #965 for beginning of these discussions. @ironfroggy @selimnairb \r\n\r\nOn 3/7 in the face to face meeting, we covered the issue of how apps can bind to and do things with specific files from a resource. We evaluated several possible options and settled upon a \"metadata harvesting\" option for each file in a resource, separately. In this option, at the time a file is uploaded, the __validator__ for the resource harvests metadata on the file, including\r\n\r\n* its MIME type\r\n* its FILE type as determined by its magic number. \r\n* as applicable, embedded metadata about its contents. \r\n    * variable names in GeoTiff\r\n    * variable names in NetCDF \r\n    * time series metadata in WaterML2\r\n    * whether database resources in SQLite conform to common schemas, \r\n      including ODM1.1 and ODM2. \r\n\r\nThis metadata is then made part of resource metadata and is made available to apps, which can then act on this type information. \r\n\r\nThe whole solution contains two parts, one for the app side and one for the \"app resource\" side. \r\n* In the app resource, there is a representation of what file types are expected for the app. This allows the HydroShare resource landing page to put up a list of apps that can process the resource. At the beginning, this will be a list of type information and qualifications, treated as logical \"or\". Geotiff.inundation might mean a Geotiff type file containing an inundation map, etc. \r\n* In the app itself, metadata describes the attributes of each file. The app can then choose which files to process. \r\n\r\nThis mechanism has several strengths: \r\n\r\n1. it occurs as part of resource validation, so there is a natural place for it in resource upload. \r\n2. it doesn't require human categorization of files; they are categorized according to their content. There is thus no extra user work in this categorization; just extra programming. \r\n\r\nThis fits into the resource type hierarchy in an easily explainable way: the resource type describes the intent of the resource, i;e., why data was collected. This metadata instead enables visualization of specific files in the resource. An app might use both in determining the best visualization. "},"comment":{"url":"https://api.github.com/repos/hydroshare/hydroshare/issues/comments/231871357","html_url":"https://github.com/hydroshare/hydroshare/issues/1034#issuecomment-231871357","issue_url":"https://api.github.com/repos/hydroshare/hydroshare/issues/1034","id":231871357,"user":{"login":"pkdash","id":2523434,"avatar_url":"https://avatars.githubusercontent.com/u/2523434?v=3","gravatar_id":"","url":"https://api.github.com/users/pkdash","html_url":"https://github.com/pkdash","followers_url":"https://api.github.com/users/pkdash/followers","following_url":"https://api.github.com/users/pkdash/following{/other_user}","gists_url":"https://api.github.com/users/pkdash/gists{/gist_id}","starred_url":"https://api.github.com/users/pkdash/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pkdash/subscriptions","organizations_url":"https://api.github.com/users/pkdash/orgs","repos_url":"https://api.github.com/users/pkdash/repos","events_url":"https://api.github.com/users/pkdash/events{/privacy}","received_events_url":"https://api.github.com/users/pkdash/received_events","type":"User","site_admin":false},"created_at":"2016-07-11T21:29:49Z","updated_at":"2016-07-11T21:29:49Z","body":"As it relates to resource level metadata and file type metadata for a generic resource,  here are my assumptions:\r\n\r\n- For supported file types, the 'format' core metadata element will be part of the file metadata\r\n- For unsupported file types, the 'format' core element will be part of the resource metadata\r\n- The 'coverage' core element will remain as part of the resource level metadata and the coverage data needs to be computed from the known file types within the resource.\r\n- In the case of ODM2 SQLite file (file type) currently in timeseries resource as part of the metadata extraction we extract metadata for core metadata elements such as 'title', 'abstract' and 'keywords'. However, timeseries as a supported file type in a generic resource, we should not be  generating  'title', 'abstract' and 'keywords' from the extracted metadata."}}
{"actor":{"display_login":"pkdash"}}