{"action":"created","issue":{"url":"https://api.github.com/repos/open-gamma-ray-astro/gamma-astro-data-formats/issues/45","repository_url":"https://api.github.com/repos/open-gamma-ray-astro/gamma-astro-data-formats","labels_url":"https://api.github.com/repos/open-gamma-ray-astro/gamma-astro-data-formats/issues/45/labels{/name}","comments_url":"https://api.github.com/repos/open-gamma-ray-astro/gamma-astro-data-formats/issues/45/comments","events_url":"https://api.github.com/repos/open-gamma-ray-astro/gamma-astro-data-formats/issues/45/events","html_url":"https://github.com/open-gamma-ray-astro/gamma-astro-data-formats/issues/45","id":152484883,"number":45,"title":"SED and Binned Likelihood Formats","user":{"login":"woodmd","id":5733106,"avatar_url":"https://avatars.githubusercontent.com/u/5733106?v=3","gravatar_id":"","url":"https://api.github.com/users/woodmd","html_url":"https://github.com/woodmd","followers_url":"https://api.github.com/users/woodmd/followers","following_url":"https://api.github.com/users/woodmd/following{/other_user}","gists_url":"https://api.github.com/users/woodmd/gists{/gist_id}","starred_url":"https://api.github.com/users/woodmd/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/woodmd/subscriptions","organizations_url":"https://api.github.com/users/woodmd/orgs","repos_url":"https://api.github.com/users/woodmd/repos","events_url":"https://api.github.com/users/woodmd/events{/privacy}","received_events_url":"https://api.github.com/users/woodmd/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/open-gamma-ray-astro/gamma-astro-data-formats/labels/type:%20question","name":"type: question","color":"fbca04"}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":{"url":"https://api.github.com/repos/open-gamma-ray-astro/gamma-astro-data-formats/milestones/4","html_url":"https://github.com/open-gamma-ray-astro/gamma-astro-data-formats/milestones/whishlist","labels_url":"https://api.github.com/repos/open-gamma-ray-astro/gamma-astro-data-formats/milestones/4/labels","id":1504140,"number":4,"title":"whishlist","description":"","creator":{"login":"cdeil","id":852409,"avatar_url":"https://avatars.githubusercontent.com/u/852409?v=3","gravatar_id":"","url":"https://api.github.com/users/cdeil","html_url":"https://github.com/cdeil","followers_url":"https://api.github.com/users/cdeil/followers","following_url":"https://api.github.com/users/cdeil/following{/other_user}","gists_url":"https://api.github.com/users/cdeil/gists{/gist_id}","starred_url":"https://api.github.com/users/cdeil/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cdeil/subscriptions","organizations_url":"https://api.github.com/users/cdeil/orgs","repos_url":"https://api.github.com/users/cdeil/repos","events_url":"https://api.github.com/users/cdeil/events{/privacy}","received_events_url":"https://api.github.com/users/cdeil/received_events","type":"User","site_admin":false},"open_issues":12,"closed_issues":0,"state":"open","created_at":"2016-01-13T10:24:11Z","updated_at":"2016-06-20T06:37:53Z","due_on":null,"closed_at":null},"comments":18,"created_at":"2016-05-02T05:46:02Z","updated_at":"2016-09-02T09:33:23Z","closed_at":null,"body":"This is an open thread to collect comments and feedback on the proposed formats for [Binned Likelihood Profiles](http://gamma-astro-data-formats.readthedocs.io/en/latest/results/binned_likelihoods/index.html).  Since these formats are effectively just SEDs with extra information in each bin it would probably be a good idea to try to share conventions with the [Flux points format](http://gamma-astro-data-formats.readthedocs.io/en/latest/results/flux_points/index.html).  @zblz Did you have any thoughts on this?\r\n\r\nHere are a few open questions:\r\n\r\n- Do we want a fixed 95% CL for the UL columns?  I saw that the naima format specifies the confidence level with a header keyword so maybe we should do something similar.\r\n\r\n- Do we need a column to specify the bin center?  The current format only specifies the lower and upper edges (`E_MIN` and `E_MAX`).  Differential quantities are assumed to be evaluated at the geometric center of the bin but we could add a bin center column (`E_CTR`) that would allow the evaluation point to be explicitly defined. \r\n\r\n- Which columns do we want to make optional?  The current specification calls for five different unit conversion factors (`DFDE`,`E2DFDE`,`FLUX`,`EFLUX`,`NPRED`).  I mainly did this for convenience so that one doesn't need to know anything about the underlying spectral model to convert between units.  Of these `E2DFDE` could clearly be optional (or maybe dropped entirely) since it can be trivially computed from `DFDE`.  `NPRED` provides additional information that can't be rederived from the other four unit columns but since it may not always be available it should probably also be optional.\r\n\r\n- Do we want a column for exposure?  This can be inferred from `FLUX` and `NPRED` but having it as an independent column could be useful in some circumstances.\r\n\r\n- Do we need a column that describes the model for the spectral distribution within each bin?  For instance when making LAT SEDs we typically substitute the source with an index=2.0 power-law.  For many applications this doesn't really matter (e.g. when using narrow bins) but when using wide bins or when the effect of energy dispersion is significant the assumed shape within the bin can have a big influence on the measured flux.  One option would be a column for the logarithmic slope of the bin model evaluated at the bin center. "},"comment":{"url":"https://api.github.com/repos/open-gamma-ray-astro/gamma-astro-data-formats/issues/comments/244329080","html_url":"https://github.com/open-gamma-ray-astro/gamma-astro-data-formats/issues/45#issuecomment-244329080","issue_url":"https://api.github.com/repos/open-gamma-ray-astro/gamma-astro-data-formats/issues/45","id":244329080,"user":{"login":"cdeil","id":852409,"avatar_url":"https://avatars.githubusercontent.com/u/852409?v=3","gravatar_id":"","url":"https://api.github.com/users/cdeil","html_url":"https://github.com/cdeil","followers_url":"https://api.github.com/users/cdeil/followers","following_url":"https://api.github.com/users/cdeil/following{/other_user}","gists_url":"https://api.github.com/users/cdeil/gists{/gist_id}","starred_url":"https://api.github.com/users/cdeil/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cdeil/subscriptions","organizations_url":"https://api.github.com/users/cdeil/orgs","repos_url":"https://api.github.com/users/cdeil/repos","events_url":"https://api.github.com/users/cdeil/events{/privacy}","received_events_url":"https://api.github.com/users/cdeil/received_events","type":"User","site_admin":false},"created_at":"2016-09-02T09:33:23Z","updated_at":"2016-09-02T09:33:23Z","body":"@woodmd - Thanks for volunteering to do the work and updating / adding to the spec here!\r\n\r\nBefore you do: I'm still in favour of just making more columns of the existing format optional.\r\nTo me, that seems like the simplest solution both for the spec.\r\nWhat we do here is to define field names and semantics, so that we have a format for data exchange.\r\nTools can then easily check which fields are present for a given SED and process or plot it.\r\n\r\nYou wrote:\r\n\r\n> The SED format is explicitly defined as a binned format whereas it seems the flux point formats currently used by gammapy and naima are intended for unbinned differential flux points. Perhaps a solution for reconciling the two formats would be to define flux points as a subset of the SED format. e.g. the only required columns would be a sequence of energies and differential fluxes (e.g. E_REF, REF_DFDE, and NORM in the current version of the SED format).\r\n\r\nE.g. Fermi catalogs only contain integral flux per bin.\r\nTeV instruments usually only publish differential flux measured in energy bins.\r\nIn both cases we do (often) want to store e_min and e_max, i.e. the energy bin.\r\nI don't consider either of those \"unbinned\" and I think they fit well with the fields you have already defined.\r\n\r\n+1 to use a subset of the SED format.\r\n\r\nBut apart from maybe E_REF you can't have any required columns, they must all be optional, to support storing the info given for flux points in the Fermi catalogs or as published by TeV instruments.\r\n\r\nYou also wrote:\r\n\r\n> See my previous comment about the difference between binned and unbinned flux formats. For a binned format I think it's hard to drop any additional columns. Regarding the normalization I think we should require at least one integral and one differential measure of the flux. If we only want to require one measure of integral flux then I would vote for using energy flux rather than flux.\r\n\r\nI don't understan why it's hard to change the columns from required to optional.\r\nTechnically, in the spec, it's super-easy.\r\n\r\nFor tools, well, they can just access the fields they need and give an error if one is missing.\r\nYes, there is some benefit to having more required fields you can rely on there.\r\nBut I don't think it's enough to warrant introducing a new \"format\" which is just the same, only with more columns optional.\r\n\r\nAm I not getting something?\r\n\r\nIf there's no agreement on this for now, then I'm still +1 to go ahead and define the simple flux point spec now, and then discuss this question again at some future telcon or f2f meeting."}}
{"actor":{"display_login":"cdeil"}}