{"action":"created","comment":{"url":"https://api.github.com/repos/astropy/astropy-APEs/pulls/comments/78610671","id":78610671,"diff_hunk":"@@ -0,0 +1,273 @@\n+Rework NDData framework\n+-----------------------\n+\n+author: Perry Greenfield, Russell Owen (others?)\n+\n+date-created: 2016 March 31\n+\n+date-last-revised: 2016 July 26\n+\n+type: Standard Track\n+\n+status: Discussion\n+\n+\n+Abstract\n+--------\n+\n+This APE proposes a new architecture for NDData that accommodates\n+the needs of the LSST project and the community in general\n+\n+\n+Detailed description\n+--------------------\n+\n+This APE originated from a meeting on 2016 March 26-27 with LSST to see if \n+there could be better integration of LSST software with Astropy core objects.\n+In particular, could NDData accommodate these needs? The changes discussed\n+below are believed to be generally useful to the community while also \n+accommodating the LSST needs. They do raise a number of issues regarding\n+NDData that have never been completely resolved, and it is hoped that this \n+APE will help in resolving some of them. \n+\n+Background\n+++++++++++\n+\n+The issues that LSST raised mainly are itemized as follows:\n+\n+1. They wish to have two levels of NDData-like objects, one essentially\n+with no metadata, and the other incorporating extra information regarding\n+exposures including standard metadata, optional metadata and WCS. The \n+advantage of this split is that numerical operations on the simpler object\n+have unambiguous interpretations.\n+\n+2. They require masks that support multiple bit planes and options for\n+generating boolean masks from selections on bit planes. This issue was\n+also discussed at a pyastro unconference session a few days earlier with\n+some tentative agreement on how to accomplish this.\n+\n+3. The simpler NDData-like object was required to have information on the\n+origin of the array, if sliced from another array. It also holds the mask\n+and uncertainty information. This corresponds to what they call their\n+MaskedImage object. They support simple arithmetic operations for these\n+objects, propagating masks and uncertainty. They represent uncertainty as\n+variance.\n+\n+4. Their Exposure object corresponds more to astropy’s NDData object, though\n+the organization is somewhat different. It holds a reference to a MaskedImage\n+object, and another attribute is what is called the ExposureInfo object.\n+The ExposureInfo object has some standard attributes such as filter, exposure\n+time and a few others, as well as WCS and a junk drawer for\n+all other metadata (what we currently use .meta for).\n+\n+Names\n++++++\n+\n+In accommodating this separation (and augmentation to a limited degree), it \n+is important to deal with naming issues carefully as each community reads\n+different meaning into terms since they have used different terminology\n+previously. The names used in this APE are subject to discussion and are\n+an important aspect of this APE.\n+\n+One question is whether we should continue to use NDData to represent the \n+more complex object (thus breaking backward compatibility), or simply \n+decide on a different object while deprecating use of NDData. There may be\n+ways of minimizing the effects of changes to NDData by deprecating use of \n+attributes currently planned. These are all issues for discussion.\n+\n+For the purposes of this APE, we will assume that the name for the simpler\n+object is NDArr. This name does not appear to have any other usage in the\n+Python community.  Since metadata and other more complex\n+aspects have been removed from the concept, supporting arithmetic operations\n+becomes much more straightforward, even with masks. Propagation of uncertainty \n+is included, though propagation of uncertainty is certainly not universally straightforward. For both masks and uncertainty, it will be possible to \n+create the NDArr object with options not to propagate one or the other or \n+both to forestall complaints from those for which these operations will be \n+considered invalid.\n+\n+NDArr will support optional units since the propagation of units is\n+unambiguous in mathematical operations.\n+\n+Supported numerical operations for NDData are: (+,-,*,/). While the discussion\n+precluded use of ufuncs, there is no intrinsic reason ufuncs cannot be\n+supported, so this APE will presume they are supported (eventually; there \n+is no implied commitment to support this functionality immediately). Ufuncs\n+shall only work on dimensionless units if units are present.\n+\n+NDArr objects will support simple slicing (no striding, at least in the\n+initial proposal). Slices will be based on array indices and no other\n+interpretation.\n+\n+NDArr objects will retain information about their parent if derived through\n+a slicing operation. This information shall include a pointer to the parent\n+NDArr object, and the bounding box used to obtain the slice. When involving \n+binary arithmetic operations, between two NDArr objects that have different\n+parents (or are themselves distinct parents), there will be no pointer to\n+the original parent(s) present, nor will the bounding box information be\n+retained\n+\n+If operations are unary, parentage and bounding box information is propagated.\n+For binary operations with two NDArr objects, operations\n+will retain parentage and bounding boxes only if both operands share the same\n+parents and bounding box. If they don't, this information will be removed from\n+the result.\n+\n+The NDArr object will permit indexing relative to the parent through a special\n+attribute (details at 11).\n+\n+NDData Details\n+++++++++++++++\n+\n+Composition instead of Inheritance\n+----------------------------------\n+\n+NDData will not be a subclass of NDArr. Instead it will contain references\n+to an NDArr object. To avoid confusion, the attribute that contains the \n+NDArr object will be .arr\n+\n+Need for ExposureInfo object?\n+-----------------------------\n+\n+This is used by LSST to store information about the exposure such as filters,\n+exposure time, etc. but all that is really needed is an isomorphic mapping\n+to corresponding information in the astropy object. The question is: does\n+it make sense to encapsulate that information in a single attribute, and \n+whether all think that these attributes are \"astropy-universal”. For the \n+purposes of this, the only standard attribute is most likely the WCS.\n+\n+Items such as exposure time and filters could be relegated to the meta\n+attribute. Making attributes such as these at the top level could be done\n+through subclasses, though it may lead to many variants. Perhaps the best way\n+to deal with this is for applications or libraries to state their requirements\n+for items required to be in meta and leave it at that.\n+\n+NDData Slicing and Related\n+++++++++++++++++++++++++++\n+\n+Slicing of NDData is supported (eventually) as well as striding. The corresponding \n+WCS attribute, if present, is updated accordingly. Eventually there may be support\n+for slicing based on world coordinates, but this will not be part of the initial\n+implementation.\n+\n+NDData does not support mathematical operations. Utilities will be provided to aid\n+in how metadata items are to be propagated when the user merges NDData instances.\n+\n+Mask Object\n++++++++++++\n+\n+The mask attribute now will be an instance of a Mask Object (but to support\n+backward compatibilty, the attribute must have a different name than mask since\n+the current implementation treats that attribute as a boolean array (or any\n+object that mimics one). [An alternative option is to assume that the mask\n+attribute of NDArr is a callable whereas the mask attribute of NDData represents\n+a boolean array; this would be workable, but also likely confusing]\n+This APE proposes that the new attribute have the name: lone_ranger. \n+Well, perhaps dmask.","path":"APE9.rst","position":165,"original_position":165,"commit_id":"6d366658e1331bcae1df68a52cda23ffc1ce8607","original_commit_id":"6d366658e1331bcae1df68a52cda23ffc1ce8607","user":{"login":"perrygreenfield","id":981803,"avatar_url":"https://avatars.githubusercontent.com/u/981803?v=3","gravatar_id":"","url":"https://api.github.com/users/perrygreenfield","html_url":"https://github.com/perrygreenfield","followers_url":"https://api.github.com/users/perrygreenfield/followers","following_url":"https://api.github.com/users/perrygreenfield/following{/other_user}","gists_url":"https://api.github.com/users/perrygreenfield/gists{/gist_id}","starred_url":"https://api.github.com/users/perrygreenfield/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/perrygreenfield/subscriptions","organizations_url":"https://api.github.com/users/perrygreenfield/orgs","repos_url":"https://api.github.com/users/perrygreenfield/repos","events_url":"https://api.github.com/users/perrygreenfield/events{/privacy}","received_events_url":"https://api.github.com/users/perrygreenfield/received_events","type":"User","site_admin":false},"body":"I've been away a while, but I'm back... I would be happy with any sensible consensus (e.g., if the consensus was .dq_aka_data_quality_mask I don't think I would agree :-)","created_at":"2016-09-13T17:53:17Z","updated_at":"2016-09-13T17:53:17Z","html_url":"https://github.com/astropy/astropy-APEs/pull/14#discussion_r78610671","pull_request_url":"https://api.github.com/repos/astropy/astropy-APEs/pulls/14","_links":{"self":{"href":"https://api.github.com/repos/astropy/astropy-APEs/pulls/comments/78610671"},"html":{"href":"https://github.com/astropy/astropy-APEs/pull/14#discussion_r78610671"},"pull_request":{"href":"https://api.github.com/repos/astropy/astropy-APEs/pulls/14"}}},"pull_request":{"url":"https://api.github.com/repos/astropy/astropy-APEs/pulls/14","id":78882011,"html_url":"https://github.com/astropy/astropy-APEs/pull/14","diff_url":"https://github.com/astropy/astropy-APEs/pull/14.diff","patch_url":"https://github.com/astropy/astropy-APEs/pull/14.patch","issue_url":"https://api.github.com/repos/astropy/astropy-APEs/issues/14","number":14,"state":"open","locked":false,"title":"Remaking NDData to support LSST needs (APE11)","user":{"login":"perrygreenfield","id":981803,"avatar_url":"https://avatars.githubusercontent.com/u/981803?v=3","gravatar_id":"","url":"https://api.github.com/users/perrygreenfield","html_url":"https://github.com/perrygreenfield","followers_url":"https://api.github.com/users/perrygreenfield/followers","following_url":"https://api.github.com/users/perrygreenfield/following{/other_user}","gists_url":"https://api.github.com/users/perrygreenfield/gists{/gist_id}","starred_url":"https://api.github.com/users/perrygreenfield/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/perrygreenfield/subscriptions","organizations_url":"https://api.github.com/users/perrygreenfield/orgs","repos_url":"https://api.github.com/users/perrygreenfield/repos","events_url":"https://api.github.com/users/perrygreenfield/events{/privacy}","received_events_url":"https://api.github.com/users/perrygreenfield/received_events","type":"User","site_admin":false},"body":"This is a first draft of an APE to support the issues raised by LSST and the subsequent email discussions regarding that","created_at":"2016-07-26T16:20:33Z","updated_at":"2016-09-13T17:53:17Z","closed_at":null,"merged_at":null,"merge_commit_sha":"72a6db4a435fdd5626f268372b8d6ef0dae260a0","assignee":null,"assignees":[],"milestone":null,"commits_url":"https://api.github.com/repos/astropy/astropy-APEs/pulls/14/commits","review_comments_url":"https://api.github.com/repos/astropy/astropy-APEs/pulls/14/comments","review_comment_url":"https://api.github.com/repos/astropy/astropy-APEs/pulls/comments{/number}","comments_url":"https://api.github.com/repos/astropy/astropy-APEs/issues/14/comments","statuses_url":"https://api.github.com/repos/astropy/astropy-APEs/statuses/6d366658e1331bcae1df68a52cda23ffc1ce8607","head":{"label":"perrygreenfield:master","ref":"master","sha":"6d366658e1331bcae1df68a52cda23ffc1ce8607","user":{"login":"perrygreenfield","id":981803,"avatar_url":"https://avatars.githubusercontent.com/u/981803?v=3","gravatar_id":"","url":"https://api.github.com/users/perrygreenfield","html_url":"https://github.com/perrygreenfield","followers_url":"https://api.github.com/users/perrygreenfield/followers","following_url":"https://api.github.com/users/perrygreenfield/following{/other_user}","gists_url":"https://api.github.com/users/perrygreenfield/gists{/gist_id}","starred_url":"https://api.github.com/users/perrygreenfield/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/perrygreenfield/subscriptions","organizations_url":"https://api.github.com/users/perrygreenfield/orgs","repos_url":"https://api.github.com/users/perrygreenfield/repos","events_url":"https://api.github.com/users/perrygreenfield/events{/privacy}","received_events_url":"https://api.github.com/users/perrygreenfield/received_events","type":"User","site_admin":false},"repo":{"id":55178085,"name":"astropy-APEs","full_name":"perrygreenfield/astropy-APEs","owner":{"login":"perrygreenfield","id":981803,"avatar_url":"https://avatars.githubusercontent.com/u/981803?v=3","gravatar_id":"","url":"https://api.github.com/users/perrygreenfield","html_url":"https://github.com/perrygreenfield","followers_url":"https://api.github.com/users/perrygreenfield/followers","following_url":"https://api.github.com/users/perrygreenfield/following{/other_user}","gists_url":"https://api.github.com/users/perrygreenfield/gists{/gist_id}","starred_url":"https://api.github.com/users/perrygreenfield/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/perrygreenfield/subscriptions","organizations_url":"https://api.github.com/users/perrygreenfield/orgs","repos_url":"https://api.github.com/users/perrygreenfield/repos","events_url":"https://api.github.com/users/perrygreenfield/events{/privacy}","received_events_url":"https://api.github.com/users/perrygreenfield/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/perrygreenfield/astropy-APEs","description":"A repository storing the Astropy Proposals for Enhancement.","fork":true,"url":"https://api.github.com/repos/perrygreenfield/astropy-APEs","forks_url":"https://api.github.com/repos/perrygreenfield/astropy-APEs/forks","keys_url":"https://api.github.com/repos/perrygreenfield/astropy-APEs/keys{/key_id}","collaborators_url":"https://api.github.com/repos/perrygreenfield/astropy-APEs/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/perrygreenfield/astropy-APEs/teams","hooks_url":"https://api.github.com/repos/perrygreenfield/astropy-APEs/hooks","issue_events_url":"https://api.github.com/repos/perrygreenfield/astropy-APEs/issues/events{/number}","events_url":"https://api.github.com/repos/perrygreenfield/astropy-APEs/events","assignees_url":"https://api.github.com/repos/perrygreenfield/astropy-APEs/assignees{/user}","branches_url":"https://api.github.com/repos/perrygreenfield/astropy-APEs/branches{/branch}","tags_url":"https://api.github.com/repos/perrygreenfield/astropy-APEs/tags","blobs_url":"https://api.github.com/repos/perrygreenfield/astropy-APEs/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/perrygreenfield/astropy-APEs/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/perrygreenfield/astropy-APEs/git/refs{/sha}","trees_url":"https://api.github.com/repos/perrygreenfield/astropy-APEs/git/trees{/sha}","statuses_url":"https://api.github.com/repos/perrygreenfield/astropy-APEs/statuses/{sha}","languages_url":"https://api.github.com/repos/perrygreenfield/astropy-APEs/languages","stargazers_url":"https://api.github.com/repos/perrygreenfield/astropy-APEs/stargazers","contributors_url":"https://api.github.com/repos/perrygreenfield/astropy-APEs/contributors","subscribers_url":"https://api.github.com/repos/perrygreenfield/astropy-APEs/subscribers","subscription_url":"https://api.github.com/repos/perrygreenfield/astropy-APEs/subscription","commits_url":"https://api.github.com/repos/perrygreenfield/astropy-APEs/commits{/sha}","git_commits_url":"https://api.github.com/repos/perrygreenfield/astropy-APEs/git/commits{/sha}","comments_url":"https://api.github.com/repos/perrygreenfield/astropy-APEs/comments{/number}","issue_comment_url":"https://api.github.com/repos/perrygreenfield/astropy-APEs/issues/comments{/number}","contents_url":"https://api.github.com/repos/perrygreenfield/astropy-APEs/contents/{+path}","compare_url":"https://api.github.com/repos/perrygreenfield/astropy-APEs/compare/{base}...{head}","merges_url":"https://api.github.com/repos/perrygreenfield/astropy-APEs/merges","archive_url":"https://api.github.com/repos/perrygreenfield/astropy-APEs/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/perrygreenfield/astropy-APEs/downloads","issues_url":"https://api.github.com/repos/perrygreenfield/astropy-APEs/issues{/number}","pulls_url":"https://api.github.com/repos/perrygreenfield/astropy-APEs/pulls{/number}","milestones_url":"https://api.github.com/repos/perrygreenfield/astropy-APEs/milestones{/number}","notifications_url":"https://api.github.com/repos/perrygreenfield/astropy-APEs/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/perrygreenfield/astropy-APEs/labels{/name}","releases_url":"https://api.github.com/repos/perrygreenfield/astropy-APEs/releases{/id}","deployments_url":"https://api.github.com/repos/perrygreenfield/astropy-APEs/deployments","created_at":"2016-03-31T19:41:44Z","updated_at":"2016-03-31T14:58:08Z","pushed_at":"2016-08-13T10:18:07Z","git_url":"git://github.com/perrygreenfield/astropy-APEs.git","ssh_url":"git@github.com:perrygreenfield/astropy-APEs.git","clone_url":"https://github.com/perrygreenfield/astropy-APEs.git","svn_url":"https://github.com/perrygreenfield/astropy-APEs","homepage":null,"size":160,"stargazers_count":0,"watchers_count":0,"language":null,"has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":1,"mirror_url":null,"open_issues_count":1,"forks":1,"open_issues":1,"watchers":0,"default_branch":"master"}},"base":{"label":"astropy:master","ref":"master","sha":"012787a5a2c7c78a2409de7748b75ffa91ca25fd","user":{"login":"astropy","id":847984,"avatar_url":"https://avatars.githubusercontent.com/u/847984?v=3","gravatar_id":"","url":"https://api.github.com/users/astropy","html_url":"https://github.com/astropy","followers_url":"https://api.github.com/users/astropy/followers","following_url":"https://api.github.com/users/astropy/following{/other_user}","gists_url":"https://api.github.com/users/astropy/gists{/gist_id}","starred_url":"https://api.github.com/users/astropy/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/astropy/subscriptions","organizations_url":"https://api.github.com/users/astropy/orgs","repos_url":"https://api.github.com/users/astropy/repos","events_url":"https://api.github.com/users/astropy/events{/privacy}","received_events_url":"https://api.github.com/users/astropy/received_events","type":"Organization","site_admin":false},"repo":{"id":14154504,"name":"astropy-APEs","full_name":"astropy/astropy-APEs","owner":{"login":"astropy","id":847984,"avatar_url":"https://avatars.githubusercontent.com/u/847984?v=3","gravatar_id":"","url":"https://api.github.com/users/astropy","html_url":"https://github.com/astropy","followers_url":"https://api.github.com/users/astropy/followers","following_url":"https://api.github.com/users/astropy/following{/other_user}","gists_url":"https://api.github.com/users/astropy/gists{/gist_id}","starred_url":"https://api.github.com/users/astropy/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/astropy/subscriptions","organizations_url":"https://api.github.com/users/astropy/orgs","repos_url":"https://api.github.com/users/astropy/repos","events_url":"https://api.github.com/users/astropy/events{/privacy}","received_events_url":"https://api.github.com/users/astropy/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/astropy/astropy-APEs","description":"A repository storing the Astropy Proposals for Enhancement.","fork":false,"url":"https://api.github.com/repos/astropy/astropy-APEs","forks_url":"https://api.github.com/repos/astropy/astropy-APEs/forks","keys_url":"https://api.github.com/repos/astropy/astropy-APEs/keys{/key_id}","collaborators_url":"https://api.github.com/repos/astropy/astropy-APEs/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/astropy/astropy-APEs/teams","hooks_url":"https://api.github.com/repos/astropy/astropy-APEs/hooks","issue_events_url":"https://api.github.com/repos/astropy/astropy-APEs/issues/events{/number}","events_url":"https://api.github.com/repos/astropy/astropy-APEs/events","assignees_url":"https://api.github.com/repos/astropy/astropy-APEs/assignees{/user}","branches_url":"https://api.github.com/repos/astropy/astropy-APEs/branches{/branch}","tags_url":"https://api.github.com/repos/astropy/astropy-APEs/tags","blobs_url":"https://api.github.com/repos/astropy/astropy-APEs/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/astropy/astropy-APEs/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/astropy/astropy-APEs/git/refs{/sha}","trees_url":"https://api.github.com/repos/astropy/astropy-APEs/git/trees{/sha}","statuses_url":"https://api.github.com/repos/astropy/astropy-APEs/statuses/{sha}","languages_url":"https://api.github.com/repos/astropy/astropy-APEs/languages","stargazers_url":"https://api.github.com/repos/astropy/astropy-APEs/stargazers","contributors_url":"https://api.github.com/repos/astropy/astropy-APEs/contributors","subscribers_url":"https://api.github.com/repos/astropy/astropy-APEs/subscribers","subscription_url":"https://api.github.com/repos/astropy/astropy-APEs/subscription","commits_url":"https://api.github.com/repos/astropy/astropy-APEs/commits{/sha}","git_commits_url":"https://api.github.com/repos/astropy/astropy-APEs/git/commits{/sha}","comments_url":"https://api.github.com/repos/astropy/astropy-APEs/comments{/number}","issue_comment_url":"https://api.github.com/repos/astropy/astropy-APEs/issues/comments{/number}","contents_url":"https://api.github.com/repos/astropy/astropy-APEs/contents/{+path}","compare_url":"https://api.github.com/repos/astropy/astropy-APEs/compare/{base}...{head}","merges_url":"https://api.github.com/repos/astropy/astropy-APEs/merges","archive_url":"https://api.github.com/repos/astropy/astropy-APEs/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/astropy/astropy-APEs/downloads","issues_url":"https://api.github.com/repos/astropy/astropy-APEs/issues{/number}","pulls_url":"https://api.github.com/repos/astropy/astropy-APEs/pulls{/number}","milestones_url":"https://api.github.com/repos/astropy/astropy-APEs/milestones{/number}","notifications_url":"https://api.github.com/repos/astropy/astropy-APEs/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/astropy/astropy-APEs/labels{/name}","releases_url":"https://api.github.com/repos/astropy/astropy-APEs/releases{/id}","deployments_url":"https://api.github.com/repos/astropy/astropy-APEs/deployments","created_at":"2013-11-05T21:20:15Z","updated_at":"2016-08-31T21:54:11Z","pushed_at":"2016-08-29T00:08:49Z","git_url":"git://github.com/astropy/astropy-APEs.git","ssh_url":"git@github.com:astropy/astropy-APEs.git","clone_url":"https://github.com/astropy/astropy-APEs.git","svn_url":"https://github.com/astropy/astropy-APEs","homepage":null,"size":148,"stargazers_count":5,"watchers_count":5,"language":null,"has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":13,"mirror_url":null,"open_issues_count":4,"forks":13,"open_issues":4,"watchers":5,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/astropy/astropy-APEs/pulls/14"},"html":{"href":"https://github.com/astropy/astropy-APEs/pull/14"},"issue":{"href":"https://api.github.com/repos/astropy/astropy-APEs/issues/14"},"comments":{"href":"https://api.github.com/repos/astropy/astropy-APEs/issues/14/comments"},"review_comments":{"href":"https://api.github.com/repos/astropy/astropy-APEs/pulls/14/comments"},"review_comment":{"href":"https://api.github.com/repos/astropy/astropy-APEs/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/astropy/astropy-APEs/pulls/14/commits"},"statuses":{"href":"https://api.github.com/repos/astropy/astropy-APEs/statuses/6d366658e1331bcae1df68a52cda23ffc1ce8607"}}}}
{"actor":{"display_login":"perrygreenfield"}}