{"action":"created","issue":{"url":"https://api.github.com/repos/GalSim-developers/GalSim/issues/706","repository_url":"https://api.github.com/repos/GalSim-developers/GalSim","labels_url":"https://api.github.com/repos/GalSim-developers/GalSim/issues/706/labels{/name}","comments_url":"https://api.github.com/repos/GalSim-developers/GalSim/issues/706/comments","events_url":"https://api.github.com/repos/GalSim-developers/GalSim/issues/706/events","html_url":"https://github.com/GalSim-developers/GalSim/pull/706","id":129946882,"number":706,"title":"#654 Updates to DES module","user":{"login":"rmjarvis","id":623887,"avatar_url":"https://avatars.githubusercontent.com/u/623887?v=3","gravatar_id":"","url":"https://api.github.com/users/rmjarvis","html_url":"https://github.com/rmjarvis","followers_url":"https://api.github.com/users/rmjarvis/followers","following_url":"https://api.github.com/users/rmjarvis/following{/other_user}","gists_url":"https://api.github.com/users/rmjarvis/gists{/gist_id}","starred_url":"https://api.github.com/users/rmjarvis/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rmjarvis/subscriptions","organizations_url":"https://api.github.com/users/rmjarvis/orgs","repos_url":"https://api.github.com/users/rmjarvis/repos","events_url":"https://api.github.com/users/rmjarvis/events{/privacy}","received_events_url":"https://api.github.com/users/rmjarvis/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":12,"created_at":"2016-01-30T06:08:38Z","updated_at":"2016-03-17T17:30:52Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/GalSim-developers/GalSim/pulls/706","html_url":"https://github.com/GalSim-developers/GalSim/pull/706","diff_url":"https://github.com/GalSim-developers/GalSim/pull/706.diff","patch_url":"https://github.com/GalSim-developers/GalSim/pull/706.patch"},"body":"This PR is mostly about updates to the DES module, but there are a few things of interest to non-DES folks as well.\r\n\r\n* The original issue #654 was based on a bug @ckrawiec found in the MEDS code, where the code had gotten out of sync with other changes we had made in GalSim, which she fixed.  I added a unit test for the bug she found and moved some other tests that had been in the des module into tests/test_des.py, so they are now part of the standard test suite to make sure we notice if things get out of sync again.  I also added more unit tests of the various classes in the des module to get better coverage of that module including tests using @esheldon's meds code to make sure the meds files we write stay in sync with what that code is expecting to read in.\r\n\r\n* I added a feature to our fits reader code to catch pyfits.VerifyErrors when reading an hdu.  We had an image from SV data that had some bad header entries (with NAN.0 values), which pyfits didn't like.  However, `hdu.verify('fix')` works to fix them as the VerifyError error message kindly informed me about.  So now GalSim will try to catch these errors and if it catches one, it runs verify('fix') before moving on.  (cf. Issue #602)  Reading this file is now a unit test in test_des.py.\r\n\r\n* After I upgraded my astropy version a month or so ago, I started getting warnings that look like\r\n```\r\n/Library/Python/2.7/site-packages/galsim/fits.py:793: UserWarning: No C++ Image template instantiation for pixel type <type 'numpy.uint16'>\r\n/Library/Python/2.7/site-packages/galsim/fits.py:794: UserWarning:    Using numpy.float64 instead.\r\n```\r\nI get these warnings even if I go way back to old versions of GalSim.  I'm not sure why these warnings didn't show up before, but we definitely do have unit tests that try to read in images with unsigned int data.  Some of the new des meds tests do.  But also some hsm tests.  So I now explicitly recast such images into the corresponding signed integer types and use that.  I think that's ok, but we might want to consider if we should add unsigned int types to the list of `Image<T>` templates we instantiate.\r\n\r\n* Making the above change broke one of the hsm tests that reads in a FITS file with a uint16 image.  It had been converting the unit16 data to float64, which is what it does when it doesn't know how to handle an input type.  But the hsm code is only instantiated for int32, float32, and float64.  Not int16, which is what this is converted to now.  So I added a `_convertImage` function in hsm.py to convert int16 images into int32 before passing them down into the C++ layer.  @rmandelb you might want to check if you think this is ok, or if it would be better to add the int16 instantiation.  The line in particular that failed is [here](https://github.com/GalSim-developers/GalSim/blob/master/tests/test_hsm.py#L221).  The data arrays in the images here are all uint16.\r\n\r\n* I added a number of example configuration files in examples/des that show some more sophisticated kinds of customizations that are now possible using the new refactored config framework from #691.  In particular, blendset.yaml is pretty complex.  It creates sets of nearby (blended) objects and puts each one in the center of a postage stamp as though cutout from a larger observation image (like we do in DES with the MEDS files), so the same noise realization for each stamp in the set, just recentered.  In addition, it produces a second file that has all the same galaxies and noise, but this time without the neighboring flux.  So the only impact of the neighbors is the additional Poisson noise from their flux.  This simulation was requested by @beckermr to make sure the new config code had enough hooks in place to be able to do something this complicated.\r\n@TallJimbo I think you'll be interested in checking out this and the other files in this directory with an eye towards the kinds of simulations we'll need for Great4.  \r\n@beckermr You might also want to take a look, since the syntax has changed a bit from what you saw a few weeks ago for this.  In particular, Jim's suggestion of using builder objects rather than free functions here allows for variables to be saved in `self` attributes, rather than in the config dict, which is a bit cleaner.\r\n\r\n* Parametric galaxies created by COSMOSCatalog now set an index attribute with their index into the full COSMOS catalog just like the attribute that gets set for real galaxies.  This is related to @beckermr's request in #694, but extended to the parametric galaxies.  It seemed appropriate to just have all galaxies made by COSMOSCatalog set this attribute.  In the examples/des/meds.yaml simulation, the index is written to the truth file, and it's convenient to have it still work if the galaxies are switched from real to parametric.\r\n\r\n* I made a few more minor changes to the config processing code.  Most of the changes I needed for these simulations, I already put into #691, but I had a few more after code review had already started there.  The most significant one wasn't really required here, but I realized that I had some logic wrong.  It had been the case that input types needed to register all the value and gsobject types that would use that input type.  But I realized that the logic here is backwards.  A user might make a new builder that uses an existing input type in a different way, so that's where the registration of this connection should be, so I switched it to have the builder types register the connection to any input types they use rather than the other way around.\r\n"},"comment":{"url":"https://api.github.com/repos/GalSim-developers/GalSim/issues/comments/197989156","html_url":"https://github.com/GalSim-developers/GalSim/pull/706#issuecomment-197989156","issue_url":"https://api.github.com/repos/GalSim-developers/GalSim/issues/706","id":197989156,"user":{"login":"joezuntz","id":220537,"avatar_url":"https://avatars.githubusercontent.com/u/220537?v=3","gravatar_id":"","url":"https://api.github.com/users/joezuntz","html_url":"https://github.com/joezuntz","followers_url":"https://api.github.com/users/joezuntz/followers","following_url":"https://api.github.com/users/joezuntz/following{/other_user}","gists_url":"https://api.github.com/users/joezuntz/gists{/gist_id}","starred_url":"https://api.github.com/users/joezuntz/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/joezuntz/subscriptions","organizations_url":"https://api.github.com/users/joezuntz/orgs","repos_url":"https://api.github.com/users/joezuntz/repos","events_url":"https://api.github.com/users/joezuntz/events{/privacy}","received_events_url":"https://api.github.com/users/joezuntz/received_events","type":"User","site_admin":false},"created_at":"2016-03-17T17:30:52Z","updated_at":"2016-03-17T17:30:52Z","body":"@rmandelb no problem - downloading these to the server now."}}