{"action":"created","issue":{"url":"https://api.github.com/repos/SuperDARN/rst/issues/165","repository_url":"https://api.github.com/repos/SuperDARN/rst","labels_url":"https://api.github.com/repos/SuperDARN/rst/issues/165/labels{/name}","comments_url":"https://api.github.com/repos/SuperDARN/rst/issues/165/comments","events_url":"https://api.github.com/repos/SuperDARN/rst/issues/165/events","html_url":"https://github.com/SuperDARN/rst/pull/165","id":337561934,"node_id":"MDExOlB1bGxSZXF1ZXN0MTk4NzEyMjI2","number":165,"title":"Add warning message for bad num_bytes","user":{"login":"ksterne","id":3009489,"node_id":"MDQ6VXNlcjMwMDk0ODk=","avatar_url":"https://avatars2.githubusercontent.com/u/3009489?v=4","gravatar_id":"","url":"https://api.github.com/users/ksterne","html_url":"https://github.com/ksterne","followers_url":"https://api.github.com/users/ksterne/followers","following_url":"https://api.github.com/users/ksterne/following{/other_user}","gists_url":"https://api.github.com/users/ksterne/gists{/gist_id}","starred_url":"https://api.github.com/users/ksterne/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ksterne/subscriptions","organizations_url":"https://api.github.com/users/ksterne/orgs","repos_url":"https://api.github.com/users/ksterne/repos","events_url":"https://api.github.com/users/ksterne/events{/privacy}","received_events_url":"https://api.github.com/users/ksterne/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":39,"created_at":"2018-07-02T15:19:49Z","updated_at":"2018-11-16T19:01:37Z","closed_at":null,"author_association":"CONTRIBUTOR","pull_request":{"url":"https://api.github.com/repos/SuperDARN/rst/pulls/165","html_url":"https://github.com/SuperDARN/rst/pull/165","diff_url":"https://github.com/SuperDARN/rst/pull/165.diff","patch_url":"https://github.com/SuperDARN/rst/pull/165.patch"},"body":"This is a follow-on from #139 though it doesn't relate to original issue that it fixed in #116.  I'm catching up to where some of the discussion was and realized that the last record is supposed to return `num_byte` as zero.  So I've added in a check here to check for that value being less than zero.  My examples for tests here are:\r\n```\r\nmake_fit -old 2001102410w.dat w_fit\r\ncmpraw 2001102410w.dat 2001102412w.dat\r\n```\r\nI understand the `2001102410w.dat` is likely corrupted, but adding this kind of warning message (I've been debating upgrading it to an error) will help in detecting problem files in the future.  The `develop` branch though will end the `make_fit` call with a `File not found.` which leaves a user (and some light developers) to believe they might have made a typo."},"comment":{"url":"https://api.github.com/repos/SuperDARN/rst/issues/comments/439493411","html_url":"https://github.com/SuperDARN/rst/pull/165#issuecomment-439493411","issue_url":"https://api.github.com/repos/SuperDARN/rst/issues/165","id":439493411,"node_id":"MDEyOklzc3VlQ29tbWVudDQzOTQ5MzQxMQ==","user":{"login":"ksterne","id":3009489,"node_id":"MDQ6VXNlcjMwMDk0ODk=","avatar_url":"https://avatars2.githubusercontent.com/u/3009489?v=4","gravatar_id":"","url":"https://api.github.com/users/ksterne","html_url":"https://github.com/ksterne","followers_url":"https://api.github.com/users/ksterne/followers","following_url":"https://api.github.com/users/ksterne/following{/other_user}","gists_url":"https://api.github.com/users/ksterne/gists{/gist_id}","starred_url":"https://api.github.com/users/ksterne/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ksterne/subscriptions","organizations_url":"https://api.github.com/users/ksterne/orgs","repos_url":"https://api.github.com/users/ksterne/repos","events_url":"https://api.github.com/users/ksterne/events{/privacy}","received_events_url":"https://api.github.com/users/ksterne/received_events","type":"User","site_admin":false},"created_at":"2018-11-16T19:01:37Z","updated_at":"2018-11-16T19:01:37Z","author_association":"CONTRIBUTOR","body":"@mts299, yea, the `-old` should work well with dat files.  So, should try what I mentioned before on recompiling and such.\r\n\r\nI feel as though the error vs. warning part is getting to be a little over my head here since I'm not as familiar with the older data and don't work with it very often.  Maybe @egthomas can help out here with how this code behaves with older data.\r\n\r\nKeep in mind, we've *very* likely identified a corrupt file that should be not in distribution with the `2001102410w.dat` file.  I went back and look at some of the source of all of this in #116.  The original problem came from some random files where the reading at the end of the file wasn't working well.  That's been cleaned up with #139.  Since I was looking through #116, I did pull one of the files there and tried `make_fitex -old 1994010100k.dat k_fitex` (note a change in your command what has the '>').  The result was a decent sized file.  So, that does work here.  There's also no file that's generated with `raw_cp 2001102410w.dat` unless you do a `>` redirection.\r\n\r\nI get these are small spot checks, but I don't have the time here or think it's worth it to try to run down all of the cases with older data that I'm not familiar with.  I really had found one specific file which is corrupted due to what could be a random issue and thought this PR would help identify any corrupted files in the future.  I get it would be better to run this across all pre-20060701 data for all of the binaries which are effected, but I just don't think that is worth the effort and I don't have the time to create that kind of testing code.  If that is the case, I'll close the pull request and remove the branch since I think it's getting down a dead end on a good total solution.\r\n\r\nAnd I know I'm not taking it more than face value and I'm not taking anything to heart by closing and removing here.  I thought this would be a fairly simple and it's scope-creeped a little (though I think for the good and appreciate the guidance on it!) but I don't know enough of these routines to probably make the best solution.\r\n\r\n"}}
{"actor":{"display_login":"ksterne"}}