{"action":"created","issue":{"url":"https://api.github.com/repos/SeisComP3/seiscomp3/issues/172","repository_url":"https://api.github.com/repos/SeisComP3/seiscomp3","labels_url":"https://api.github.com/repos/SeisComP3/seiscomp3/issues/172/labels{/name}","comments_url":"https://api.github.com/repos/SeisComP3/seiscomp3/issues/172/comments","events_url":"https://api.github.com/repos/SeisComP3/seiscomp3/issues/172/events","html_url":"https://github.com/SeisComP3/seiscomp3/issues/172","id":318553316,"number":172,"title":"Arclink server: fails on large RESPONSE requests","user":{"login":"petrrr","id":3065632,"avatar_url":"https://avatars0.githubusercontent.com/u/3065632?v=4","gravatar_id":"","url":"https://api.github.com/users/petrrr","html_url":"https://github.com/petrrr","followers_url":"https://api.github.com/users/petrrr/followers","following_url":"https://api.github.com/users/petrrr/following{/other_user}","gists_url":"https://api.github.com/users/petrrr/gists{/gist_id}","starred_url":"https://api.github.com/users/petrrr/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/petrrr/subscriptions","organizations_url":"https://api.github.com/users/petrrr/orgs","repos_url":"https://api.github.com/users/petrrr/repos","events_url":"https://api.github.com/users/petrrr/events{/privacy}","received_events_url":"https://api.github.com/users/petrrr/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2018-04-27T21:20:04Z","updated_at":"2018-05-02T14:15:05Z","closed_at":null,"author_association":"NONE","body":"After some debugging on a user's problem who was not able to request metadata in dataless format, we conclude that the problem is likely due to a server bug and to the size of the underlying request, which causes a unexpected error server side.\r\n\r\nOn one specific request we got the following message:\r\n```\r\nvolume ID: dataless, dcid: INGV, Status: ERROR, Size: 0, Encrypted: False, Info: blockette 41 has invalid length: 1207 instead of 1206\r\n```\r\n\r\nWe first were trying to identify one particular station which might generated this issue but were non able to identify one. Rather we conclude that the problem is no specific to any Inventory records related to one station. Instead we see a causal relation by the size of the request, presumably due to some size limit or some sort of buffer overflow.\r\n\r\n### Details\r\n\r\nThe following request (reduced from larger one) was originally found to fail. It also fails against the GFZ node server. \r\n\r\n```\r\narclink_fetch -a eida.ingv.it:18002  -u test@ingv.it -vvvv  -k dseed -o  test  request.compromised\r\n```\r\n* [request.compromised.zip](https://github.com/SeisComP3/seiscomp3/files/1956666/request.compromised.zip)\r\n\r\nHere some evidence why we believe the size is relevant for this issue:\r\n\r\n```\r\nsysop@eida:~/tmp/debug-response-merge$ cat request.compromized | head -n 822 > request.test; arclink_fetch -a eida.ingv.it:18002  -u test@ingv.it -vvvv  -k dseed -o  test  request.test\r\nWarnings detected on the command line:\r\n    Default password file (dcidpasswords.txt) not found\r\n\r\nrequesting response data from eida.ingv.it:18002\r\n\r\nthe following data requests were sent:\r\n datacenter name: Italian Seismic Data Center\r\n request ID: 20912630, Label: , Type: RESPONSE, Encrypted: False, Args: resp_dict=true compression=bzip2\r\n status: READY, Size: 156887, Info: \r\n     volume ID: dataless, dcid: INGV, Status: OK, Size: 156887, Encrypted: False, Info: \r\n         request: 2018,4,11,4,41,22 2018,4,11,4,46,52 Z3 A001A HHZ 00\r\n         status: OK, Size: 0, Info: \r\n         request: 2018,4,11,4,41,22 2018,4,11,4,46,52 Z3 A002B HHZ 00\r\n         status: OK, Size: 0, Info: \r\n[...]\r\n         request: 2018,4,11,4,41,22 2018,4,11,4,46,52 IV APRC HHN .\r\n         status: OK, Size: 0, Info: \r\n         request: 2018,4,11,4,41,22 2018,4,11,4,46,52 FR ARBF HHZ 00\r\n         status: OK, Size: 0, Info: \r\n\r\nsaved file: test.20912630.dataless\r\n```\r\nvs.\r\n```\r\nsysop@eida:~/tmp/debug-response-merge$ cat request.compromized | head -n 825 > request.test; arclink_fetch -a eida.ingv.it:18002  -u test@ingv.it -vvvv  -k dseed -o  test  request.test\r\nWarnings detected on the command line:\r\n    Default password file (dcidpasswords.txt) not found\r\n\r\nrequesting response data from eida.ingv.it:18002\r\neida.ingv.it:18002: request 20912658 returned no data (no downloadable volumes found)\r\n\r\nthe following data requests were sent:\r\n datacenter name: Italian Seismic Data Center\r\n request ID: 20912658, Label: , Type: RESPONSE, Encrypted: False, Args: resp_dict=true compression=bzip2\r\n status: READY, Size: 0, Info: \r\n     volume ID: dataless, dcid: INGV, Status: ERROR, Size: 0, Encrypted: False, Info: blockette 41 has invalid length: 1207 instead of 1206\r\n         request: 2018,4,11,4,41,22 2018,4,11,4,46,52 Z3 A001A HHZ 00\r\n         status: OK, Size: 0, Info: \r\n         request: 2018,4,11,4,41,22 2018,4,11,4,46,52 Z3 A002B HHZ 00\r\n         status: OK, Size: 0, Info: \r\n\r\n[...]\r\n         request: 2018,4,11,4,41,22 2018,4,11,4,46,52 FR ARBF HHE 00\r\n         status: OK, Size: 0, Info: \r\n\r\nnone of the requests returned data\r\n```\r\n\r\nand the related log from `arclink.log`:\r\n\r\n```\r\n﻿﻿Fri Apr 27 20:27:52 2018 - arclink: [27978] STATUS LINE 823 PROCESSING dataless\r\nFri Apr 27 20:27:52 2018 - arclink: [27978] STATUS LINE 823 OK\r\nFri Apr 27 20:27:52 2018 - arclink: [27978] STATUS LINE 824 PROCESSING dataless\r\nFri Apr 27 20:27:52 2018 - arclink: [27978] STATUS LINE 824 OK\r\nFri Apr 27 20:27:54 2018 - arclink: [27978] STATUS VOLUME dataless MESSAGE blockette 41 has invalid length: 1207 instead of 1206\r\nFri Apr 27 20:27:54 2018 - arclink: [27978] STATUS VOLUME dataless ERROR\r\nFri Apr 27 20:27:54 2018 - arclink: [27978] END\r\n```\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/SeisComP3/seiscomp3/issues/comments/385992536","html_url":"https://github.com/SeisComP3/seiscomp3/issues/172#issuecomment-385992536","issue_url":"https://api.github.com/repos/SeisComP3/seiscomp3/issues/172","id":385992536,"user":{"login":"petrrr","id":3065632,"avatar_url":"https://avatars0.githubusercontent.com/u/3065632?v=4","gravatar_id":"","url":"https://api.github.com/users/petrrr","html_url":"https://github.com/petrrr","followers_url":"https://api.github.com/users/petrrr/followers","following_url":"https://api.github.com/users/petrrr/following{/other_user}","gists_url":"https://api.github.com/users/petrrr/gists{/gist_id}","starred_url":"https://api.github.com/users/petrrr/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/petrrr/subscriptions","organizations_url":"https://api.github.com/users/petrrr/orgs","repos_url":"https://api.github.com/users/petrrr/repos","events_url":"https://api.github.com/users/petrrr/events{/privacy}","received_events_url":"https://api.github.com/users/petrrr/received_events","type":"User","site_admin":false},"created_at":"2018-05-02T14:15:05Z","updated_at":"2018-05-02T14:15:05Z","author_association":"NONE","body":"@andres-h: I can confirm that using the `-n` option works around the problem reported here, i.e. for a request which is send as a single arclink request.\r\n\r\nHowever, when going back to the initial problem (the one reported by the user), we still observe the original problem, i.e. the merge of the volumes fails. That one regards a larger request with 4639 lines, which is apparently split client side. \r\n\r\n```\r\narclink_fetch -a eida.ingv.it:18002  -u test@ingv.it -vvvv  -k dseed -n -o test-all arclink_request.txt\r\n\r\nWarnings detected on the command line:\r\n        Default password file (dcidpasswords.txt) not found\r\n\r\nrequesting response data from eida.ingv.it:18002\r\n\r\nthe following data requests were sent:\r\n datacenter name: Italian Seismic Data Center\r\n request ID: 20984682, Label: , Type: RESPONSE, Encrypted: False, Args: resp_dict=false compression=bzip2\r\n status: READY, Size: 149592, Info: \r\n     volume ID: dataless, dcid: INGV, Status: OK, Size: 149592, Encrypted: False, Info: \r\n         request: 2018,4,11,4,41,22 2018,4,11,4,46,52 Z3 A001A HHZ 00\r\n         status: OK, Size: 0, Info: \r\n[...]\r\ncannot merge volumes saving volumes as individual files\r\nsaved file: test-all.20984682.dataless\r\nsaved file: test-all.20984685.dataless\r\nsaved file: test-all.20984687.dataless\r\nsaved file: test-all.20984696.dataless\r\nsaved file: test-all.20984698.dataless\r\n```\r\n\r\nI assumed that the failing request was causal of the merge failure, but the `-n` option does not fix that problem. But this seems to be a different problem and related to the client, so should I open a different issue?\r\n\r\n"}}
{"actor":{"display_login":"petrrr"}}