{"action":"created","issue":{"url":"https://api.github.com/repos/mozilla/addons-server/issues/9572","repository_url":"https://api.github.com/repos/mozilla/addons-server","labels_url":"https://api.github.com/repos/mozilla/addons-server/issues/9572/labels{/name}","comments_url":"https://api.github.com/repos/mozilla/addons-server/issues/9572/comments","events_url":"https://api.github.com/repos/mozilla/addons-server/issues/9572/events","html_url":"https://github.com/mozilla/addons-server/issues/9572","id":365889640,"node_id":"MDU6SXNzdWUzNjU4ODk2NDA=","number":9572,"title":"Validate submitted source code","user":{"login":"wagnerand","id":873258,"node_id":"MDQ6VXNlcjg3MzI1OA==","avatar_url":"https://avatars2.githubusercontent.com/u/873258?v=4","gravatar_id":"","url":"https://api.github.com/users/wagnerand","html_url":"https://github.com/wagnerand","followers_url":"https://api.github.com/users/wagnerand/followers","following_url":"https://api.github.com/users/wagnerand/following{/other_user}","gists_url":"https://api.github.com/users/wagnerand/gists{/gist_id}","starred_url":"https://api.github.com/users/wagnerand/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/wagnerand/subscriptions","organizations_url":"https://api.github.com/users/wagnerand/orgs","repos_url":"https://api.github.com/users/wagnerand/repos","events_url":"https://api.github.com/users/wagnerand/events{/privacy}","received_events_url":"https://api.github.com/users/wagnerand/received_events","type":"User","site_admin":false},"labels":[{"id":403938612,"node_id":"MDU6TGFiZWw0MDM5Mzg2MTI=","url":"https://api.github.com/repos/mozilla/addons-server/labels/component:%20devhub","name":"component: devhub","color":"107c05","default":false},{"id":353728203,"node_id":"MDU6TGFiZWwzNTM3MjgyMDM=","url":"https://api.github.com/repos/mozilla/addons-server/labels/priority:%20p2","name":"priority: p2","color":"d70022","default":false},{"id":540035677,"node_id":"MDU6TGFiZWw1NDAwMzU2Nzc=","url":"https://api.github.com/repos/mozilla/addons-server/labels/triaged","name":"triaged","color":"ededed","default":false}],"state":"closed","locked":false,"assignee":{"login":"diox","id":187006,"node_id":"MDQ6VXNlcjE4NzAwNg==","avatar_url":"https://avatars0.githubusercontent.com/u/187006?v=4","gravatar_id":"","url":"https://api.github.com/users/diox","html_url":"https://github.com/diox","followers_url":"https://api.github.com/users/diox/followers","following_url":"https://api.github.com/users/diox/following{/other_user}","gists_url":"https://api.github.com/users/diox/gists{/gist_id}","starred_url":"https://api.github.com/users/diox/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/diox/subscriptions","organizations_url":"https://api.github.com/users/diox/orgs","repos_url":"https://api.github.com/users/diox/repos","events_url":"https://api.github.com/users/diox/events{/privacy}","received_events_url":"https://api.github.com/users/diox/received_events","type":"User","site_admin":false},"assignees":[{"login":"diox","id":187006,"node_id":"MDQ6VXNlcjE4NzAwNg==","avatar_url":"https://avatars0.githubusercontent.com/u/187006?v=4","gravatar_id":"","url":"https://api.github.com/users/diox","html_url":"https://github.com/diox","followers_url":"https://api.github.com/users/diox/followers","following_url":"https://api.github.com/users/diox/following{/other_user}","gists_url":"https://api.github.com/users/diox/gists{/gist_id}","starred_url":"https://api.github.com/users/diox/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/diox/subscriptions","organizations_url":"https://api.github.com/users/diox/orgs","repos_url":"https://api.github.com/users/diox/repos","events_url":"https://api.github.com/users/diox/events{/privacy}","received_events_url":"https://api.github.com/users/diox/received_events","type":"User","site_admin":false}],"milestone":{"url":"https://api.github.com/repos/mozilla/addons-server/milestones/135","html_url":"https://github.com/mozilla/addons-server/milestone/135","labels_url":"https://api.github.com/repos/mozilla/addons-server/milestones/135/labels","id":3762492,"node_id":"MDk6TWlsZXN0b25lMzc2MjQ5Mg==","number":135,"title":"2018.11.15","description":"","creator":{"login":"muffinresearch","id":1514,"node_id":"MDQ6VXNlcjE1MTQ=","avatar_url":"https://avatars1.githubusercontent.com/u/1514?v=4","gravatar_id":"","url":"https://api.github.com/users/muffinresearch","html_url":"https://github.com/muffinresearch","followers_url":"https://api.github.com/users/muffinresearch/followers","following_url":"https://api.github.com/users/muffinresearch/following{/other_user}","gists_url":"https://api.github.com/users/muffinresearch/gists{/gist_id}","starred_url":"https://api.github.com/users/muffinresearch/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/muffinresearch/subscriptions","organizations_url":"https://api.github.com/users/muffinresearch/orgs","repos_url":"https://api.github.com/users/muffinresearch/repos","events_url":"https://api.github.com/users/muffinresearch/events{/privacy}","received_events_url":"https://api.github.com/users/muffinresearch/received_events","type":"User","site_admin":false},"open_issues":1,"closed_issues":15,"state":"open","created_at":"2018-10-23T16:57:49Z","updated_at":"2018-11-13T19:25:53Z","due_on":"2018-11-13T08:00:00Z","closed_at":null},"comments":10,"created_at":"2018-10-02T13:09:23Z","updated_at":"2018-11-14T10:44:24Z","closed_at":"2018-11-13T11:40:59Z","author_association":"MEMBER","body":"https://github.com/mozilla/addons-server/blob/d71d63cc1e350ddebe495cda6ea4247edcb35414/src/olympia/versions/models.py#L42-L44 allows different file formats for uploading source code.\r\n\r\nWe rely on them being valid for doing code-search (and review of course), but we don't do any kind of validation (other than checking the file extension). The result is, that there are some files either being corrupt/incomplete or have the wrong file extension (presumably to make the upload pass), but in fact use an unsupported compression algorithm.\r\n\r\nTherefore, we need to add some validation to uploaded source code files:\r\n* Check the compression algorithm, for example using the file signature (magic number), instead of just the file extension and\r\n* Verify the archive is valid.\r\n\r\nLooking into [pyunpack](https://github.com/ponty/pyunpack) might be worthwhile.\r\n\r\nJustification for P2: The daily code-extraction script that provides the data for code-search relies on valid files and usage of one of the supported compression algorithms above. We recently mitigated some issues, but we cannot anticipate or support every potential compression algorithm."},"comment":{"url":"https://api.github.com/repos/mozilla/addons-server/issues/comments/438618818","html_url":"https://github.com/mozilla/addons-server/issues/9572#issuecomment-438618818","issue_url":"https://api.github.com/repos/mozilla/addons-server/issues/9572","id":438618818,"node_id":"MDEyOklzc3VlQ29tbWVudDQzODYxODgxOA==","user":{"login":"AlexandraMoga","id":31961530,"node_id":"MDQ6VXNlcjMxOTYxNTMw","avatar_url":"https://avatars3.githubusercontent.com/u/31961530?v=4","gravatar_id":"","url":"https://api.github.com/users/AlexandraMoga","html_url":"https://github.com/AlexandraMoga","followers_url":"https://api.github.com/users/AlexandraMoga/followers","following_url":"https://api.github.com/users/AlexandraMoga/following{/other_user}","gists_url":"https://api.github.com/users/AlexandraMoga/gists{/gist_id}","starred_url":"https://api.github.com/users/AlexandraMoga/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AlexandraMoga/subscriptions","organizations_url":"https://api.github.com/users/AlexandraMoga/orgs","repos_url":"https://api.github.com/users/AlexandraMoga/repos","events_url":"https://api.github.com/users/AlexandraMoga/events{/privacy}","received_events_url":"https://api.github.com/users/AlexandraMoga/received_events","type":"User","site_admin":false},"created_at":"2018-11-14T10:44:24Z","updated_at":"2018-11-14T10:44:24Z","author_association":"NONE","body":"In that case, thing seem to work as expected, so I'm marking this issue as verified fixed. "}}
{"actor":{"display_login":"AlexandraMoga"}}