{"action":"created","issue":{"url":"https://api.github.com/repos/dtcooper/python-fitparse/issues/75","repository_url":"https://api.github.com/repos/dtcooper/python-fitparse","labels_url":"https://api.github.com/repos/dtcooper/python-fitparse/issues/75/labels{/name}","comments_url":"https://api.github.com/repos/dtcooper/python-fitparse/issues/75/comments","events_url":"https://api.github.com/repos/dtcooper/python-fitparse/issues/75/events","html_url":"https://github.com/dtcooper/python-fitparse/issues/75","id":353319212,"node_id":"MDU6SXNzdWUzNTMzMTkyMTI=","number":75,"title":"New uncovered (edge-?)case : parse_string ValueError","user":{"login":"jeanbaptistemora","id":1786967,"node_id":"MDQ6VXNlcjE3ODY5Njc=","avatar_url":"https://avatars3.githubusercontent.com/u/1786967?v=4","gravatar_id":"","url":"https://api.github.com/users/jeanbaptistemora","html_url":"https://github.com/jeanbaptistemora","followers_url":"https://api.github.com/users/jeanbaptistemora/followers","following_url":"https://api.github.com/users/jeanbaptistemora/following{/other_user}","gists_url":"https://api.github.com/users/jeanbaptistemora/gists{/gist_id}","starred_url":"https://api.github.com/users/jeanbaptistemora/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jeanbaptistemora/subscriptions","organizations_url":"https://api.github.com/users/jeanbaptistemora/orgs","repos_url":"https://api.github.com/users/jeanbaptistemora/repos","events_url":"https://api.github.com/users/jeanbaptistemora/events{/privacy}","received_events_url":"https://api.github.com/users/jeanbaptistemora/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":3,"created_at":"2018-08-23T10:25:50Z","updated_at":"2018-09-18T12:46:50Z","closed_at":"2018-09-18T12:46:50Z","author_association":"NONE","body":"Hello everyone,\r\n\r\nI wanted to report a small bug - or rather an uncovered case - in the get_messages() function, and see if anyone else sees the same thing.\r\n\r\nWe have a system in production extracting ~800 fit files per day and since 15 days we have seen ~ 50 (increasing by the day) fit files with an exception : \r\n\r\n```\r\nFile \"/xxxx/garmin.py\", line 224, in build_sessions\r\n    for message in fit.get_messages(as_dict=True):\r\n  File \"/usr/local/lib/python3.5/dist-packages/fitparse/base.py\", line 450, in get_messages\r\n    message = self._parse_message()\r\n  File \"/usr/local/lib/python3.5/dist-packages/fitparse/base.py\", line 160, in _parse_message\r\n    message = self._parse_data_message(header)\r\n  File \"/usr/local/lib/python3.5/dist-packages/fitparse/base.py\", line 320, in _parse_data_message\r\n    raw_values = self._parse_raw_values_from_data_message(def_mesg)\r\n  File \"/usr/local/lib/python3.5/dist-packages/fitparse/base.py\", line 271, in _parse_raw_values_from_data_message\r\n    raw_value = base_type.parse(raw_value)\r\n  File \"/usr/local/lib/python3.5/dist-packages/fitparse/records.py\", line 336, in parse_string\r\n    end = string.index(0x00)\r\nValueError: substring not found\r\n```\r\n\r\nI'm not a specialist but I've tried to debug this and started by outputting the raw_value for these files, here are some of them : \r\n\r\nb'IM G 33 NTAP'\r\nb'S0E2 - 1h + 4*10'\r\nb'CAP Fartleck VMA'\r\nb'Basse Intensit\\xc3\\xa9'\r\nb'RF5 or Cross-Tra'\r\n\r\nI think it's user inputs, and they are confusing the parse_string() function because for some reason don't contain the 0x00 substring.\r\nIt must be a new functionality from Garmin that's incorporated in the fit files for the people who have updated their hardware ?\r\nFurthemore the mesg_type.name == 'developer_data_id' (not sure what this means though).\r\n\r\nI'm not sure how to handle these messages but if I do that in base.py:70 : \r\n```\r\n    # Otherwise, just scrub the singular value\r\n    try:\r\n        raw_value = base_type.parse(raw_value)\r\n    except:\r\n        print('Substring \"0x00\" not found, skipping value')\r\n        print(raw_value)\r\n```\r\n\r\nIt works just fine (for us), but I don't know what the consequences downstream could be if we just append the raw_value without parsing it...\r\n\r\nI would love to make a PR to incorporate the handling of this upstream but I'm not comfortable enough without having more information and feedback.\r\n\r\nDoes anyone want to chime in on this ?\r\nThanks\r\n"},"comment":{"url":"https://api.github.com/repos/dtcooper/python-fitparse/issues/comments/422378108","html_url":"https://github.com/dtcooper/python-fitparse/issues/75#issuecomment-422378108","issue_url":"https://api.github.com/repos/dtcooper/python-fitparse/issues/75","id":422378108,"node_id":"MDEyOklzc3VlQ29tbWVudDQyMjM3ODEwOA==","user":{"login":"jeanbaptistemora","id":1786967,"node_id":"MDQ6VXNlcjE3ODY5Njc=","avatar_url":"https://avatars3.githubusercontent.com/u/1786967?v=4","gravatar_id":"","url":"https://api.github.com/users/jeanbaptistemora","html_url":"https://github.com/jeanbaptistemora","followers_url":"https://api.github.com/users/jeanbaptistemora/followers","following_url":"https://api.github.com/users/jeanbaptistemora/following{/other_user}","gists_url":"https://api.github.com/users/jeanbaptistemora/gists{/gist_id}","starred_url":"https://api.github.com/users/jeanbaptistemora/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jeanbaptistemora/subscriptions","organizations_url":"https://api.github.com/users/jeanbaptistemora/orgs","repos_url":"https://api.github.com/users/jeanbaptistemora/repos","events_url":"https://api.github.com/users/jeanbaptistemora/events{/privacy}","received_events_url":"https://api.github.com/users/jeanbaptistemora/received_events","type":"User","site_admin":false},"created_at":"2018-09-18T12:46:50Z","updated_at":"2018-09-18T12:46:50Z","author_association":"NONE","body":"Sorry for the response time, I've been quite swamped.\r\n\r\nThank you so much for your answers ! @polyvertex your solution is quite more elegant than mine, I've changed our hotfix for this one, awaiting for your PR to be merged :)\r\n\r\nI'll close this issue."}}
{"actor":{"display_login":"jeanbaptistemora"}}