{"action":"created","issue":{"url":"https://api.github.com/repos/kaitai-io/kaitai_struct/issues/9","repository_url":"https://api.github.com/repos/kaitai-io/kaitai_struct","labels_url":"https://api.github.com/repos/kaitai-io/kaitai_struct/issues/9/labels{/name}","comments_url":"https://api.github.com/repos/kaitai-io/kaitai_struct/issues/9/comments","events_url":"https://api.github.com/repos/kaitai-io/kaitai_struct/issues/9/events","html_url":"https://github.com/kaitai-io/kaitai_struct/issues/9","id":166752838,"number":9,"title":"Bit field data types","user":{"login":"Tyilo","id":568036,"avatar_url":"https://avatars.githubusercontent.com/u/568036?v=3","gravatar_id":"","url":"https://api.github.com/users/Tyilo","html_url":"https://github.com/Tyilo","followers_url":"https://api.github.com/users/Tyilo/followers","following_url":"https://api.github.com/users/Tyilo/following{/other_user}","gists_url":"https://api.github.com/users/Tyilo/gists{/gist_id}","starred_url":"https://api.github.com/users/Tyilo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Tyilo/subscriptions","organizations_url":"https://api.github.com/users/Tyilo/orgs","repos_url":"https://api.github.com/users/Tyilo/repos","events_url":"https://api.github.com/users/Tyilo/events{/privacy}","received_events_url":"https://api.github.com/users/Tyilo/received_events","type":"User","site_admin":false},"labels":[{"id":329236478,"url":"https://api.github.com/repos/kaitai-io/kaitai_struct/labels/enhancement","name":"enhancement","color":"84b6eb","default":true}],"state":"open","locked":false,"assignee":{"login":"GreyCat","id":157154,"avatar_url":"https://avatars.githubusercontent.com/u/157154?v=3","gravatar_id":"","url":"https://api.github.com/users/GreyCat","html_url":"https://github.com/GreyCat","followers_url":"https://api.github.com/users/GreyCat/followers","following_url":"https://api.github.com/users/GreyCat/following{/other_user}","gists_url":"https://api.github.com/users/GreyCat/gists{/gist_id}","starred_url":"https://api.github.com/users/GreyCat/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/GreyCat/subscriptions","organizations_url":"https://api.github.com/users/GreyCat/orgs","repos_url":"https://api.github.com/users/GreyCat/repos","events_url":"https://api.github.com/users/GreyCat/events{/privacy}","received_events_url":"https://api.github.com/users/GreyCat/received_events","type":"User","site_admin":false},"assignees":[{"login":"GreyCat","id":157154,"avatar_url":"https://avatars.githubusercontent.com/u/157154?v=3","gravatar_id":"","url":"https://api.github.com/users/GreyCat","html_url":"https://github.com/GreyCat","followers_url":"https://api.github.com/users/GreyCat/followers","following_url":"https://api.github.com/users/GreyCat/following{/other_user}","gists_url":"https://api.github.com/users/GreyCat/gists{/gist_id}","starred_url":"https://api.github.com/users/GreyCat/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/GreyCat/subscriptions","organizations_url":"https://api.github.com/users/GreyCat/orgs","repos_url":"https://api.github.com/users/GreyCat/repos","events_url":"https://api.github.com/users/GreyCat/events{/privacy}","received_events_url":"https://api.github.com/users/GreyCat/received_events","type":"User","site_admin":false}],"milestone":null,"comments":22,"created_at":"2016-07-21T07:16:02Z","updated_at":"2016-12-12T11:43:29Z","closed_at":null,"body":"Sometimes a file format doesn't use whole bytes for every field, so it would be nice to support bit fields. Something like `b[1-64]{le,be}`.\n"},"comment":{"url":"https://api.github.com/repos/kaitai-io/kaitai_struct/issues/comments/266408081","html_url":"https://github.com/kaitai-io/kaitai_struct/issues/9#issuecomment-266408081","issue_url":"https://api.github.com/repos/kaitai-io/kaitai_struct/issues/9","id":266408081,"user":{"login":"GreyCat","id":157154,"avatar_url":"https://avatars.githubusercontent.com/u/157154?v=3","gravatar_id":"","url":"https://api.github.com/users/GreyCat","html_url":"https://github.com/GreyCat","followers_url":"https://api.github.com/users/GreyCat/followers","following_url":"https://api.github.com/users/GreyCat/following{/other_user}","gists_url":"https://api.github.com/users/GreyCat/gists{/gist_id}","starred_url":"https://api.github.com/users/GreyCat/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/GreyCat/subscriptions","organizations_url":"https://api.github.com/users/GreyCat/orgs","repos_url":"https://api.github.com/users/GreyCat/repos","events_url":"https://api.github.com/users/GreyCat/events{/privacy}","received_events_url":"https://api.github.com/users/GreyCat/received_events","type":"User","site_admin":false},"created_at":"2016-12-12T11:43:29Z","updated_at":"2016-12-12T11:43:29Z","body":"Ok, I guess we're more or less confident with the proposed interface, let's talk about implementation.\r\n\r\nFrom implementation point of view, I see two very distinct (and probably complimentary) options:\r\n\r\n(1) We add more call to runtimes: something like `read_bits(n)` and `read_bitset(n)`, which read `n` bits and return result either as integer, or as bitset. This is unaligned reading, so it obviously requires storing some additional state in the KaitaiStream object (i.e. index of current bit or current bitmask). It is easy to implement from compiler's POV, as it's the same one-to-one mapping. Stuff like that:\r\n\r\n```yaml\r\nseq:\r\n  - id: x\r\n    type: b4\r\n  - id: y\r\n    type: b3\r\n  - id: z\r\n    type: b1\r\n```\r\n\r\nwould compile to 3 distinct statements:\r\n\r\n```java\r\nx = _io.readBits(4);\r\ny = _io.readBits(3);\r\nz = _io.readBits(1);\r\n```\r\n\r\nIt's simple, straightforward and it works in general case. However, it's not as optimal for instances-based approach we have right now. Reading structures with big and complex flag fields (like, 10-20-30 flags) is probably much more efficient with current instances approach: we just need to read several bytes, remember then and effectively skip over them, and we'll get to decipher individual flag values only on demand. In real world tasks with big flag fields you rarely need all of them at once, you usually need only 1 or 2, and instances-based approach allows you to do precisely that: read flags as bytes and then extract only a few individual flag values.\r\n\r\nSo, we come to alternative approach.\r\n\r\n(2) We don't touch anything with language runtimes and implement byte-aligned bit reads using existing tools. That means that we're gathering groups of bits on pre-processing stage, and generate flag-extracting instances automatically, i.e. previous example becomes something like that:\r\n\r\n```java\r\nprivate _read() {\r\n    _bf1 = _io.readU1();\r\n}\r\n\r\npublic int x() {\r\n    if (x != null)\r\n        return x;\r\n    x = (_bf1 & 240) >> 4;\r\n    return x;\r\n}\r\n\r\npublic int y() {\r\n    if (y != null)\r\n        return y;\r\n    y = (_bf1 & 14) >> 1;\r\n    return y;\r\n}\r\n\r\npublic int z() {\r\n    if (z != null)\r\n        return z;\r\n    z = _bf1 & 1;\r\n    return z;\r\n}\r\n```\r\n\r\nSo, what do you think of these two approaches? I guess we should implement both (probably preferring second approach where it's possible — i.e. when we have fixed bit offsets), but which shall we implement first? Both would be kind of difficult, as approach (1) requires lots of modifications in every runtime, and (2) requires some magic within the compiler to gather the bitfields."}}
{"actor":{"display_login":"GreyCat"}}