{"action":"created","issue":{"url":"https://api.github.com/repos/taviso/ctypes.sh/issues/35","repository_url":"https://api.github.com/repos/taviso/ctypes.sh","labels_url":"https://api.github.com/repos/taviso/ctypes.sh/issues/35/labels{/name}","comments_url":"https://api.github.com/repos/taviso/ctypes.sh/issues/35/comments","events_url":"https://api.github.com/repos/taviso/ctypes.sh/issues/35/events","html_url":"https://github.com/taviso/ctypes.sh/issues/35","id":161088854,"number":35,"title":"Implement automatic structure definitions","user":{"login":"taviso","id":123814,"avatar_url":"https://avatars.githubusercontent.com/u/123814?v=3","gravatar_id":"","url":"https://api.github.com/users/taviso","html_url":"https://github.com/taviso","followers_url":"https://api.github.com/users/taviso/followers","following_url":"https://api.github.com/users/taviso/following{/other_user}","gists_url":"https://api.github.com/users/taviso/gists{/gist_id}","starred_url":"https://api.github.com/users/taviso/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/taviso/subscriptions","organizations_url":"https://api.github.com/users/taviso/orgs","repos_url":"https://api.github.com/users/taviso/repos","events_url":"https://api.github.com/users/taviso/events{/privacy}","received_events_url":"https://api.github.com/users/taviso/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/taviso/ctypes.sh/labels/enhancement","name":"enhancement","color":"84b6eb"}],"state":"open","locked":false,"assignee":null,"milestone":{"url":"https://api.github.com/repos/taviso/ctypes.sh/milestones/2","html_url":"https://github.com/taviso/ctypes.sh/milestones/1.1-RELEASE","labels_url":"https://api.github.com/repos/taviso/ctypes.sh/milestones/2/labels","id":1836881,"number":2,"title":"1.1-RELEASE","description":null,"creator":{"login":"taviso","id":123814,"avatar_url":"https://avatars.githubusercontent.com/u/123814?v=3","gravatar_id":"","url":"https://api.github.com/users/taviso","html_url":"https://github.com/taviso","followers_url":"https://api.github.com/users/taviso/followers","following_url":"https://api.github.com/users/taviso/following{/other_user}","gists_url":"https://api.github.com/users/taviso/gists{/gist_id}","starred_url":"https://api.github.com/users/taviso/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/taviso/subscriptions","organizations_url":"https://api.github.com/users/taviso/orgs","repos_url":"https://api.github.com/users/taviso/repos","events_url":"https://api.github.com/users/taviso/events{/privacy}","received_events_url":"https://api.github.com/users/taviso/received_events","type":"User","site_admin":false},"open_issues":1,"closed_issues":0,"state":"open","created_at":"2016-06-19T19:51:25Z","updated_at":"2016-06-19T19:54:55Z","due_on":null,"closed_at":null},"comments":2,"created_at":"2016-06-19T19:54:55Z","updated_at":"2016-06-20T17:45:19Z","closed_at":null,"body":"Let's fix the ugly problem of accessing system structures for version 1.1.\r\n\r\nIn issue #27 @cemeyer suggests automatic creation of standard structures by parsing the dwarf data. I think that could work.\r\n\r\nThe utility pahole from the [dwarves](https://git.kernel.org/cgit/devel/pahole/pahole.git/)package almost does what we want. Here's an example of pahole output:\r\n\r\n```\r\n$ ./pahole -C stat /usr/lib/debug/lib64/libc.so.6.debug \r\nstruct stat {\r\n\t__dev_t                    st_dev;               /*     0     8 */\r\n\t__ino_t                    st_ino;               /*     8     8 */\r\n\t__nlink_t                  st_nlink;             /*    16     8 */\r\n\t__mode_t                   st_mode;              /*    24     4 */\r\n\t__uid_t                    st_uid;               /*    28     4 */\r\n\t__gid_t                    st_gid;               /*    32     4 */\r\n\tint                        __pad0;               /*    36     4 */\r\n\t__dev_t                    st_rdev;              /*    40     8 */\r\n\t__off_t                    st_size;              /*    48     8 */\r\n\t__blksize_t                st_blksize;           /*    56     8 */\r\n\t/* --- cacheline 1 boundary (64 bytes) --- */\r\n\t__blkcnt_t                 st_blocks;            /*    64     8 */\r\n\tstruct timespec            st_atim;              /*    72    16 */\r\n\tstruct timespec            st_mtim;              /*    88    16 */\r\n\tstruct timespec            st_ctim;              /*   104    16 */\r\n\t__syscall_slong_t          __glibc_reserved[3];  /*   120    24 */\r\n\t/* --- cacheline 2 boundary (128 bytes) was 16 bytes ago --- */\r\n\r\n\t/* size: 144, cachelines: 3, members: 15 */\r\n\t/* last cacheline: 16 bytes */\r\n};\r\n```\r\n\r\nit can also expand types into primitive types (like long, int, etc) with --expand-types. I think if we import it we can make builtin command `struct` that does something like this:\r\n\r\n```\r\n# make foo an array of the right size and type prefixes populated correctly\r\n$ struct stat foo \r\n$ echo ${foo[@]}\r\nlong long unsigned unsigned int ..etc\r\n\r\n# struct command can also define a bunch of integers like _structname_membername so this works:\r\n$ echo ${foo[_stat_st_uid]}\r\nint:0\r\n$ echo ${foo[_stat_sts_size]}\r\nlong:8192\r\n\r\n```\r\n\r\nI quickly converted pahole to a builtin and I think it's going to work:\r\n\r\n```\r\n$ enable -f .libs/ctypes.so struct\r\n$ struct stat /usr/lib/debug/usr/lib64/libc-2.22.so.debug\r\nstruct stat {\r\n\tlong unsigned int          st_dev;                                               /*     0     8 */\r\n\tlong unsigned int          st_ino;                                               /*     8     8 */\r\n\tlong unsigned int          st_nlink;                                             /*    16     8 */\r\n\tunsigned int               st_mode;                                              /*    24     4 */\r\n\tunsigned int               st_uid;                                               /*    28     4 */\r\n\tunsigned int               st_gid;                                               /*    32     4 */\r\n\tint                        __pad0;                                               /*    36     4 */\r\n\tlong unsigned int          st_rdev;                                              /*    40     8 */\r\n\tlong int                   st_size;                                              /*    48     8 */\r\n\tlong int                   st_blksize;                                           /*    56     8 */\r\n\tlong int                   st_blocks;                                            /*    64     8 */\r\n\tstruct timespec {\r\n\t\tlong int           tv_sec;                                               /*    72     8 */\r\n\t\tlong int           tv_nsec;                                              /*    80     8 */\r\n\t} st_atim; /*    72    16 */\r\n\tstruct timespec {\r\n\t\tlong int           tv_sec;                                               /*    88     8 */\r\n\t\tlong int           tv_nsec;                                              /*    96     8 */\r\n\t} st_mtim; /*    88    16 */\r\n\tstruct timespec {\r\n\t\tlong int           tv_sec;                                               /*   104     8 */\r\n\t\tlong int           tv_nsec;                                              /*   112     8 */\r\n\t} st_ctim; /*   104    16 */\r\n\tlong int                   __glibc_reserved[3];                                  /*   120    24 */\r\n};\r\n$ type struct\r\nstruct is a shell builtin\r\n\r\n```\r\n\r\nSo I just need to use `bind_array_variable` instead of printing out the types and I think it will work. We will also need a sizeof builtin, then a script like this should work:\r\n\r\n```bash\r\n#!/bin/bash\r\nsource ctypes.sh\r\n\r\n# create a bash version of the stat structure\r\nstruct stat passwd\r\n\r\n# Allocate some space for native stat buffer\r\ndlcall -n statbuf -r pointer malloc $(sizeof stat)\r\n\r\n# Call stat()\r\ndlcall -r int stat \"/etc/passwd\" $statbuf\r\n\r\n# parse the native struct into bash struct\r\nunpack $statbuf passwd\r\n\r\n# dont need the native version any more\r\ndlcall free $statbuf\r\n\r\nprintf \"/etc/passwd\\n\"\r\nprintf \"\\tuid:  %u\\n\" ${passwd[st_uid]##*:}\r\nprintf \"\\tgid:  %u\\n\" ${passwd[st_gid]##*:}\r\nprintf \"\\tmode: %o\\n\" ${passwd[st_mode]##*:}\r\nprintf \"\\tsize: %u\\n\" ${passwd[st_size]##*:}\r\n\r\n```\r\n\r\nSo TODO:\r\n- Parse .gnu_debuglink so user doesn't have to specify any files\r\n- Automatically check all dlopen'd libraries for a matching structure name\r\n- Add a sizeof builtin so user can find out the size of a structure when using malloc\r\n- Find out if this approach is going to work on FreeBSD @cemeyer ?\r\n\r\nWe can't use associative arrays for this (unfortunately) because internally bash stores associative arrays as hash tables and doesn't maintain the order of elements, so we will need to define the indexes. \r\n\r\nThis means we can't make `${stat[st_size]}` work without defining the variable `$st_size`. It would be nice if we could do stat[stat.st_size], but I don't think bash will parse that correctly. I think the best we can do is:\r\n\r\n`echo ${stat[_stat_st_size]}`\r\n\r\nBut I think I can live with that.\r\n "},"comment":{"url":"https://api.github.com/repos/taviso/ctypes.sh/issues/comments/227215648","html_url":"https://github.com/taviso/ctypes.sh/issues/35#issuecomment-227215648","issue_url":"https://api.github.com/repos/taviso/ctypes.sh/issues/35","id":227215648,"user":{"login":"cemeyer","id":124738,"avatar_url":"https://avatars.githubusercontent.com/u/124738?v=3","gravatar_id":"","url":"https://api.github.com/users/cemeyer","html_url":"https://github.com/cemeyer","followers_url":"https://api.github.com/users/cemeyer/followers","following_url":"https://api.github.com/users/cemeyer/following{/other_user}","gists_url":"https://api.github.com/users/cemeyer/gists{/gist_id}","starred_url":"https://api.github.com/users/cemeyer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cemeyer/subscriptions","organizations_url":"https://api.github.com/users/cemeyer/orgs","repos_url":"https://api.github.com/users/cemeyer/repos","events_url":"https://api.github.com/users/cemeyer/events{/privacy}","received_events_url":"https://api.github.com/users/cemeyer/received_events","type":"User","site_admin":false},"created_at":"2016-06-20T17:45:19Z","updated_at":"2016-06-20T17:45:19Z","body":"> Find out if this approach is going to work on FreeBSD @cemeyer ?\r\n\r\nWe have DWARF debuginfo.  But binaries are stripped by default.  I don't know that this will work with stripped binaries, but it ought to work with unstripped ones.  (We also don't have a port (package) of libdwarves yet, but that can be fixed.)"}}
{"actor":{"display_login":"cemeyer"}}