{"action":"created","issue":{"url":"https://api.github.com/repos/golang/go/issues/16051","repository_url":"https://api.github.com/repos/golang/go","labels_url":"https://api.github.com/repos/golang/go/issues/16051/labels{/name}","comments_url":"https://api.github.com/repos/golang/go/issues/16051/comments","events_url":"https://api.github.com/repos/golang/go/issues/16051/events","html_url":"https://github.com/golang/go/issues/16051","id":159996100,"number":16051,"title":"proposal: a faster C-call mechanism for non-blocking C functions","user":{"login":"eloff","id":27574,"avatar_url":"https://avatars.githubusercontent.com/u/27574?v=3","gravatar_id":"","url":"https://api.github.com/users/eloff","html_url":"https://github.com/eloff","followers_url":"https://api.github.com/users/eloff/followers","following_url":"https://api.github.com/users/eloff/following{/other_user}","gists_url":"https://api.github.com/users/eloff/gists{/gist_id}","starred_url":"https://api.github.com/users/eloff/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/eloff/subscriptions","organizations_url":"https://api.github.com/users/eloff/orgs","repos_url":"https://api.github.com/users/eloff/repos","events_url":"https://api.github.com/users/eloff/events{/privacy}","received_events_url":"https://api.github.com/users/eloff/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/golang/go/labels/Proposal","name":"Proposal","color":"ededed"}],"state":"open","locked":false,"assignee":null,"milestone":{"url":"https://api.github.com/repos/golang/go/milestones/30","html_url":"https://github.com/golang/go/milestones/Proposal","labels_url":"https://api.github.com/repos/golang/go/milestones/30/labels","id":1373555,"number":30,"title":"Proposal","description":"Proposals that are pending (not yet accepted/rejected).\r\n","creator":{"login":"rsc","id":104030,"avatar_url":"https://avatars.githubusercontent.com/u/104030?v=3","gravatar_id":"","url":"https://api.github.com/users/rsc","html_url":"https://github.com/rsc","followers_url":"https://api.github.com/users/rsc/followers","following_url":"https://api.github.com/users/rsc/following{/other_user}","gists_url":"https://api.github.com/users/rsc/gists{/gist_id}","starred_url":"https://api.github.com/users/rsc/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rsc/subscriptions","organizations_url":"https://api.github.com/users/rsc/orgs","repos_url":"https://api.github.com/users/rsc/repos","events_url":"https://api.github.com/users/rsc/events{/privacy}","received_events_url":"https://api.github.com/users/rsc/received_events","type":"User","site_admin":false},"open_issues":48,"closed_issues":24,"state":"open","created_at":"2015-10-24T01:40:49Z","updated_at":"2016-06-14T19:53:23Z","due_on":null,"closed_at":null},"comments":6,"created_at":"2016-06-13T16:51:09Z","updated_at":"2016-06-15T19:56:48Z","closed_at":null,"body":"I'm not sure this is the place to submit this, the golang-dev list might also have been a good candidate.\r\n\r\nThe cgo FFI mechanism is very general, and makes some pessimistic assumptions about the function being called - specifically it has to handle the case where the called function may block, e.g. in a blocking syscall. Having to handle the worst case adds a lot of overhead to each call into C, including just calling a math function or other small function that does no syscalls or does not block.\r\n\r\nThe problem is for well-behaved calls that neither block nor call back into Go, if the called function does not do much work then the cgo overhead dominates the runtime and causes poor performance. This is a well known limitation (and often complained about on golang-nuts and elsewhere, google \"cgo overhead\" returns about 45,000 hits.) Some of this can be mitigated by making the C function do more work, e.g. exporting a chunkier vs chattier API - but that's not always feasible or desirable.\r\n\r\nFor some classes of problems this overhead is just not admissible, and people resort to doing bad things like calling C from assembly on a Go stack, which just isn't built for it. The Go runtime itself finds the C call overhead too much at times, and Dmitry Vyukov had to work around it when integrating the TSAN race checker with Go. A look at https://golang.org/src/runtime/race_amd64.s?m=text shows a custom C call mechanism that just puts the arguments into the right registers, switches stacks from the Go stack to the C stack, and calls the TSAN runtime function.\r\n\r\nSince this seems to be a requirement that cannot always be worked around, and it's a requirement of both Go users and the Go runtime itself, and the workarounds are difficult, fraught, and fragile - I would like to see a solution for this in Go itself. Such a solution should very closely mirror the functionality in race_amd64.s, ideally because that approach is about as low overhead as possible, and because it can be used to replace that code. It could take any form, but since it needs to be backwards compatible and ideally one does not introduce new syntax - something like an attribute on the function declaration, or a cgo directive would be nice. It is up to the programmer to ensure that C functions called in this way do not block, and don't call back into Go.\r\n```\r\n\r\ne.g. __attribute__ ((nonblocking)) in the declaration,\r\nor // #cgo nonblocking directly preceeding the callsite or the declaration\r\n```\r\n\r\nIs this a reasonable feature for a future version of Go?"},"comment":{"url":"https://api.github.com/repos/golang/go/issues/comments/226301643","html_url":"https://github.com/golang/go/issues/16051#issuecomment-226301643","issue_url":"https://api.github.com/repos/golang/go/issues/16051","id":226301643,"user":{"login":"eloff","id":27574,"avatar_url":"https://avatars.githubusercontent.com/u/27574?v=3","gravatar_id":"","url":"https://api.github.com/users/eloff","html_url":"https://github.com/eloff","followers_url":"https://api.github.com/users/eloff/followers","following_url":"https://api.github.com/users/eloff/following{/other_user}","gists_url":"https://api.github.com/users/eloff/gists{/gist_id}","starred_url":"https://api.github.com/users/eloff/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/eloff/subscriptions","organizations_url":"https://api.github.com/users/eloff/orgs","repos_url":"https://api.github.com/users/eloff/repos","events_url":"https://api.github.com/users/eloff/events{/privacy}","received_events_url":"https://api.github.com/users/eloff/received_events","type":"User","site_admin":false},"created_at":"2016-06-15T19:56:47Z","updated_at":"2016-06-15T19:56:47Z","body":"Can we clarify for ignorant like myself what the risks are here if on an unusual code path a call blocks that is marked `#cgo nonblocking`? As far as I understand it it's non-optimal, but not going to cause deadlocks. What would cause deadlocks is if the code did something that blocks on another goroutine which won't run now that we're blocking the scheduler. That's still a problem but a lot more specific than just it needs to be a function that never blocks. It needs to be a function that usually doesn't block, and that if it does it never blocks on another thread in the process - e.g. with a mutex, or something. That's a lot easier to guarantee. I can call functions from the C standard library not sure that they would block or not, but I can be sure if they will or won't block on another goroutine in my program - especially since any goroutine blocked in the scheduler cannot be in the middle of another C non blocking call. Like the mutex(s) in malloc aren't a problem because a goroutine can't get unscheduled in the middle of a malloc. Any mutexes only used inside C also won't be a problem because it's not possible to put a goroutine to sleep while it's in the middle of a C call. Of course my understanding may well be flawed, but if it's not, then it seems like this is not an unreasonable burden to place on the programmer for use of an opt-in, advanced feature. \r\n\r\nRemember that the alternatives when cgo performance is not enough, are more dangerous. They carry these exact same risks, plus are fragile to the Go implementation changing, plus involve assembly and access to go runtime internals. Worse yet, if one avoids doing it right by digging into the go runtime to change stacks, and just do the call on the goroutine stack - now you're making assumptions about how much stack space your C program needs - and that's an extremely unsafe assumption that can't be easily made. Who knows which standard library functions allocate a big buffer on the stack or use recursion."}}
{"actor":{"display_login":"eloff"}}