{"action":"closed","issue":{"url":"https://api.github.com/repos/kubernetes/kubernetes/issues/82088","repository_url":"https://api.github.com/repos/kubernetes/kubernetes","labels_url":"https://api.github.com/repos/kubernetes/kubernetes/issues/82088/labels{/name}","comments_url":"https://api.github.com/repos/kubernetes/kubernetes/issues/82088/comments","events_url":"https://api.github.com/repos/kubernetes/kubernetes/issues/82088/events","html_url":"https://github.com/kubernetes/kubernetes/issues/82088","id":486507426,"node_id":"MDU6SXNzdWU0ODY1MDc0MjY=","number":82088,"title":"Retry sending audit events on any webhook response error code","user":{"login":"shturec","id":4983982,"node_id":"MDQ6VXNlcjQ5ODM5ODI=","avatar_url":"https://avatars2.githubusercontent.com/u/4983982?v=4","gravatar_id":"","url":"https://api.github.com/users/shturec","html_url":"https://github.com/shturec","followers_url":"https://api.github.com/users/shturec/followers","following_url":"https://api.github.com/users/shturec/following{/other_user}","gists_url":"https://api.github.com/users/shturec/gists{/gist_id}","starred_url":"https://api.github.com/users/shturec/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/shturec/subscriptions","organizations_url":"https://api.github.com/users/shturec/orgs","repos_url":"https://api.github.com/users/shturec/repos","events_url":"https://api.github.com/users/shturec/events{/privacy}","received_events_url":"https://api.github.com/users/shturec/received_events","type":"User","site_admin":false},"labels":[{"id":680704582,"node_id":"MDU6TGFiZWw2ODA3MDQ1ODI=","url":"https://api.github.com/repos/kubernetes/kubernetes/labels/area/audit","name":"area/audit","color":"0052cc","default":false},{"id":267761362,"node_id":"MDU6TGFiZWwyNjc3NjEzNjI=","url":"https://api.github.com/repos/kubernetes/kubernetes/labels/kind/feature","name":"kind/feature","color":"c7def8","default":false},{"id":173493835,"node_id":"MDU6TGFiZWwxNzM0OTM4MzU=","url":"https://api.github.com/repos/kubernetes/kubernetes/labels/sig/api-machinery","name":"sig/api-machinery","color":"d2b48c","default":false},{"id":357119284,"node_id":"MDU6TGFiZWwzNTcxMTkyODQ=","url":"https://api.github.com/repos/kubernetes/kubernetes/labels/sig/auth","name":"sig/auth","color":"d2b48c","default":false}],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2019-08-28T17:17:39Z","updated_at":"2019-10-17T08:46:55Z","closed_at":"2019-10-17T08:46:55Z","author_association":"NONE","body":"**What would you like to be added**:\r\nThe api server should retry sending audit events to configured webhook sinks regardless of the type of HTTP error code it might receive from the webhooks.\r\n\r\n**Why is this needed**:\r\nWhile some components may live with failed data transmissions, this is a particularly sensitive topic for the audit area. The API server audit mechanisms must ensure that no other reasonable option exists before dropping an audit event that may be the only trace of malicious behavior in the cluster. It seems that the current implementation may be improved further in this direction.\r\n\r\nCurrently, when the API server is configured for non-blocking sending of audit events to a webhook audit sink backend, it will retry up to 5 times with exponential backoff, should there be an error in communicating with the webhook, as implemented [here]( https://github.com/kubernetes/kubernetes/blob/master/staging/src/k8s.io/apiserver/pkg/util/webhook/webhook.go#L95-L120).   \r\n\r\nHowever, not all errors will trigger a retry. Some are considered transient and yield retry. Others simply stop the retry cycle. For example, it retries on receiving HTTP code `500`, but not on receiving `502`. The list of error codes that will yield a retry is actually quite small - three HTTP errors and one network error. Any other error will not trigger a retry and the audit events will be lost. \r\n\r\nThe code comments explain that the retry will consider only recoverable failures, which may make sense in other cases, but as far as sending audit events to an audit backend webhook is concerned, it is not sufficient policy. A more robust approach would be to consider any error and give the backend service a chance to recover instead of relying on the error codes semantics and the webhook implementations interpretation of it. That is more inline with having the transmission of audit events as top priority.\r\n\r\n/sig auth\r\n/sig api-machinery"}}
{"actor":{"display_login":"k8s-ci-robot"}}