{"action":"created","issue":{"url":"https://api.github.com/repos/natsys/tempesta/issues/469","repository_url":"https://api.github.com/repos/natsys/tempesta","labels_url":"https://api.github.com/repos/natsys/tempesta/issues/469/labels{/name}","comments_url":"https://api.github.com/repos/natsys/tempesta/issues/469/comments","events_url":"https://api.github.com/repos/natsys/tempesta/issues/469/events","html_url":"https://github.com/natsys/tempesta/issues/469","id":153104459,"number":469,"title":"skb_push causes panic on skb without linear part","user":{"login":"milabs","id":624823,"avatar_url":"https://avatars.githubusercontent.com/u/624823?v=3","gravatar_id":"","url":"https://api.github.com/users/milabs","html_url":"https://github.com/milabs","followers_url":"https://api.github.com/users/milabs/followers","following_url":"https://api.github.com/users/milabs/following{/other_user}","gists_url":"https://api.github.com/users/milabs/gists{/gist_id}","starred_url":"https://api.github.com/users/milabs/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/milabs/subscriptions","organizations_url":"https://api.github.com/users/milabs/orgs","repos_url":"https://api.github.com/users/milabs/repos","events_url":"https://api.github.com/users/milabs/events{/privacy}","received_events_url":"https://api.github.com/users/milabs/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/natsys/tempesta/labels/bug","name":"bug","color":"eb6420"},{"url":"https://api.github.com/repos/natsys/tempesta/labels/crucial","name":"crucial","color":"ff1010"}],"state":"open","locked":false,"assignee":null,"milestone":{"url":"https://api.github.com/repos/natsys/tempesta/milestones/3","html_url":"https://github.com/natsys/tempesta/milestones/0.5.0%20Web%20Server","labels_url":"https://api.github.com/repos/natsys/tempesta/milestones/3/labels","id":1015684,"number":3,"title":"0.5.0 Web Server","description":"","creator":{"login":"krizhanovsky","id":884694,"avatar_url":"https://avatars.githubusercontent.com/u/884694?v=3","gravatar_id":"","url":"https://api.github.com/users/krizhanovsky","html_url":"https://github.com/krizhanovsky","followers_url":"https://api.github.com/users/krizhanovsky/followers","following_url":"https://api.github.com/users/krizhanovsky/following{/other_user}","gists_url":"https://api.github.com/users/krizhanovsky/gists{/gist_id}","starred_url":"https://api.github.com/users/krizhanovsky/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/krizhanovsky/subscriptions","organizations_url":"https://api.github.com/users/krizhanovsky/orgs","repos_url":"https://api.github.com/users/krizhanovsky/repos","events_url":"https://api.github.com/users/krizhanovsky/events{/privacy}","received_events_url":"https://api.github.com/users/krizhanovsky/received_events","type":"User","site_admin":false},"open_issues":41,"closed_issues":95,"state":"open","created_at":"2015-03-10T23:31:23Z","updated_at":"2016-05-04T20:21:01Z","due_on":"2016-06-30T21:00:00Z","closed_at":null},"comments":1,"created_at":"2016-05-04T20:15:44Z","updated_at":"2016-05-04T20:21:44Z","closed_at":null,"body":"When running Tempesta in QEMU with CPUx8/4G/virtio_net the BUG happens:\r\n\r\n```\r\n      KERNEL: vmlinux                           \r\n    DUMPFILE: /home/ilya/vms/shared/dump2\r\n        CPUS: 8\r\n        DATE: Wed May  4 22:48:20 2016\r\n      UPTIME: 00:11:34\r\nLOAD AVERAGE: 0.25, 0.08, 0.03\r\n       TASKS: 110\r\n    NODENAME: tempesta\r\n     RELEASE: 4.1.23-tfw\r\n     VERSION: #68 SMP PREEMPT Wed May 4 22:16:44 MSK 2016\r\n     MACHINE: x86_64  (4007 Mhz)\r\n      MEMORY: 4 GB\r\n       PANIC: \"kernel BUG at net/core/skbuff.c:107!\"\r\n         PID: 0\r\n     COMMAND: \"swapper/1\"\r\n        TASK: ffff88015b084010  (1 of 8)  [THREAD_INFO: ffff88015b0d4000]\r\n         CPU: 1\r\n       STATE: TASK_RUNNING (PANIC)\r\n\r\ncrash> bt\r\nPID: 0      TASK: ffff88015b084010  CPU: 1   COMMAND: \"swapper/1\"\r\n #0 [ffff88017fc439a0] machine_kexec at ffffffff8102ffad\r\n #1 [ffff88017fc439f0] crash_kexec at ffffffff810a25be\r\n #2 [ffff88017fc43ac0] oops_end at ffffffff810069f0\r\n #3 [ffff88017fc43af0] die at ffffffff81006b86\r\n #4 [ffff88017fc43b20] do_trap at ffffffff81003d6b\r\n #5 [ffff88017fc43b80] do_error_trap at ffffffff81003ed5\r\n #6 [ffff88017fc43c40] do_invalid_op at ffffffff8100442b\r\n #7 [ffff88017fc43c50] invalid_op at ffffffff81422b1b\r\n    [exception RIP: skb_panic+0x5e]\r\n    RIP: ffffffff813450ae  RSP: ffff88017fc43d08  RFLAGS: 00010296\r\n    RAX: 0000000000000088  RBX: ffff880037bd2040  RCX: 0000000000000088\r\n    RDX: 0000000080000103  RSI: ffffffff817552fd  RDI: 00000000ffffffff\r\n    RBP: ffff88017fc43d28   R8: 0000000000000000   R9: 000000000000163c\r\n    R10: 0000000000000001  R11: 000000000000163c  R12: ffff88014dde5400\r\n    R13: 0000000000000020  R14: 0000000000000020  R15: 0000000000000000\r\n    ORIG_RAX: ffffffffffffffff  CS: 0010  SS: 0018\r\n #8 [ffff88017fc43d30] skb_push at ffffffff8134590c\r\n #9 [ffff88017fc43d40] tcp_transmit_skb at ffffffff8139f889\r\n#10 [ffff88017fc43dc0] tcp_write_xmit at ffffffff813a0373\r\n#11 [ffff88017fc43e40] __tcp_push_pending_frames at ffffffff813a126d\r\n#12 [ffff88017fc43e60] tcp_push at ffffffff8138f720\r\n#13 [ffff88017fc43e70] ss_tx_action at ffffffffa004a111 [tempesta_fw]\r\n#14 [ffff88017fc43ef0] net_tx_action at ffffffff81354993\r\n#15 [ffff88017fc43f20] __do_softirq at ffffffff81048115\r\n#16 [ffff88017fc43f80] irq_exit at ffffffff8104846d\r\n#17 [ffff88017fc43f90] smp_trace_call_function_single_interrupt at ffffffff8102832e\r\n#18 [ffff88017fc43fa0] smp_call_function_single_interrupt at ffffffff81028339\r\n#19 [ffff88017fc43fb0] call_function_single_interrupt at ffffffff8142286b\r\n--- <IRQ stack> ---\r\n#20 [ffff88015b0d7e08] call_function_single_interrupt at ffffffff8142286b\r\n    [exception RIP: native_safe_halt+0x6]\r\n    RIP: ffffffff810340b6  RSP: ffff88015b0d7eb8  RFLAGS: 00000202\r\n    RAX: 0000000000000001  RBX: ffffffff81068fe0  RCX: 0140000000000000\r\n    RDX: 0100000000000000  RSI: ffffffff817368f5  RDI: ffffffff81740807\r\n    RBP: ffff88015b0d7eb8   R8: 0000000000000000   R9: 0000000000000000\r\n    R10: ffff88017fc541e8  R11: 0000000000000000  R12: ffff88015b0d4000\r\n    R13: 000000a1a3bb19c6  R14: 0000000000000001  R15: ffff88015b0d4000\r\n    ORIG_RAX: ffffffffffffff04  CS: 0010  SS: 0018\r\n#21 [ffff88015b0d7ec0] default_idle at ffffffff8100c43e\r\n#22 [ffff88015b0d7ed0] arch_cpu_idle at ffffffff8100cf3a\r\n#23 [ffff88015b0d7ee0] cpu_startup_entry at ffffffff810747ff\r\n#24 [ffff88015b0d7f30] start_secondary at ffffffff8102892b\r\n```\r\n\r\nThe BUG and is reproducible and the reason is show below:\r\n```\r\n1421 unsigned char *skb_push(struct sk_buff *skb, unsigned int len)\r\n1422 {\r\n1423         skb->data -= len;\r\n1424         skb->len  += len;\r\n1425         if (unlikely(skb->data<skb->head))\r\n1426                 skb_under_panic(skb, len, __builtin_return_address(0));\r\n1427         return skb->data;\r\n1428 }\r\n1429 EXPORT_SYMBOL(skb_push);\r\n```\r\n\r\nThe condition `if (unlikely(skb->data<skb->head))` is met and `skb_under_panic` triggers. Looking through the Tempesta's code leads me to strange [allocation](https://github.com/natsys/tempesta/blob/52faa3d1280dbae18cd52601765c3efc5abe915a/tempesta_fw/ss_skb.c#L187) of skb with 0 size:\r\n\r\n```\r\n\t\t/* Allocate a new SKB to hold @n_excess page fragments. */\r\n\t\tnskb = alloc_skb(0, GFP_ATOMIC);\r\n\t\tif (nskb == NULL)\r\n\t\t\treturn -ENOMEM;\r\n```\r\n\r\nChanging 0 to 256 helps and the BUG doesn't appear any more.\r\n\r\nHere is the test I've run to catch the BUG and ensure that the BUG is gone:\r\nab -k -r -n 10000000 -c 1000 http://192.168.88.220:3128/index.html\r\n"},"comment":{"url":"https://api.github.com/repos/natsys/tempesta/issues/comments/216990988","html_url":"https://github.com/natsys/tempesta/issues/469#issuecomment-216990988","issue_url":"https://api.github.com/repos/natsys/tempesta/issues/469","id":216990988,"user":{"login":"keshonok","id":6352580,"avatar_url":"https://avatars.githubusercontent.com/u/6352580?v=3","gravatar_id":"","url":"https://api.github.com/users/keshonok","html_url":"https://github.com/keshonok","followers_url":"https://api.github.com/users/keshonok/followers","following_url":"https://api.github.com/users/keshonok/following{/other_user}","gists_url":"https://api.github.com/users/keshonok/gists{/gist_id}","starred_url":"https://api.github.com/users/keshonok/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/keshonok/subscriptions","organizations_url":"https://api.github.com/users/keshonok/orgs","repos_url":"https://api.github.com/users/keshonok/repos","events_url":"https://api.github.com/users/keshonok/events{/privacy}","received_events_url":"https://api.github.com/users/keshonok/received_events","type":"User","site_admin":false},"created_at":"2016-05-04T20:21:44Z","updated_at":"2016-05-04T20:21:44Z","body":"The SKBs allocated in `__extend_pgfrags()` are attached to the parent skb as skb fragments through `frag_list`. SKB fragments don't need the linear part. For this bug to happen, one of these SKBs must be turned into a standalone SKB in the egress path in the kernel."}}