{"action":"created","issue":{"url":"https://api.github.com/repos/raspberrypi/firmware/issues/764","repository_url":"https://api.github.com/repos/raspberrypi/firmware","labels_url":"https://api.github.com/repos/raspberrypi/firmware/issues/764/labels{/name}","comments_url":"https://api.github.com/repos/raspberrypi/firmware/issues/764/comments","events_url":"https://api.github.com/repos/raspberrypi/firmware/issues/764/events","html_url":"https://github.com/raspberrypi/firmware/issues/764","id":214311921,"node_id":"MDU6SXNzdWUyMTQzMTE5MjE=","number":764,"title":"bootcode.bin randomly doesn't PXE boot correctly.","user":{"login":"ali1234","id":651970,"node_id":"MDQ6VXNlcjY1MTk3MA==","avatar_url":"https://avatars2.githubusercontent.com/u/651970?v=4","gravatar_id":"","url":"https://api.github.com/users/ali1234","html_url":"https://github.com/ali1234","followers_url":"https://api.github.com/users/ali1234/followers","following_url":"https://api.github.com/users/ali1234/following{/other_user}","gists_url":"https://api.github.com/users/ali1234/gists{/gist_id}","starred_url":"https://api.github.com/users/ali1234/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ali1234/subscriptions","organizations_url":"https://api.github.com/users/ali1234/orgs","repos_url":"https://api.github.com/users/ali1234/repos","events_url":"https://api.github.com/users/ali1234/events{/privacy}","received_events_url":"https://api.github.com/users/ali1234/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":30,"created_at":"2017-03-15T08:04:30Z","updated_at":"2018-07-05T11:38:53Z","closed_at":"2017-05-24T11:34:45Z","author_association":"NONE","body":"I am using the latest version of bootcode.bin: https://github.com/raspberrypi/firmware/blob/f85646a8831d9579c2a745478149598da1ecfde5/boot/bootcode.bin\r\n\r\nIt is the only file on my SD card. I am using a Raspberry Pi 3.\r\n\r\nSometimes (but rarely) PXE boot works and sometimes it does not. I have to power cycle the Pi several times to make it boot.\r\n\r\nLooking at the failed tcpdump log you can see that dnsmasq is replying to the boot request, but the Pi ignores it and sends another, for a total of 5 requests. Then it tries to request tftp files from 0.0.0.0.\r\n\r\n\r\ndnsmasq log of failed session:\r\n```\r\ndnsmasq-dhcp: 653460281 available DHCP subnet: xxx.xxx.xxx.255/255.255.255.0\r\ndnsmasq-dhcp: 653460281 vendor class: PXEClient:Arch:00000:UNDI:002001\r\ndnsmasq-dhcp: 653460281 PXE(enp0s31f6) b8:27:eb:xx:xx:xx proxy\r\ndnsmasq-dhcp: 653460281 tags: enp0s31f6\r\ndnsmasq-dhcp: 653460281 broadcast response\r\ndnsmasq-dhcp: 653460281 sent size:  1 option: 53 message-type  2\r\ndnsmasq-dhcp: 653460281 sent size:  4 option: 54 server-identifier  xxx.xxx.xxx.5\r\ndnsmasq-dhcp: 653460281 sent size:  9 option: 60 vendor-class  50:58:xx:xx:xx:xx:xx:xx:xx\r\ndnsmasq-dhcp: 653460281 sent size: 17 option: 97 client-machine-id  00:f5:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx...\r\ndnsmasq-dhcp: 653460281 sent size: 32 option: 43 vendor-encap  06:01:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx...\r\ndnsmasq-dhcp: 653460281 available DHCP subnet: xxx.xxx.xxx.255/255.255.255.0\r\ndnsmasq-dhcp: 653460281 vendor class: PXEClient:Arch:00000:UNDI:002001\r\ndnsmasq-dhcp: 653460281 PXE(enp0s31f6) b8:27:eb:xx:xx:xx proxy\r\ndnsmasq-dhcp: 653460281 tags: enp0s31f6\r\ndnsmasq-dhcp: 653460281 broadcast response\r\ndnsmasq-dhcp: 653460281 sent size:  1 option: 53 message-type  2\r\ndnsmasq-dhcp: 653460281 sent size:  4 option: 54 server-identifier  xxx.xxx.xxx.5\r\ndnsmasq-dhcp: 653460281 sent size:  9 option: 60 vendor-class  50:58:xx:xx:xx:xx:xx:xx:xx\r\ndnsmasq-dhcp: 653460281 sent size: 17 option: 97 client-machine-id  00:f5:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx...\r\ndnsmasq-dhcp: 653460281 sent size: 32 option: 43 vendor-encap  06:01:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx...\r\ndnsmasq-dhcp: 653460281 available DHCP subnet: xxx.xxx.xxx.255/255.255.255.0\r\ndnsmasq-dhcp: 653460281 vendor class: PXEClient:Arch:00000:UNDI:002001\r\ndnsmasq-dhcp: 653460281 PXE(enp0s31f6) b8:27:eb:xx:xx:xx proxy\r\ndnsmasq-dhcp: 653460281 tags: enp0s31f6\r\ndnsmasq-dhcp: 653460281 broadcast response\r\ndnsmasq-dhcp: 653460281 sent size:  1 option: 53 message-type  2\r\ndnsmasq-dhcp: 653460281 sent size:  4 option: 54 server-identifier  xxx.xxx.xxx.5\r\ndnsmasq-dhcp: 653460281 sent size:  9 option: 60 vendor-class  50:58:xx:xx:xx:xx:xx:xx:xx\r\ndnsmasq-dhcp: 653460281 sent size: 17 option: 97 client-machine-id  00:f5:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx...\r\ndnsmasq-dhcp: 653460281 sent size: 32 option: 43 vendor-encap  06:01:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx...\r\ndnsmasq-dhcp: 653460281 available DHCP subnet: xxx.xxx.xxx.255/255.255.255.0\r\ndnsmasq-dhcp: 653460281 vendor class: PXEClient:Arch:00000:UNDI:002001\r\ndnsmasq-dhcp: 653460281 PXE(enp0s31f6) b8:27:eb:xx:xx:xx proxy\r\ndnsmasq-dhcp: 653460281 tags: enp0s31f6\r\ndnsmasq-dhcp: 653460281 broadcast response\r\ndnsmasq-dhcp: 653460281 sent size:  1 option: 53 message-type  2\r\ndnsmasq-dhcp: 653460281 sent size:  4 option: 54 server-identifier  xxx.xxx.xxx.5\r\ndnsmasq-dhcp: 653460281 sent size:  9 option: 60 vendor-class  50:58:xx:xx:xx:xx:xx:xx:xx\r\ndnsmasq-dhcp: 653460281 sent size: 17 option: 97 client-machine-id  00:f5:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx...\r\ndnsmasq-dhcp: 653460281 sent size: 32 option: 43 vendor-encap  06:01:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx...\r\ndnsmasq-dhcp: 653460281 available DHCP subnet: xxx.xxx.xxx.255/255.255.255.0\r\ndnsmasq-dhcp: 653460281 vendor class: PXEClient:Arch:00000:UNDI:002001\r\ndnsmasq-dhcp: 653460281 PXE(enp0s31f6) b8:27:eb:xx:xx:xx proxy\r\ndnsmasq-dhcp: 653460281 tags: enp0s31f6\r\ndnsmasq-dhcp: 653460281 broadcast response\r\ndnsmasq-dhcp: 653460281 sent size:  1 option: 53 message-type  2\r\ndnsmasq-dhcp: 653460281 sent size:  4 option: 54 server-identifier  xxx.xxx.xxx.5\r\ndnsmasq-dhcp: 653460281 sent size:  9 option: 60 vendor-class  50:58:xx:xx:xx:xx:xx:xx:xx\r\ndnsmasq-dhcp: 653460281 sent size: 17 option: 97 client-machine-id  00:f5:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx...\r\ndnsmasq-dhcp: 653460281 sent size: 32 option: 43 vendor-encap  06:01:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx...\r\n```\r\n\r\ntcpdump port tftp or port bootpc from failed session:\r\n```\r\n07:46:27.892426 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from b8:27:eb:xx:xx:xx (oui Unknown), length 322\r\nE..^......9..........D.C.J......&..9.....................'..............................................................................................................................................................................................................c.Sc5..7.+<C........B..]...^....a..................< PXEClient:Arch:00000:UNDI:002001.\r\n07:46:27.892915 IP pxeserver.lan.bootps > 255.255.255.255.bootpc: BOOTP/DHCP, Reply, length 314\r\nE..V....@............C.D.B......&..9.....................'..............................................................................................................................................................................................................c.Sc5..6.....<\tPXEClienta..................+ ...\r\n..PXE\t....Raspberry Pi Boot..\r\n07:46:32.893294 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from b8:27:eb:xx:xx:xx (oui Unknown), length 322\r\nE..^......9..........D.C.J......&..9.....................'..............................................................................................................................................................................................................c.Sc5..7.+<C........B..]...^....a..................< PXEClient:Arch:00000:UNDI:002001.\r\n07:46:32.893647 IP pxeserver.lan.bootps > 255.255.255.255.bootpc: BOOTP/DHCP, Reply, length 314\r\nE..V....@..v.........C.D.B......&..9.....................'..............................................................................................................................................................................................................c.Sc5..6.....<\tPXEClienta..................+ ...\r\n..PXE\t....Raspberry Pi Boot..\r\n07:46:38.640376 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from b8:27:eb:xx:xx:xx (oui Unknown), length 322\r\nE..^......9..........D.C.J......&..9.....................'..............................................................................................................................................................................................................c.Sc5..7.+<C........B..]...^....a..................< PXEClient:Arch:00000:UNDI:002001.\r\n07:46:38.640795 IP pxeserver.lan.bootps > 255.255.255.255.bootpc: BOOTP/DHCP, Reply, length 314\r\nE..V.\r\n..@.. .........C.D.B......&..9.....................'..............................................................................................................................................................................................................c.Sc5..6.....<\tPXEClienta..................+ ...\r\n..PXE\t....Raspberry Pi Boot..\r\n07:46:44.639980 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from b8:27:eb:xx:xx:xx (oui Unknown), length 322\r\nE..^......9..........D.C.J......&..9.....................'..............................................................................................................................................................................................................c.Sc5..7.+<C........B..]...^....a..................< PXEClient:Arch:00000:UNDI:002001.\r\n07:46:44.640339 IP pxeserver.lan.bootps > 255.255.255.255.bootpc: BOOTP/DHCP, Reply, length 314\r\nE..V.a..@............C.D.B......&..9.....................'..............................................................................................................................................................................................................c.Sc5..6.....<\tPXEClienta..................+ ...\r\n..PXE\t....Raspberry Pi Boot..\r\n07:46:50.639971 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from b8:27:eb:xx:xx:xx (oui Unknown), length 322\r\nE..^......9..........D.C.J......&..9.....................'..............................................................................................................................................................................................................c.Sc5..7.+<C........B..]...^....a..................< PXEClient:Arch:00000:UNDI:002001.\r\n07:46:50.640411 IP pxeserver.lan.bootps > 255.255.255.255.bootpc: BOOTP/DHCP, Reply, length 314\r\nE..V.b..@............C.D.B......&..9.....................'..............................................................................................................................................................................................................c.Sc5..6.....<\tPXEClienta..................+ ...\r\n..PXE\t....Raspberry Pi Boot..\r\n07:46:56.657068 IP 0.0.0.0.49153 > 0.0.0.0.tftp:  29 RRQ \"autoboot.txt\" octet tsize 0\r\nE..9......:............E.%....autoboot.txt.octet.tsize.0.\r\n07:46:57.657080 IP 0.0.0.0.49154 > 0.0.0.0.tftp:  27 RRQ \"config.txt\" octet tsize 0\r\nE..7......:............E.#....config.txt.octet.tsize.0.\r\n07:46:58.657351 IP 0.0.0.0.49155 > 0.0.0.0.tftp:  29 RRQ \"recovery.elf\" octet tsize 0\r\nE..9......:............E.%....recovery.elf.octet.tsize.0.\r\n07:46:59.657412 IP 0.0.0.0.49156 > 0.0.0.0.tftp:  26 RRQ \"start.elf\" octet tsize 0\r\nE..6......:............E.\"....start.elf.octet.tsize.0.\r\n07:47:00.773516 IP 0.0.0.0.49157 > 0.0.0.0.tftp:  26 RRQ \"fixup.dat\" octet tsize 0\r\nE..6......:............E.\"....fixup.dat.octet.tsize.0.\r\n```"},"comment":{"url":"https://api.github.com/repos/raspberrypi/firmware/issues/comments/402694617","html_url":"https://github.com/raspberrypi/firmware/issues/764#issuecomment-402694617","issue_url":"https://api.github.com/repos/raspberrypi/firmware/issues/764","id":402694617,"node_id":"MDEyOklzc3VlQ29tbWVudDQwMjY5NDYxNw==","user":{"login":"bunyevacz","id":40023510,"node_id":"MDQ6VXNlcjQwMDIzNTEw","avatar_url":"https://avatars0.githubusercontent.com/u/40023510?v=4","gravatar_id":"","url":"https://api.github.com/users/bunyevacz","html_url":"https://github.com/bunyevacz","followers_url":"https://api.github.com/users/bunyevacz/followers","following_url":"https://api.github.com/users/bunyevacz/following{/other_user}","gists_url":"https://api.github.com/users/bunyevacz/gists{/gist_id}","starred_url":"https://api.github.com/users/bunyevacz/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bunyevacz/subscriptions","organizations_url":"https://api.github.com/users/bunyevacz/orgs","repos_url":"https://api.github.com/users/bunyevacz/repos","events_url":"https://api.github.com/users/bunyevacz/events{/privacy}","received_events_url":"https://api.github.com/users/bunyevacz/received_events","type":"User","site_admin":false},"created_at":"2018-07-05T11:38:53Z","updated_at":"2018-07-05T11:38:53Z","author_association":"NONE","body":"I have made a 30 raspberry pi setup. One 24 port PoE enabled managed switch (ES-24-250W), and two unmanaged PoE switch (TP-Link TL-SG1008P, TP-Link TL-SF1008P).\r\n\r\nI have a raspberry pi acting as master: \r\n- wifi -> ethernet bridge for internet connectivity\r\n- DHCP server\r\n- TFTP server\r\n- NFS server\r\n\r\nAnd I have 29 raspberry pi 3, without any SD cards. So they boot from network entirely, and gets its power from PoE. All the raspberry pi have the official touchscreen. (even the master)\r\n\r\nI can switch on and off each port of the managed PoE switch. Essentially the exact same as pluggin in and unplugging the cable. \r\n\r\nHere are my experiences (22 raspberry pi clients, 1 raspberry pi server, 1 laptop):\r\n1) Switching on each raspberry pi at the same time from the PoE switch. Ie. within half a second or less.\r\n3 raspberry pis booting up fine (usually between 1-5)\r\n14 raspberry pis stuck at the rainbow screen (usually about half the pis)\r\n5 raspberry pis stays black (and consuming 1.5W or less)\r\n\r\nIf the screen is on, the raspberry pi consumes between 5.5-7W.\r\n\r\n2) Apply the dhcp-replay-delay=1 to dnsmasq.conf \r\nAlmost all the raspberry pi gets to the rainbow at least. Usually:\r\n10 raspberry pis finishes booting\r\n10 raspberry pis stuck at rainbow\r\n2 raspberry pis stays black screen.\r\n\r\n3) Apply the dhcp-replay-delay=1, and power each raspberry pi with 10sec delay \r\nSo the managed switch apply (via a bash script from my laptop) power to each PoE interface 10 sec apart.\r\n\r\nAlmost all raspberry pi boots up fine. Worst case was 20/22.\r\nOnce the raspberry pi boots up, it send heartbeat message to the master. The master figures out, which raspberry pi failed to boot, and unplug-replug power via the managed switch (the switch can provide which mac address belongs to which physical interface).\r\n\r\nThe timing is as follow:\r\n0-16 black screen\r\n7s: ethernet port starts to blink\r\n16-18sec rainbow\r\n50sec: finished booting into text autologin, and starts the X server\r\n1m3s: X server started , it is now completely grey\r\n1m15: chromium started in kiosk mode. Boot finished.\r\n\r\nI'm on this problem (unreliable starting) on a week now. I'm generating ICMPv6 packets, because I believed this helps. Because if I start a computer or a raspberry pi with sd card, then all the others raspberry pis starting up more likely. \r\nAnd judging from tcpdump, I believed ICMPv6 packets made a difference.\r\nIn reality it turns out, all it does is just makes dnsmasq occupied, which results response time going up, which results more likely raspberry pi starting. \r\n\r\nI'm now optimizing boot time from nfs. Disabling all services which is not needed. But the hard part was figuring out the unreliability. I think a pointer on the official site, or the netboot tutorial would be more appropriate, rather then the rather vague \"packets on the network helps booting up\". \r\nAlso this\r\n\"\"\"\r\nIf it doesn't boot on the first attempt, keep trying. It can take a minute or so for the Raspberry Pi to boot, so be patient.\r\n\"\"\"\r\nphrase is totally misleading and wrong. If a raspberry pi does not start (rainbow picture) after 20 sec, 99% sure it will never start at all.\r\nI booted up the raspberry pis like 400 times or more, and only one raspberry pi managed to start after 1 minute mark, once. \r\n\r\nWhen I started this adventure, I was totally unaware how untested this codepath is.:("}}
{"actor":{"display_login":"bunyevacz"}}