{"action":"created","issue":{"url":"https://api.github.com/repos/niklasf/fishnet/issues/23","repository_url":"https://api.github.com/repos/niklasf/fishnet","labels_url":"https://api.github.com/repos/niklasf/fishnet/issues/23/labels{/name}","comments_url":"https://api.github.com/repos/niklasf/fishnet/issues/23/comments","events_url":"https://api.github.com/repos/niklasf/fishnet/issues/23/events","html_url":"https://github.com/niklasf/fishnet/issues/23","id":140866726,"number":23,"title":"missing process?","user":{"login":"ornicar","id":140370,"avatar_url":"https://avatars.githubusercontent.com/u/140370?v=3","gravatar_id":"","url":"https://api.github.com/users/ornicar","html_url":"https://github.com/ornicar","followers_url":"https://api.github.com/users/ornicar/followers","following_url":"https://api.github.com/users/ornicar/following{/other_user}","gists_url":"https://api.github.com/users/ornicar/gists{/gist_id}","starred_url":"https://api.github.com/users/ornicar/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ornicar/subscriptions","organizations_url":"https://api.github.com/users/ornicar/orgs","repos_url":"https://api.github.com/users/ornicar/repos","events_url":"https://api.github.com/users/ornicar/events{/privacy}","received_events_url":"https://api.github.com/users/ornicar/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":{"url":"https://api.github.com/repos/niklasf/fishnet/milestones/1","html_url":"https://github.com/niklasf/fishnet/milestones/v1%20-%20Lichess%20servers","labels_url":"https://api.github.com/repos/niklasf/fishnet/milestones/1/labels","id":1644715,"number":1,"title":"v1 - Lichess servers","description":"Handle all of lichess moves and analysis.\r\n\r\nRun on lichess servers only.\r\n\r\nAll clients are 100% trusted.","creator":{"login":"ornicar","id":140370,"avatar_url":"https://avatars.githubusercontent.com/u/140370?v=3","gravatar_id":"","url":"https://api.github.com/users/ornicar","html_url":"https://github.com/ornicar","followers_url":"https://api.github.com/users/ornicar/followers","following_url":"https://api.github.com/users/ornicar/following{/other_user}","gists_url":"https://api.github.com/users/ornicar/gists{/gist_id}","starred_url":"https://api.github.com/users/ornicar/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ornicar/subscriptions","organizations_url":"https://api.github.com/users/ornicar/orgs","repos_url":"https://api.github.com/users/ornicar/repos","events_url":"https://api.github.com/users/ornicar/events{/privacy}","received_events_url":"https://api.github.com/users/ornicar/received_events","type":"User","site_admin":false},"open_issues":1,"closed_issues":0,"state":"open","created_at":"2016-03-15T04:17:30Z","updated_at":"2016-03-15T04:23:32Z","due_on":null,"closed_at":null},"comments":2,"created_at":"2016-03-15T04:11:04Z","updated_at":"2016-03-15T11:52:05Z","closed_at":null,"body":"Happens sometimes (~20%) on a prod server. One of the 2 processes never logs a thing.\r\n\r\n```\r\nroot@homer:/home/fishnet/fishnet# python fishnet.py polyglot.ini --verbose                                                                                        [780/1976]\r\n  _____ _     _     _   _      _\r\n |  ___(_)___| |__ | \\ | | ___| |_\r\n | |_  | / __| '_ \\|  \\| |/ _ \\ __|\r\n |  _| | \\__ \\ | | | |\\  |  __/ |_\r\n |_|   |_|___/_| |_|_| \\_|\\___|\\__| 0.0.1\r\n Distributed Stockfish analysis for lichess.org\r\n\r\nINFO: MainThread: Hashtable size per process: 256 MB\r\nINFO: MainThread: Cores: 7 + 1\r\nINFO: MainThread: Available memmory: 38631 MB / 60%\r\nDEBUG: Process 2: 28075 << uci\r\nDEBUG: Process 2: 28075 >> Stockfish 140316 64 POPCNT by T. Romstad, M. Costalba, J. Kiiski, G. Linscott\r\nDEBUG: Process 2: 28075 >> id name Stockfish 140316 64 POPCNT\r\nDEBUG: Process 2: 28075 >> id author T. Romstad, M. Costalba, J. Kiiski, G. Linscott\r\nDEBUG: Process 2: 28075 >> \r\nDEBUG: Process 2: 28075 >> option name Write Debug Log type check default false\r\nDEBUG: Process 2: 28075 >> option name Contempt type spin default 0 min -100 max 100\r\nDEBUG: Process 2: 28075 >> option name Threads type spin default 1 min 1 max 128\r\nDEBUG: Process 2: 28075 >> option name Hash type spin default 16 min 1 max 1048576\r\nDEBUG: Process 2: 28075 >> option name Clear Hash type button\r\nDEBUG: Process 2: 28075 >> option name Ponder type check default false\r\nDEBUG: Process 2: 28075 >> option name MultiPV type spin default 1 min 1 max 500\r\nDEBUG: Process 2: 28075 >> option name Skill Level type spin default 20 min 0 max 20\r\nDEBUG: Process 2: 28075 >> option name Move Overhead type spin default 30 min 0 max 5000\r\nDEBUG: Process 2: 28075 >> option name Minimum Thinking Time type spin default 20 min 0 max 5000\r\nDEBUG: Process 2: 28075 >> option name Slow Mover type spin default 89 min 10 max 1000\r\nDEBUG: Process 2: 28075 >> option name nodestime type spin default 0 min 0 max 10000\r\nDEBUG: Process 2: 28075 >> option name UCI_Chess960 type check default false\r\nDEBUG: Process 2: 28075 >> option name UCI_Atomic type check default false\r\nDEBUG: Process 2: 28075 >> option name UCI_Horde type check default false\r\nDEBUG: Process 2: 28075 >> option name UCI_House type check default false\r\nDEBUG: Process 2: 28075 >> option name UCI_KingOfTheHill type check default false\r\nDEBUG: Process 2: 28075 >> option name UCI_Race type check default false\r\nDEBUG: Process 2: 28075 >> option name UCI_3Check type check default false\r\nDEBUG: Process 2: 28075 >> option name SyzygyPath type string default <empty>\r\nDEBUG: Process 2: 28075 >> option name SyzygyProbeDepth type spin default 1 min 1 max 100\r\nDEBUG: Process 2: 28075 >> option name Syzygy50MoveRule type check default true\r\nDEBUG: Process 2: 28075 >> option name SyzygyProbeLimit type spin default 6 min 0 max 6\r\nDEBUG: Process 2: 28075 >> uciok\r\nINFO: Process 2: Started engine process, pid: 28075, threads: 3, identification: Stockfish 140316 64 POPCNT\r\nDEBUG: Process 2: 28075 << setoption name Threads value 3\r\nDEBUG: Process 2: 28075 << setoption name hash value 256\r\nDEBUG: Process 2: 28075 << isready\r\nDEBUG: Process 2: 28075 >> readyok\r\nDEBUG: Process 2: 28075 << bench\r\n```\r\nlog continues with Process 2 working normally. \r\n\r\nOnce, when I `^C` fishnet during this bug, the following message appeared:\r\n```\r\n^CException in thread Process 1 (most likely raised during interpreter shutdown):\r\n```\r\n\r\nSometimes it's Process 1 not logging, sometimes it's Process 2.\r\n\r\nIt can be reproduced on homer, which you have access to, in `/home/fishnet/fishnet`. Give it several tries to reproduce it. It's ok if fishnet is down on homer."},"comment":{"url":"https://api.github.com/repos/niklasf/fishnet/issues/comments/196785478","html_url":"https://github.com/niklasf/fishnet/issues/23#issuecomment-196785478","issue_url":"https://api.github.com/repos/niklasf/fishnet/issues/23","id":196785478,"user":{"login":"niklasf","id":402777,"avatar_url":"https://avatars.githubusercontent.com/u/402777?v=3","gravatar_id":"","url":"https://api.github.com/users/niklasf","html_url":"https://github.com/niklasf","followers_url":"https://api.github.com/users/niklasf/followers","following_url":"https://api.github.com/users/niklasf/following{/other_user}","gists_url":"https://api.github.com/users/niklasf/gists{/gist_id}","starred_url":"https://api.github.com/users/niklasf/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/niklasf/subscriptions","organizations_url":"https://api.github.com/users/niklasf/orgs","repos_url":"https://api.github.com/users/niklasf/repos","events_url":"https://api.github.com/users/niklasf/events{/privacy}","received_events_url":"https://api.github.com/users/niklasf/received_events","type":"User","site_admin":false},"created_at":"2016-03-15T11:52:05Z","updated_at":"2016-03-15T11:52:05Z","body":"this is as ugly as it gets. a thread-safety bug fixed in python 3. i'll see if i can find a workaround. alternatively someone backported the changes: https://github.com/google/python-subprocess32, but this would be our first external dependency"}}