{"action":"opened","issue":{"url":"https://api.github.com/repos/vstakhov/rspamd/issues/2345","repository_url":"https://api.github.com/repos/vstakhov/rspamd","labels_url":"https://api.github.com/repos/vstakhov/rspamd/issues/2345/labels{/name}","comments_url":"https://api.github.com/repos/vstakhov/rspamd/issues/2345/comments","events_url":"https://api.github.com/repos/vstakhov/rspamd/issues/2345/events","html_url":"https://github.com/vstakhov/rspamd/issues/2345","id":341496607,"node_id":"MDU6SXNzdWUzNDE0OTY2MDc=","number":2345,"title":"per_user function doesn't work as excepted with redis backend in statistics module","user":{"login":"georgeandreev","id":26550694,"node_id":"MDQ6VXNlcjI2NTUwNjk0","avatar_url":"https://avatars1.githubusercontent.com/u/26550694?v=4","gravatar_id":"","url":"https://api.github.com/users/georgeandreev","html_url":"https://github.com/georgeandreev","followers_url":"https://api.github.com/users/georgeandreev/followers","following_url":"https://api.github.com/users/georgeandreev/following{/other_user}","gists_url":"https://api.github.com/users/georgeandreev/gists{/gist_id}","starred_url":"https://api.github.com/users/georgeandreev/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/georgeandreev/subscriptions","organizations_url":"https://api.github.com/users/georgeandreev/orgs","repos_url":"https://api.github.com/users/georgeandreev/repos","events_url":"https://api.github.com/users/georgeandreev/events{/privacy}","received_events_url":"https://api.github.com/users/georgeandreev/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2018-07-16T12:21:43Z","updated_at":"2018-07-16T12:21:43Z","closed_at":null,"author_association":"NONE","body":"### Classification (Please choose *one* option):\r\n\r\n* [ ] Crash/Hang/Data loss\r\n* [ ] WebUI/Usability\r\n* [ ] Serious bug\r\n* [x] Other bug\r\n* [ ] Feature\r\n* [ ] Enhancement\r\n\r\n### Reproducibility (Please choose *one* option):\r\n\r\n* [X] Always\r\n* [ ] Sometimes\r\n* [ ] Rarely\r\n* [ ] Unable\r\n* [ ] I didn’t try\r\n* [ ] Not applicable\r\n\r\n### Rspamd version:\r\nRspamd daemon version 1.7.8\r\n### Operation system, CPU, memory and environment:\r\nCentOS 7\r\nMemTotal:        3145728 kB\r\nMemFree:          806704 kB\r\nIntel(R) Core(TM) i5-2400 CPU @ 3.10GHz\r\n\r\n\r\n### Configuration:\r\n\r\n/etc/rspamd/override.d/statistic.conf:\r\n\r\nclassifier \"bayes\" {\r\n  name = \"bayes_user\";\r\n\r\n  tokenizer {\r\n    name = \"osb\";\r\n  }\r\n\r\n  min_tokens = 8;\r\n  backend = \"redis\";\r\n  servers = \"127.0.0.1:6379\";\r\n\r\n  per_user = <<EOD\r\nreturn function(task)\r\n    local logger=require \"rspamd_logger\" \r\n    local rcpt = task:get_recipients(1)\r\n\r\nif rcpt then\r\n    one_rcpt = rcpt[1]\r\n    if one_rcpt['domain'] then\r\n        logger.infox(task, 'Returning domain \"%1\" as user for \"%2\"', one_rcpt[\"domain\"], one_rcpt[\"addr\"])\r\n        return one_rcpt['domain']\r\n    end\r\nend\r\nlogger.infox(task, 'No user found')\r\n\r\nreturn nil\r\nend\r\nEOD\r\n\r\n  statfile {\r\n    symbol = \"BAYES_HAM_USER\";\r\n    spam = false;\r\n  }\r\n  statfile {\r\n    symbol = \"BAYES_SPAM_USER\";\r\n    spam = true;\r\n  }\r\n\r\n### Description (Please provide a descriptive summary of the issue):\r\n\r\n`per_user` LUA-function in the `bayes_user` classificator doesn't work as expected with redis backend. The code is executed, but metric is applied for all users (BAYES_USER_SPAM)\r\n\r\nTest email:\r\n\r\n`cat test1.eml`\r\n`To: info@example.com\r\nFrom: example@example.com\r\nSubject: Good morning\r\n\r\nHello, how are you? Today is a very good day. Let's go walk. Bye.`\r\n\r\n`rspamc learn_spam -c bayes_user -d test1@user1.com -r test1@user1.com test1.eml`\r\n`Results for file: test1.eml (0.003 seconds)\r\nsuccess = true;\r\nfilename = \"test1.eml\";\r\nscan_time = 0.002779;`\r\n\r\n`rspamc stat | fgrep BAYES`\r\n`Statfile: BAYES_SPAM_USER type: redis; length: 1.37k; free blocks: 0; total blocks: 36; free: 0.00%; learned: 1; users: 1; languages: 0\r\nStatfile: BAYES_HAM_USER type: redis; length: 0; free blocks: 0; total blocks: 0; free: 0.00%; learned: 0; users: 0; languages: 0`\r\n\r\n`rspamc symbols -d test1@user1.com -r test1@user1.com test1.eml | fgrep BAYES_`\r\n`Symbol: BAYES_SPAM_USER (0.00)[100.00%]`\r\n\r\n`tail /var/log/rspamd/rspamd.log | fgrep lua`\r\n`2018-07-16 14:49:05 #9903(controller) <6fcb76>; lua; [string \"return function(ta...:8: Returning domain \"user1.com\" as user for \"test1@user1.com\" \r\n2018-07-16 14:49:05 #9903(controller) <6fcb76>; lua; [string \"return function(ta...:8: Returning domain \"user1.com\" as user for \"test1@user1.com\"`\r\n\r\nNow metric is applied for all users:\r\n\r\n`rspamc symbols -d test2@example.com -r test2@example.com test1.eml | fgrep BAYES_`\r\n`Symbol: BAYES_SPAM_USER (0.00)[100.00%]`\r\n\r\n`tail /var/log/rspamd/rspamd.log | fgrep lua`\r\n`2018-07-16 14:50:51 #9903(controller) <62ce6c>; lua; [string \"return function(ta...:8: Returning domain \"example.com\" as user for \"test2@example.com\" \r\n2018-07-16 14:50:51 #9903(controller) <62ce6c>; lua; [string \"return function(ta...:8: Returning domain \"example.com\" as user for \"test2@example.com\"`\r\n\r\nNow if we feed the same message as ham for the second user:\r\n\r\n`rspamc learn_ham -c bayes_user -d user2@example.com -r user2@example.com test1.eml`\r\n`Results for file: test1.eml (0.002 seconds)\r\nsuccess = true;\r\nfilename = \"test1.eml\";\r\nscan_time = 0.001693;`\r\n\r\n`tail /var/log/rspamd/rspamd.log | fgrep lua`\r\n`2018-07-16 14:53:22 #9903(controller) <e40bef>; lua; [string \"return function(ta...:8: Returning domain \"example.com\" as user for \"user2@example.com\" \r\n2018-07-16 14:53:22 #9903(controller) <e40bef>; lua; [string \"return function(ta...:8: Returning domain \"example.com\" as user for \"user2@example.com\"`\r\n\r\nMessage is recognised as HAM for all users:\r\n\r\n`rspamc symbols -d test2@example.com -r test2@example.com test1.eml | fgrep BAYES_`\r\n`Symbol: BAYES_HAM_USER (0.00)[99.60%]`\r\n\r\n`rspamc symbols -d test1@user1.com -r test1@user1.com test1.eml | fgrep BAYES_`\r\n`Symbol: BAYES_HAM_USER (0.00)[98.10%]`\r\n\r\nBesides, rspamc stat always shows \"users: 1\" \r\n\r\n`for i in `seq 1 10`; do rspamc learn_spam -d test@user$i.com -r test@user$i.com test1.eml; done`\r\n\r\n`rspamc stat | fgrep BAYES`\r\n`Statfile: BAYES_SPAM_USER type: redis; length: 1.37k; free blocks: 0; total blocks: 36; free: 0.00%; learned: 10; users: 1; languages: 0\r\nStatfile: BAYES_HAM_USER type: redis; length: 1.37k; free blocks: 0; total blocks: 36; free: 0.00%; learned: 1; users: 1; languages: 0`\r\n\r\nBut if `per_user` is set to `true` in statistic.conf it works correctly. Metric is applied for single user.\r\n\r\nfor i in `seq 1 10`; do rspamc learn_spam -d test@user$i.com -r test@user$i.com test1.eml; done\r\n\r\n`rspamc stat | fgrep BAYES`\r\n`Statfile: BAYES_SPAM_USER type: redis; length: 13.68k; free blocks: 0; total blocks: 360; free: 0.00%; learned: 10; users: 10; languages: 0\r\nStatfile: BAYES_HAM_USER type: redis; length: 0; free blocks: 0; total blocks: 0; free: 0.00%; learned: 0; users: 0; languages: 0`\r\n\r\n`rspamc symbols -d test@user5.com -r test@user5.com test1.eml | fgrep BAYES`\r\n`Symbol: BAYES_SPAM_USER (0.00)[100.00%]`\r\n\r\n`rspamc symbols -d test@example.com -r test@example.com test1.eml | fgrep BAYES -c`\r\n`0`\r\n"}}
{"actor":{"display_login":"georgeandreev"}}