{"action":"created","issue":{"url":"https://api.github.com/repos/vermaseren/form/issues/215","repository_url":"https://api.github.com/repos/vermaseren/form","labels_url":"https://api.github.com/repos/vermaseren/form/issues/215/labels{/name}","comments_url":"https://api.github.com/repos/vermaseren/form/issues/215/comments","events_url":"https://api.github.com/repos/vermaseren/form/issues/215/events","html_url":"https://github.com/vermaseren/form/issues/215","id":241343179,"number":215,"title":"Escaping from stage4 sortings for $-variables?","user":{"login":"tueda","id":1481665,"avatar_url":"https://avatars0.githubusercontent.com/u/1481665?v=4","gravatar_id":"","url":"https://api.github.com/users/tueda","html_url":"https://github.com/tueda","followers_url":"https://api.github.com/users/tueda/followers","following_url":"https://api.github.com/users/tueda/following{/other_user}","gists_url":"https://api.github.com/users/tueda/gists{/gist_id}","starred_url":"https://api.github.com/users/tueda/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tueda/subscriptions","organizations_url":"https://api.github.com/users/tueda/orgs","repos_url":"https://api.github.com/users/tueda/repos","events_url":"https://api.github.com/users/tueda/events{/privacy}","received_events_url":"https://api.github.com/users/tueda/received_events","type":"User","site_admin":false},"labels":[{"id":55951994,"url":"https://api.github.com/repos/vermaseren/form/labels/question","name":"question","color":"cc317c","default":true}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":10,"created_at":"2017-07-07T18:25:40Z","updated_at":"2018-02-14T13:20:22Z","closed_at":null,"author_association":"COLLABORATOR","body":"Because of really dangerous memory bugs of #211 for $-variables, I would like to avoid stage4 sortings as much as possible. But I am not sure which parameters are really relevant for that. And it is preferable to save the memory usage. Suppose that I don't have any big local expressions, but have $-variables which become potentially very long polynomials. Does the following setting make sense?\r\n```\r\n* Adopt the default values.\r\n#:filepatches             256\r\n#:largepatches            256\r\n#:largesize          50000000\r\n#:smallextension     20000000\r\n#:smallsize          10000000\r\n#:sortiosize           100000\r\n#:termsinsmall         100000\r\n\r\n* Try to avoid stage4 sortings for $-variables.\r\n* Setting the same values as non-sub parameters may save us from\r\n* the FORM's confusion of buffer sizes(?)\r\n#:subfilepatches          256\r\n#:sublargepatches         256\r\n#:sublargesize       50000000\r\n#:subsmallextension  20000000\r\n#:subsmallsize       10000000\r\n#:subsortiosize        100000\r\n#:subtermsinsmall      100000\r\n\r\n* Slightly smaller maxtermsize for each monomial in polynomials.\r\n#:maxtermsize           10000\r\n```\r\nwhich consumes 407MB of virtual memory for an empty program. Or better recommendation?"},"comment":{"url":"https://api.github.com/repos/vermaseren/form/issues/comments/365604753","html_url":"https://github.com/vermaseren/form/issues/215#issuecomment-365604753","issue_url":"https://api.github.com/repos/vermaseren/form/issues/215","id":365604753,"user":{"login":"vermaseren","id":5435666,"avatar_url":"https://avatars1.githubusercontent.com/u/5435666?v=4","gravatar_id":"","url":"https://api.github.com/users/vermaseren","html_url":"https://github.com/vermaseren","followers_url":"https://api.github.com/users/vermaseren/followers","following_url":"https://api.github.com/users/vermaseren/following{/other_user}","gists_url":"https://api.github.com/users/vermaseren/gists{/gist_id}","starred_url":"https://api.github.com/users/vermaseren/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/vermaseren/subscriptions","organizations_url":"https://api.github.com/users/vermaseren/orgs","repos_url":"https://api.github.com/users/vermaseren/repos","events_url":"https://api.github.com/users/vermaseren/events{/privacy}","received_events_url":"https://api.github.com/users/vermaseren/received_events","type":"User","site_admin":false},"created_at":"2018-02-14T13:20:21Z","updated_at":"2018-02-14T13:20:21Z","author_association":"OWNER","body":"Hi Stephen,\n\nThe problem is historical.\nOriginally Form did not have the hide system. Hence in each module it had to consider\nwhat to do with each expression (act on it, skip it or drop it). As acting and skipping each\ntake much more time than looking what to do, there was no problem.\nThen the hide system came and now, if you have 30000 expressions in the hide, it\nlooks 30000 times and sees that it has to do nothing. In the example I gave, there were\n30000 modules and hence it sees 30000^2 times that it can go on. Even though that\nloop does not take much time, each nsec translates into almost a whole sec in real time.\nAt the moment I am thinking what would be a not too complicated way to get around this.\nMaking a double administration could introduce errors easily. The ideal would of course be\nthat the Hide example I gave is about as fast as the drop case.\n\nIn principle I could make an option that disables stage4 for dollars and function arguments,\neven though I did spend quite some time to get that to run on a number of examples, but that\nis of course never the same as having it running in real calculations. And in real calculations\nit is very rare. (Same as stage5 in a regular expression which I ran into only once in a real\ncalculation. It worked though.)\nThe program prints a message though when entering stage4. Hence if you grep your output\nfile for it, you will know whether you got there.\n\nCheers\n\nJos\n \n> On 14 Feb 2018, at 14:05, Stephen Jones <notifications@github.com> wrote:\n> \n> Dear @vermaseren <https://github.com/vermaseren>,\n> \n> Thank you very much for the changes (a261560 <https://github.com/vermaseren/form/commit/a261560b11a71bf825487e32cec5358ac1c76c4d>, 536e778 <https://github.com/vermaseren/form/commit/536e77866ee1c9e54476a3463754586b2bf82acb>, 7599180 <https://github.com/vermaseren/form/commit/7599180782adfa9ca63b9d9cf1de0fd4ff590f21>)! They hugely reduce the time for our code to run and have moved the bottleneck elsewhere. I suppose that now I will not rewrite our code to use $-variables, instead I will leave it using expressions.\n> \n> In your response you mentioned that the Hide trick \"creates 30000 modules in each of which it has to run through the list of variables to see that for almost all it has nothing to do, again something of O(30000^2)\". Did you mean that in each module it has to run through the list of expressions ?\n> \n> Regarding stage4 for dollars, as I will continue to use expressions I am currently not too worried about the memory errors discussed in #211 <https://github.com/vermaseren/form/issues/211> . Originally I was concerned that if I rewrite my code to use $-variables it may start to give wrong results just because $-variables accidentally enter stage4. Personally I would rather FORM crash or print a warning than give wrong results. My original question just wondered if there is a way for us to detect that something may have gone wrong due to $-variables entering stage4.\n> \n> —\n> You are receiving this because you were mentioned.\n> Reply to this email directly, view it on GitHub <https://github.com/vermaseren/form/issues/215#issuecomment-365601355>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AFLxEiyKHB3mxoH1vcmogSXDl1GBmSXqks5tUtoqgaJpZM4ORR4n>.\n> \n\n"}}
{"actor":{"display_login":"vermaseren"}}