{"action":"created","issue":{"url":"https://api.github.com/repos/gap-system/gap/issues/2377","repository_url":"https://api.github.com/repos/gap-system/gap","labels_url":"https://api.github.com/repos/gap-system/gap/issues/2377/labels{/name}","comments_url":"https://api.github.com/repos/gap-system/gap/issues/2377/comments","events_url":"https://api.github.com/repos/gap-system/gap/issues/2377/events","html_url":"https://github.com/gap-system/gap/issues/2377","id":315832040,"number":2377,"title":"Regression in testextra/grpauto.tst (memory usage)","user":{"login":"alex-konovalov","id":5751387,"avatar_url":"https://avatars1.githubusercontent.com/u/5751387?v=4","gravatar_id":"","url":"https://api.github.com/users/alex-konovalov","html_url":"https://github.com/alex-konovalov","followers_url":"https://api.github.com/users/alex-konovalov/followers","following_url":"https://api.github.com/users/alex-konovalov/following{/other_user}","gists_url":"https://api.github.com/users/alex-konovalov/gists{/gist_id}","starred_url":"https://api.github.com/users/alex-konovalov/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/alex-konovalov/subscriptions","organizations_url":"https://api.github.com/users/alex-konovalov/orgs","repos_url":"https://api.github.com/users/alex-konovalov/repos","events_url":"https://api.github.com/users/alex-konovalov/events{/privacy}","received_events_url":"https://api.github.com/users/alex-konovalov/received_events","type":"User","site_admin":false},"labels":[{"id":502555547,"url":"https://api.github.com/repos/gap-system/gap/labels/packages","name":"packages","color":"d4c5f9","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":{"url":"https://api.github.com/repos/gap-system/gap/milestones/14","html_url":"https://github.com/gap-system/gap/milestone/14","labels_url":"https://api.github.com/repos/gap-system/gap/milestones/14/labels","id":2420535,"number":14,"title":"GAP 4.9.1","description":"First public release of GAP 4.9","creator":{"login":"alex-konovalov","id":5751387,"avatar_url":"https://avatars1.githubusercontent.com/u/5751387?v=4","gravatar_id":"","url":"https://api.github.com/users/alex-konovalov","html_url":"https://github.com/alex-konovalov","followers_url":"https://api.github.com/users/alex-konovalov/followers","following_url":"https://api.github.com/users/alex-konovalov/following{/other_user}","gists_url":"https://api.github.com/users/alex-konovalov/gists{/gist_id}","starred_url":"https://api.github.com/users/alex-konovalov/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/alex-konovalov/subscriptions","organizations_url":"https://api.github.com/users/alex-konovalov/orgs","repos_url":"https://api.github.com/users/alex-konovalov/repos","events_url":"https://api.github.com/users/alex-konovalov/events{/privacy}","received_events_url":"https://api.github.com/users/alex-konovalov/received_events","type":"User","site_admin":false},"open_issues":10,"closed_issues":17,"state":"open","created_at":"2017-03-29T21:59:07Z","updated_at":"2018-04-19T16:10:37Z","due_on":"2018-04-01T07:00:00Z","closed_at":null},"comments":10,"created_at":"2018-04-19T11:13:43Z","updated_at":"2018-04-20T18:40:49Z","closed_at":null,"author_association":"OWNER","body":"The following is observed in `teststandard` tests on Jenkins:\r\n```\r\n########> Diff in /circa/scratch/gap-jenkins/workspace/GAP-master-test/GAPCOPT\\\r\nS/64build/GAPTARGET/standard/label/kovacs/GAP-master-snapshot/tst/testextra/gr\\\r\npauto.tst:105\r\n# Input is:\r\nIsomorphismGroups(G,PcGroupCode(CodePcGroup(G),Size(G)))=fail;\r\n# Expected output:\r\nfalse\r\n# But found:\r\nError, reached the pre-set memory limit\r\n(change it with the -o command line option)\r\n########\r\n```\r\nIn the master branch this appens on both macos and linux jenkins slaves. In the stable-4.9 branch - only on macos. I've tried to run just that example starting GAP with `-o 1g` option and so war was able to reproduce this problem only once. Apparently, there is some interaction between other tests and loaded packages, and balancing around the 1g limit with some randomness and garbage collections making this hard to reproduce.\r\n\r\nThis is the input:\r\n```\r\nLoadAllPackages();\r\nG:=PcGroupCode( 741231213963541373679312045151639276850536621925972119311,11664);;\r\nH:=PcGroupCode( 888658311993669104086576972570546890038187728096037768975,11664);;\r\nIsomorphismGroups(G,H);\r\nIsomorphismGroups(G,PcGroupCode(CodePcGroup(G),Size(G)))=fail;\r\n```\r\nand this is the memory usage that I observe under macos:\r\n```\r\nLoad GAP                131           131 \r\nLoad all packages       ---           344\r\nAfter 1st iso test      142           436\r\nAfter 2nd iso test      548           1.01g\r\n```\r\n(the last 1.01g value is measured after a break loop happened and I've entered \"return\" to resume the calculation).\r\n\r\nThe problem can be reproduced easier if GAP is started with just slightly smaller memory limit `-o 1000m`. Then after `Error, reached the pre-set memory limit` I see this:\r\n\r\n```\r\nbrk> Where(50);\r\nImagesRepresentative( map, gen ) at /Users/alexk/GITREPS/gap-stable/lib/mapphomo.gi:311 called from\r\nfunc( C[i] ) at /Users/alexk/GITREPS/gap-stable/lib/coll.gi:746 called from\r\nList( GeneratorsOfMagmaWithInverses( elms ), function ( gen )\r\n      return ImagesRepresentative( map, gen );\r\n  end ) at /Users/alexk/GITREPS/gap-stable/lib/mapphomo.gi:311 called from\r\nImagesSet( map, elm ) at /Users/alexk/GITREPS/gap-stable/lib/mapping.gi:185 called from\r\nImage( hom, sub ) at /Users/alexk/GITREPS/gap-stable/lib/autsr.gi:811 called from\r\nact( orb[p], acts[i] ) at /Users/alexk/GITREPS/gap-stable/lib/oprt.gi:1241 called from\r\nOrbitStabilizerAlgorithm( G, false, false, gens, acts, rec(\r\n    pnt := d,\r\n    act := act,\r\n    onlystab := true ) ) at /Users/alexk/GITREPS/gap-stable/lib/oprt.gi:2909 called from\r\norbish( G, pnt, gens, acts, act ) at /Users/alexk/GITREPS/gap-stable/lib/oprt.gd:866 called from\r\nCallFuncList( StabilizerFunc, arg ) at /Users/alexk/GITREPS/gap-stable/pkg/rcwa-4.6.1/lib/rcwagrp.gi:4586 called from\r\nStabilizer( api, v, gens, pre, asAutomorphism ) at /Users/alexk/GITREPS/gap-stable/lib/autsr.gi:953 called from\r\nPatheticIsomorphism( G, H ) at /Users/alexk/GITREPS/gap-stable/lib/morpheus.gi:2101 called from\r\n<function \"IsomorphismGroups\">( <arguments> )\r\n called from read-eval loop at *errin*:1\r\nbrk> arg[1];\r\n[ f4, f3*f4, f5, f6, f8^2*f10^2*f11, f8*f9^2*f12*f13*f14*f19^2, f13*f14, f13, f16, f15*f16, f8^2*f10*f17*f19^2*f20^2, f18, \r\n  f19, f20 ]\r\nbrk> arg[2];\r\nfunction( gen ) ... end\r\nbrk> Print(last);\r\nfunction ( gen )\r\n    return ImagesRepresentative( map, gen );\r\nend\r\nbrk>\r\n```"},"comment":{"url":"https://api.github.com/repos/gap-system/gap/issues/comments/383186495","html_url":"https://github.com/gap-system/gap/issues/2377#issuecomment-383186495","issue_url":"https://api.github.com/repos/gap-system/gap/issues/2377","id":383186495,"user":{"login":"hulpke","id":7890800,"avatar_url":"https://avatars0.githubusercontent.com/u/7890800?v=4","gravatar_id":"","url":"https://api.github.com/users/hulpke","html_url":"https://github.com/hulpke","followers_url":"https://api.github.com/users/hulpke/followers","following_url":"https://api.github.com/users/hulpke/following{/other_user}","gists_url":"https://api.github.com/users/hulpke/gists{/gist_id}","starred_url":"https://api.github.com/users/hulpke/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hulpke/subscriptions","organizations_url":"https://api.github.com/users/hulpke/orgs","repos_url":"https://api.github.com/users/hulpke/repos","events_url":"https://api.github.com/users/hulpke/events{/privacy}","received_events_url":"https://api.github.com/users/hulpke/received_events","type":"User","site_admin":false},"created_at":"2018-04-20T18:40:49Z","updated_at":"2018-04-20T18:40:49Z","author_association":"CONTRIBUTOR","body":"I've now tried a few runs to check impact of immediate methods. (Yes, I know that I am the curmudgeon who has always whined against the concept of immediate methods, so take the following with a grain of salt if you want so.)\r\n\r\nAfter an initial run to ensure memory has been allocated, I get for the Isomorphism calculation in the master branch (no assertions turned on)\r\n\r\nplain: 170s\r\nImmediate methods turned off: 80s\r\nafter loading semigroups, immediate methods on: 290s\r\nsemigroups, immediate methods off: 265s\r\n\r\nSo there are two issues: Immediate methods in the main library, and ordinary methods in semigroups.\r\n\r\nWhile it is entirely possible that my code is badly written, Is that what we want immediate methods to do?\r\n\r\nWould it make sense to also have a test run with immediate methods turned off to spot such performance hits?\r\n\r\n(Caveat: I have a patch that speeds the whole calculation up far more, so one could have this issue go away. )\r\n\r\nTesting with `TraceImmediateMethods` brings up the following immediate methods\r\n#I  immediate: IsCommutative\r\n#I  immediate: IsCyclic\r\n#I  immediate: IsFinitelyGeneratedGroup\r\n#I  immediate: IsEmpty\r\n#I  immediate: IsTrivial\r\n#I  immediate: IsNonTrivial\r\n#I  immediate: IsPerfectGroup\r\n#I  immediate: IsSolvableGroup\r\n\r\nSome of these are old, some of them have been added just recently. Some likely could be replaces by `TrueMethods`, some by simply setting certain flags when creating groups. (None strikes me as particularly useful for generic calculations, a few actually look a bit ``show-off'')\r\n\r\nWhile they are short, part of the cost is the iterated type change, as a few basic properties are deduced one-by-one.\r\n\r\n#I  immediate: Size\r\n\r\nwould set Size to infinity once IsFinite is set to false. Is this really so valuable?\r\n\r\nThe semigroups package seems to be involved only with\r\n#I  immediate: IsSemigroupIdeal\r\n\r\nShould we make these immediate methods another issue?\r\n\r\n\r\n"}}
{"actor":{"display_login":"hulpke"}}