{"action":"created","issue":{"url":"https://api.github.com/repos/dlt-rilmta/hunlp-GATE/issues/13","repository_url":"https://api.github.com/repos/dlt-rilmta/hunlp-GATE","labels_url":"https://api.github.com/repos/dlt-rilmta/hunlp-GATE/issues/13/labels{/name}","comments_url":"https://api.github.com/repos/dlt-rilmta/hunlp-GATE/issues/13/comments","events_url":"https://api.github.com/repos/dlt-rilmta/hunlp-GATE/issues/13/events","html_url":"https://github.com/dlt-rilmta/hunlp-GATE/issues/13","id":193486501,"number":13,"title":"OutOfMemoryError","user":{"login":"DavidNemeskey","id":690386,"avatar_url":"https://avatars.githubusercontent.com/u/690386?v=3","gravatar_id":"","url":"https://api.github.com/users/DavidNemeskey","html_url":"https://github.com/DavidNemeskey","followers_url":"https://api.github.com/users/DavidNemeskey/followers","following_url":"https://api.github.com/users/DavidNemeskey/following{/other_user}","gists_url":"https://api.github.com/users/DavidNemeskey/gists{/gist_id}","starred_url":"https://api.github.com/users/DavidNemeskey/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/DavidNemeskey/subscriptions","organizations_url":"https://api.github.com/users/DavidNemeskey/orgs","repos_url":"https://api.github.com/users/DavidNemeskey/repos","events_url":"https://api.github.com/users/DavidNemeskey/events{/privacy}","received_events_url":"https://api.github.com/users/DavidNemeskey/received_events","type":"User","site_admin":false},"labels":[{"id":202740455,"url":"https://api.github.com/repos/dlt-rilmta/hunlp-GATE/labels/bug","name":"bug","color":"fc2929","default":true}],"state":"open","locked":false,"assignee":{"login":"sassbalint","id":12102636,"avatar_url":"https://avatars.githubusercontent.com/u/12102636?v=3","gravatar_id":"","url":"https://api.github.com/users/sassbalint","html_url":"https://github.com/sassbalint","followers_url":"https://api.github.com/users/sassbalint/followers","following_url":"https://api.github.com/users/sassbalint/following{/other_user}","gists_url":"https://api.github.com/users/sassbalint/gists{/gist_id}","starred_url":"https://api.github.com/users/sassbalint/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sassbalint/subscriptions","organizations_url":"https://api.github.com/users/sassbalint/orgs","repos_url":"https://api.github.com/users/sassbalint/repos","events_url":"https://api.github.com/users/sassbalint/events{/privacy}","received_events_url":"https://api.github.com/users/sassbalint/received_events","type":"User","site_admin":false},"assignees":[{"login":"sassbalint","id":12102636,"avatar_url":"https://avatars.githubusercontent.com/u/12102636?v=3","gravatar_id":"","url":"https://api.github.com/users/sassbalint","html_url":"https://github.com/sassbalint","followers_url":"https://api.github.com/users/sassbalint/followers","following_url":"https://api.github.com/users/sassbalint/following{/other_user}","gists_url":"https://api.github.com/users/sassbalint/gists{/gist_id}","starred_url":"https://api.github.com/users/sassbalint/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sassbalint/subscriptions","organizations_url":"https://api.github.com/users/sassbalint/orgs","repos_url":"https://api.github.com/users/sassbalint/repos","events_url":"https://api.github.com/users/sassbalint/events{/privacy}","received_events_url":"https://api.github.com/users/sassbalint/received_events","type":"User","site_admin":false}],"milestone":null,"comments":3,"created_at":"2016-12-05T12:39:21Z","updated_at":"2016-12-06T16:28:15Z","closed_at":null,"body":"I am trying the process the [Webcorpus](http://mokk.bme.hu/resources/webcorpus/) with hunlp-GATE (QT, HFST, PurePOS). I regularly run into `OutOfMemoryError`s. I am running GATE as a server and I send the data 20k-character chunks at a time.\r\n\r\nI log the input and the output XML, as well, and when I send to the server the input that supposedly caused the error, it runs without a hitch. This shows that the problem is most likely caused by a memory leak somewhere in GATE or in one of the components I use:\r\n\r\n- QunToken: not Java, the wrapper is a single source file, so I would be surprised...\r\n- HFST: not Java, the wrapper consists of several files, but again unlikely\r\n- PurePOS: implemented in Java, so the most likely candidate thus far (aside from GATE itself, that is...).\r\n\r\nI run the server with `-Xmx20g`.\r\n\r\nThis trace seems to point to PurePOS as well, but it can also be a coincidence.\r\n```\r\nException in thread \"pool-1-thread-1\" java.lang.OutOfMemoryError: GC overhead limit exceeded\r\n        at java.lang.AbstractStringBuilder.<init>(AbstractStringBuilder.java:68)\r\n        at java.lang.StringBuilder.<init>(StringBuilder.java:89)\r\n        at org.apache.commons.lang3.tuple.Pair.toString(Pair.java:162)\r\n        at hu.ppke.itk.nlpg.purepos.common.lemma.AbstractLemmaTransformation.toString(AbstractLemmaTransformation.java:64)\r\n        at hu.ppke.itk.nlpg.purepos.common.lemma.AbstractLemmaTransformation.hashCode(AbstractLemmaTransformation.java:79)\r\n        at java.util.HashMap.hash(HashMap.java:338)\r\n        at java.util.HashMap.get(HashMap.java:556)\r\n        at hu.ppke.itk.nlpg.purepos.model.internal.HashSuffixGuesser.getTagProbabilities(HashSuffixGuesser.java:129)\r\n        at hu.ppke.itk.nlpg.purepos.model.internal.HashSuffixGuesser.getTagLogProbabilities(HashSuffixGuesser.java:86)\r\n        at hu.ppke.itk.nlpg.purepos.model.internal.HashSuffixGuesser.getTagLogProbabilities(HashSuffixGuesser.java:44)\r\n        at hu.ppke.itk.nlpg.purepos.MorphTagger.findBestLemma(MorphTagger.java:116)\r\n        at hu.ppke.itk.nlpg.purepos.MorphTagger.merge(MorphTagger.java:70)\r\n        at hu.ppke.itk.nlpg.purepos.POSTagger.tagSentence(POSTagger.java:110)\r\n        at hu.ppke.itk.nlpg.purepos.POSTagger.tagSentence(POSTagger.java:100)\r\n        at hu.nytud.gate.postaggers.Magyarlanc3POSTaggerLemmatizer.execute(Magyarlanc3POSTaggerLemmatizer.java:105)\r\n        at gate.server.RequestHandler.process(RequestHandler.java:128)\r\n        at gate.server.RequestHandler.handle(RequestHandler.java:82)\r\n        at com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:79)\r\n        at sun.net.httpserver.AuthFilter.doFilter(AuthFilter.java:83)\r\n        at com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:82)\r\n        at sun.net.httpserver.ServerImpl$Exchange$LinkHandler.handle(ServerImpl.java:675)\r\n        at com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:79)\r\n        at sun.net.httpserver.ServerImpl$Exchange.run(ServerImpl.java:647)\r\n        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)\r\n        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)\r\n        at java.lang.Thread.run(Thread.java:745)\r\n```"},"comment":{"url":"https://api.github.com/repos/dlt-rilmta/hunlp-GATE/issues/comments/265196887","html_url":"https://github.com/dlt-rilmta/hunlp-GATE/issues/13#issuecomment-265196887","issue_url":"https://api.github.com/repos/dlt-rilmta/hunlp-GATE/issues/13","id":265196887,"user":{"login":"sassbalint","id":12102636,"avatar_url":"https://avatars.githubusercontent.com/u/12102636?v=3","gravatar_id":"","url":"https://api.github.com/users/sassbalint","html_url":"https://github.com/sassbalint","followers_url":"https://api.github.com/users/sassbalint/followers","following_url":"https://api.github.com/users/sassbalint/following{/other_user}","gists_url":"https://api.github.com/users/sassbalint/gists{/gist_id}","starred_url":"https://api.github.com/users/sassbalint/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sassbalint/subscriptions","organizations_url":"https://api.github.com/users/sassbalint/orgs","repos_url":"https://api.github.com/users/sassbalint/repos","events_url":"https://api.github.com/users/sassbalint/events{/privacy}","received_events_url":"https://api.github.com/users/sassbalint/received_events","type":"User","site_admin":false},"created_at":"2016-12-06T16:28:15Z","updated_at":"2016-12-06T16:28:15Z","body":"@temprimus Could you please take a look at this issue? Do you have any ideas? Thank you."}}
{"actor":{"display_login":"sassbalint"}}