{"action":"created","issue":{"url":"https://api.github.com/repos/rafaelbetatester/FreeLing/issues/2","repository_url":"https://api.github.com/repos/rafaelbetatester/FreeLing","labels_url":"https://api.github.com/repos/rafaelbetatester/FreeLing/issues/2/labels{/name}","comments_url":"https://api.github.com/repos/rafaelbetatester/FreeLing/issues/2/comments","events_url":"https://api.github.com/repos/rafaelbetatester/FreeLing/issues/2/events","html_url":"https://github.com/rafaelbetatester/FreeLing/issues/2","id":151002703,"number":2,"title":"affixation rules","user":{"login":"rafaelbetatester","id":11369578,"avatar_url":"https://avatars.githubusercontent.com/u/11369578?v=3","gravatar_id":"","url":"https://api.github.com/users/rafaelbetatester","html_url":"https://github.com/rafaelbetatester","followers_url":"https://api.github.com/users/rafaelbetatester/followers","following_url":"https://api.github.com/users/rafaelbetatester/following{/other_user}","gists_url":"https://api.github.com/users/rafaelbetatester/gists{/gist_id}","starred_url":"https://api.github.com/users/rafaelbetatester/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rafaelbetatester/subscriptions","organizations_url":"https://api.github.com/users/rafaelbetatester/orgs","repos_url":"https://api.github.com/users/rafaelbetatester/repos","events_url":"https://api.github.com/users/rafaelbetatester/events{/privacy}","received_events_url":"https://api.github.com/users/rafaelbetatester/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":4,"created_at":"2016-04-25T23:05:17Z","updated_at":"2016-04-25T23:05:24Z","closed_at":null,"body":"<a href=\"https://github.com/arademaker\"><img src=\"https://avatars.githubusercontent.com/u/86403?v=3\" align=\"left\" width=\"96\" height=\"96\" hspace=\"10\"></img></a> **Issue by [arademaker](https://github.com/arademaker)**\n_Friday Feb 19, 2016 at 20:34 GMT_\n_Originally opened as https://github.com/own-pt/freeling_pt/issues/2_\n\n----\n\nLivy:\r\n\r\n3- Affixation rules and dictionary\r\n\r\nThinking, for example, on diminutives. We have in the dictionary many entries with diminutives:\r\n\r\n```\r\nabadessazinha abade NCFS00D\r\nabadessazinhas abade NCFP00D\r\nabadessazita abade NCFS00D\r\nabadessazitas abade NCFP00D\r\nabadezinho abade NCMS00D\r\nabadezinhos abade NCMP00D\r\nabadezito abade NCMS00D\r\nabadezitos abade NCMP00D\r\n```\r\nBut for the same suffix (-inha/-inho), we also have Affixation Rules:\r\n```\r\ninhas a|e|i|o|u ^AQ0 AQDFP0 1 0 0 L 0 -\t\r\ninha a|e|i|o|u ^AQ0 AQDFS0 1 0 0 L 0 -\r\ninhos a|e|i|o|u ^AQ0 AQDMP0 1 0 0 L 0 -\r\ninho a|e|i|o|u ^AQ0 AQDMS0 1 0 0 L 0 -\r\n```\r\nI want to know what is really working with diminutives.\r\n\r\nPadro:\r\n\r\nThat depends on the suffix rule. The general procedure is:\r\n\r\n- A word is looked up in the dictionary and if it is there, it gets the found analysis.\r\n- Then suffix rules are applied.  \r\n  * If the rule has mark \"always apply\", it is applied to the word, even if it has been found in the dictionary. Analysis resulting from the rule are added to the list.\r\n  * If the rule has mark \"apply only to unknown words\", the rule is applied only if the word was not found in the dictionary.\r\n\r\nSo, for most diminutives, removing them from dictionary should work, provided there is a suffix rule to generate them from the root.\r\n\r\nLivy:\r\n\r\nAlmost all diminutive words were captured, as from our 7 million words corpus, we did not capture few diminutive words (we did not capture less than 15). Examples that should have been captured, but they don't: \"bandinha\" and \"telinha\", as in the dictionary we have \"banda\" and \"tela\". There are more complex examples, as \"rapidinho\" and \"pracinha\", that also are present in the dictionary (\"rápido/praça\"), but the root of those words change under diminutive suffixes and then maybe they are more difficult to capture.\r\n\r\nPadro:\r\n\r\nThat requires only that you add or adjust the suffix rules.\r\n\r\nE.g. the rule that should analize \"bandinha\" is:    \r\n```\r\ninha    *       ^NC     NCFS00D 1       0       0       L 0     -\r\n```\r\nThis rule is wrong. It is read as \"given a word terminated in -inha, remove the suffix -inha, add \"*\" (i.e. nothing), and check whether the result is a NC.\" So, bandinha - inha --> band;   band + * = band;  band is not in the dictionary as NC, the rule is not applied. The right rule is \r\n```\r\ninha    a       ^NC     NCFS00D 1       0       0       L 0     -\r\n```\r\nthat would make:  bandinha - inha = band;  band+a = banda;  banda is in dictionary as NC, so the rule would be applied\r\n\r\nTo deal with cases as \"pracinha\", you need a rule that treats that specific ortographic change (that is, that replaces the termination -cinha with -ça before checking the dictionary)\r\n```\r\ncinha    ça       ^NC     NCFS00D 1       0       0       L 0     -\r\n```\r\nFor rápido -> rapidinho, one of the 0/1 flags in the rule indicates whether adding accents to the root should be tried. Probably setting it to 1 would solve the problem.\r\n\r\nYou can get a better insight about how rules work checking the user manual :\r\nhttp://nlp.lsi.upc.edu/freeling/doc/userman/html/node34.html\n"},"comment":{"url":"https://api.github.com/repos/rafaelbetatester/FreeLing/issues/comments/214557613","html_url":"https://github.com/rafaelbetatester/FreeLing/issues/2#issuecomment-214557613","issue_url":"https://api.github.com/repos/rafaelbetatester/FreeLing/issues/2","id":214557613,"user":{"login":"rafaelbetatester","id":11369578,"avatar_url":"https://avatars.githubusercontent.com/u/11369578?v=3","gravatar_id":"","url":"https://api.github.com/users/rafaelbetatester","html_url":"https://github.com/rafaelbetatester","followers_url":"https://api.github.com/users/rafaelbetatester/followers","following_url":"https://api.github.com/users/rafaelbetatester/following{/other_user}","gists_url":"https://api.github.com/users/rafaelbetatester/gists{/gist_id}","starred_url":"https://api.github.com/users/rafaelbetatester/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rafaelbetatester/subscriptions","organizations_url":"https://api.github.com/users/rafaelbetatester/orgs","repos_url":"https://api.github.com/users/rafaelbetatester/repos","events_url":"https://api.github.com/users/rafaelbetatester/events{/privacy}","received_events_url":"https://api.github.com/users/rafaelbetatester/received_events","type":"User","site_admin":false},"created_at":"2016-04-25T23:05:24Z","updated_at":"2016-04-25T23:05:24Z","body":"<a href=\"https://github.com/livyreal\"><img src=\"https://avatars.githubusercontent.com/u/5545847?v=3\" align=\"left\" width=\"48\" height=\"48\" hspace=\"10\"></img></a> **Comment by [livyreal](https://github.com/livyreal)**\n_Thursday Mar 31, 2016 at 17:01 GMT_\n\n----\n\nTokens e Lemas para a regra prévia e a do alexandre:\r\n\r\n> arqui-       *    ^N|A   *    0    0    1    R    1   arqui-+$$:AQ0CN0+*\r\n\r\narqui-inimigo\r\nusa a regra mesmo se a palavra estiver no dic\r\ntoken: (arqui)+(inimigo)\r\nlema: inimigo\r\n\r\n> 1. ex-           *    ^N|A   *    0    0    1    A+L  0   -\r\n\r\nex-governador\r\nnao usa a regra se achar no dicionario \r\ntoken: ex-governador\r\nlema: ex+gov\r\n\r\n\r\nEu concordo com a regra do Alexandre, sempre que achar no dicionário, não precisa usar regras, lema é (prefixo)+(palavra), token é a expressão toda (prefixo+palavra).\r\n\r\nEntão, o padrão:\r\n\r\n>prefixo- * ^N|A * 0 0 1 A+L 0 -\r\n\r\npoderia ser o padrão básico para os prefixos achados no Bosque[1]:\r\n\r\nlema = afixo + lema\r\ntoken = - (a expressão que a regra pegou)\r\n\r\nOk, mas em que momento entra a PoS tag do afixo? (que na regra anterior entrava na retokeninzação) Neste caso, me parece, precisamos ter os afixos no dicionário.\r\n\r\nApós conversa com AR, optamos por deixar todas as regras uniformes e sem adicionar os afixos no dicionário. O PoS tag da palavra com prefixo será o mesmo PoS do lema.\r\n\r\n>[1] re, contra, mal, ex, pré, vice, inter, anti, pós, recém, neo, sub, intra, arqui\r\n\r\n\n"}}