{"action":"created","issue":{"url":"https://api.github.com/repos/statsmodels/statsmodels/issues/5328","repository_url":"https://api.github.com/repos/statsmodels/statsmodels","labels_url":"https://api.github.com/repos/statsmodels/statsmodels/issues/5328/labels{/name}","comments_url":"https://api.github.com/repos/statsmodels/statsmodels/issues/5328/comments","events_url":"https://api.github.com/repos/statsmodels/statsmodels/issues/5328/events","html_url":"https://github.com/statsmodels/statsmodels/issues/5328","id":371694025,"node_id":"MDU6SXNzdWUzNzE2OTQwMjU=","number":5328,"title":"Determine with what certainty one distribution can be chosen as a better aproximator for a desired value than another distribution.","user":{"login":"TheChymera","id":950524,"node_id":"MDQ6VXNlcjk1MDUyNA==","avatar_url":"https://avatars2.githubusercontent.com/u/950524?v=4","gravatar_id":"","url":"https://api.github.com/users/TheChymera","html_url":"https://github.com/TheChymera","followers_url":"https://api.github.com/users/TheChymera/followers","following_url":"https://api.github.com/users/TheChymera/following{/other_user}","gists_url":"https://api.github.com/users/TheChymera/gists{/gist_id}","starred_url":"https://api.github.com/users/TheChymera/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/TheChymera/subscriptions","organizations_url":"https://api.github.com/users/TheChymera/orgs","repos_url":"https://api.github.com/users/TheChymera/repos","events_url":"https://api.github.com/users/TheChymera/events{/privacy}","received_events_url":"https://api.github.com/users/TheChymera/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":16,"created_at":"2018-10-18T20:14:43Z","updated_at":"2018-10-19T16:56:57Z","closed_at":null,"author_association":"NONE","body":"I have the following situation, I am testing two variations of a process against each other. I know that the ideal outcome value is 1. I would like to know which process I can prefer (and with what certainty) based on a sampling of their output values.\r\n\r\n(for the same set of inputs, so there's a common random factor, but that bit I think is trivial).\r\n\r\nPlease see the figure below and find a full minimal working example [here](http://chymera.eu/debug/stats/least_different.zip).\r\n\r\n![boxplot](https://user-images.githubusercontent.com/950524/47180696-7380c300-d321-11e8-9df8-408400ad60d7.png)\r\n\r\nThe way I see it (for a given level of the process variation variable) H0 is that there is a greater or equal effect of the other level.\r\nSince I am unsure how to encode this variable I have opted to instead test the (imho incomplete) H0 that there is no effect for the given level and that it is not significantly different from the other level. This still lacks the constraint that there shouldn't be a greater effect from the other level. I have no idea how that could be encoded.\r\n\r\nAt any rate I fail long before getting there.\r\n\r\nI have tried to subtract one (to make zero the ideal value), and to model the data without an intercept, so I can handle the two levels individually, but given the code:\r\n\r\n```python\r\nimport pandas as pd\r\nfrom os import path\r\nimport numpy as np\r\nimport pandas as pd\r\nimport statsmodels.formula.api as smf\r\n\r\nvolume_path = path.abspath('volumes.csv')\r\ndf = pd.read_csv(volume_path)\r\n\r\ndf = df.loc[df['Processing']!='Unprocessed']\r\ndf = df.loc[((df['Processing']=='Legacy') & (df['Template']=='Legacy')) | ((df['Processing']=='Generic') & (df['Template']=='Generic'))]\r\n\r\ndf['Volume Change Factor'] = df['Volume Change Factor']-1\r\n\r\nmodel=smf.mixedlm('Q(\"Volume Change Factor\") ~ Processing-1', df, groups='uID')\r\n\r\nfit = model.fit()\r\nsummary = fit.summary()\r\nprint(fit)\r\nprint(summary)\r\n\r\nomnibus_tests = np.eye(len(fit.params))\r\nomnibus_tests = np.delete(omnibus_tests, (2), axis=0)\r\nomnibus_tests[0,1] = -1\r\nprint(\"Generic better than Legacy?\")\r\nprint(omnibus_tests)\r\nanova = fit.f_test(omnibus_tests)\r\nprint(anova)\r\n\r\nomnibus_tests = np.eye(len(fit.params))\r\nomnibus_tests = np.delete(omnibus_tests, (2), axis=0)\r\nomnibus_tests[0,1] = -1\r\nomnibus_tests[1,0] = 1\r\nomnibus_tests[1,1] = 0\r\nprint(\"Legacy better than Generic?\")\r\nprint(omnibus_tests)\r\nanova = fit.f_test(omnibus_tests)\r\nprint(anova)\r\n```\r\n\r\nI get \r\n\r\n```\r\n/usr/lib64/python3.6/site-packages/statsmodels/regression/mixed_linear_model.py:2092: ConvergenceWarning: The MLE may be on the boundary of the parameter space.\r\n  warnings.warn(msg, ConvergenceWarning)\r\n<statsmodels.regression.mixed_linear_model.MixedLMResultsWrapper object at 0x7f57bc5d5f98>\r\n                 Mixed Linear Model Regression Results\r\n=======================================================================\r\nModel:            MixedLM Dependent Variable: Q(\"Volume Change Factor\")\r\nNo. Observations: 136     Method:             REML                     \r\nNo. Groups:       68      Scale:              0.0038                   \r\nMin. group size:  2       Likelihood:         160.0737                 \r\nMax. group size:  2       Converged:          Yes                      \r\nMean group size:  2.0                                                  \r\n------------------------------------------------------------------------\r\n                        Coef.   Std.Err.     z     P>|z|  [0.025  0.975]\r\n------------------------------------------------------------------------\r\nProcessing[Generic]      0.036     0.009    4.071  0.000   0.019   0.053\r\nProcessing[Legacy]      -0.275     0.009  -31.364  0.000  -0.293  -0.258\r\nuID Var                  0.001     0.013                                \r\n=======================================================================\r\n\r\nGeneric better than Legacy?\r\n[[ 1. -1.  0.]\r\n [ 0.  1.  0.]]\r\n<F test: F=array([[578.95292511]]), p=1.1575867403053478e-66, df_denom=134, df_num=2>\r\nLegacy better than Generic?\r\n[[ 1. -1.  0.]\r\n [ 1.  0.  0.]]\r\n<F test: F=array([[578.95292511]]), p=1.1575867403053893e-66, df_denom=134, df_num=2>\r\n```\r\n\r\nBased on this it seems that both levels are almost equal in how well they approximate the value, but that is clearly not the case, as seen in the figure.\r\n\r\nAny idea how I can perform this comparison with statsmodels? "},"comment":{"url":"https://api.github.com/repos/statsmodels/statsmodels/issues/comments/431429187","html_url":"https://github.com/statsmodels/statsmodels/issues/5328#issuecomment-431429187","issue_url":"https://api.github.com/repos/statsmodels/statsmodels/issues/5328","id":431429187,"node_id":"MDEyOklzc3VlQ29tbWVudDQzMTQyOTE4Nw==","user":{"login":"TheChymera","id":950524,"node_id":"MDQ6VXNlcjk1MDUyNA==","avatar_url":"https://avatars2.githubusercontent.com/u/950524?v=4","gravatar_id":"","url":"https://api.github.com/users/TheChymera","html_url":"https://github.com/TheChymera","followers_url":"https://api.github.com/users/TheChymera/followers","following_url":"https://api.github.com/users/TheChymera/following{/other_user}","gists_url":"https://api.github.com/users/TheChymera/gists{/gist_id}","starred_url":"https://api.github.com/users/TheChymera/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/TheChymera/subscriptions","organizations_url":"https://api.github.com/users/TheChymera/orgs","repos_url":"https://api.github.com/users/TheChymera/repos","events_url":"https://api.github.com/users/TheChymera/events{/privacy}","received_events_url":"https://api.github.com/users/TheChymera/received_events","type":"User","site_admin":false},"created_at":"2018-10-19T16:56:57Z","updated_at":"2018-10-19T16:56:57Z","author_association":"NONE","body":"> An assumption here is that there WILL be deviance (because of measurement errors if nothing else). It seems the key metric would be average deviance or average squared deviance -- Would just a standard deviation (vs 1) or mean absolute error be sufficient?\r\n\r\nThat would be a ratio of t-values. How would I interpret that?"}}
{"actor":{"display_login":"TheChymera"}}