{"action":"created","issue":{"url":"https://api.github.com/repos/nabeel-oz/qlik-py-tools/issues/58","repository_url":"https://api.github.com/repos/nabeel-oz/qlik-py-tools","labels_url":"https://api.github.com/repos/nabeel-oz/qlik-py-tools/issues/58/labels{/name}","comments_url":"https://api.github.com/repos/nabeel-oz/qlik-py-tools/issues/58/comments","events_url":"https://api.github.com/repos/nabeel-oz/qlik-py-tools/issues/58/events","html_url":"https://github.com/nabeel-oz/qlik-py-tools/issues/58","id":512090834,"node_id":"MDU6SXNzdWU1MTIwOTA4MzQ=","number":58,"title":"Regressor doen't work properly","user":{"login":"SkylakeXx","id":39099082,"node_id":"MDQ6VXNlcjM5MDk5MDgy","avatar_url":"https://avatars0.githubusercontent.com/u/39099082?v=4","gravatar_id":"","url":"https://api.github.com/users/SkylakeXx","html_url":"https://github.com/SkylakeXx","followers_url":"https://api.github.com/users/SkylakeXx/followers","following_url":"https://api.github.com/users/SkylakeXx/following{/other_user}","gists_url":"https://api.github.com/users/SkylakeXx/gists{/gist_id}","starred_url":"https://api.github.com/users/SkylakeXx/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/SkylakeXx/subscriptions","organizations_url":"https://api.github.com/users/SkylakeXx/orgs","repos_url":"https://api.github.com/users/SkylakeXx/repos","events_url":"https://api.github.com/users/SkylakeXx/events{/privacy}","received_events_url":"https://api.github.com/users/SkylakeXx/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":3,"created_at":"2019-10-24T17:52:04Z","updated_at":"2019-10-29T20:46:19Z","closed_at":null,"author_association":"NONE","body":"Hi Nabeel, I have been experiencing difficulties using the regressor, I have used my own data with the classifiers but I have no clue from this point since I;m not able to notice what's the error here (I understand it but works well with any classifier which doesn't makes any sense), I left the code write down:\r\n\r\nI'm using 9 features + 1 target, the regressor is LR but crash at the same point with any of the other choices.\r\n\r\nERROR - Exception iterating responses: 9 columns passed, passed data had 10 columns\r\nTraceback (most recent call last):\r\n  File \"D:\\EDISA\\Python\\qlik-py-tools-5.1\\qlik-py-env\\lib\\site-packages\\pandas\\core\\internals\\construction.py\", line 500, in _list_to_arrays\r\n    content, columns, dtype=dtype, coerce_float=coerce_float\r\n  File \"D:\\EDISA\\Python\\qlik-py-tools-5.1\\qlik-py-env\\lib\\site-packages\\pandas\\core\\internals\\construction.py\", line 583, in _convert_object_array\r\n    \"{con} columns\".format(col=len(columns), con=len(content))\r\nAssertionError: 9 columns passed, passed data had 10 columns\r\n\r\n,"},"comment":{"url":"https://api.github.com/repos/nabeel-oz/qlik-py-tools/issues/comments/547621801","html_url":"https://github.com/nabeel-oz/qlik-py-tools/issues/58#issuecomment-547621801","issue_url":"https://api.github.com/repos/nabeel-oz/qlik-py-tools/issues/58","id":547621801,"node_id":"MDEyOklzc3VlQ29tbWVudDU0NzYyMTgwMQ==","user":{"login":"SkylakeXx","id":39099082,"node_id":"MDQ6VXNlcjM5MDk5MDgy","avatar_url":"https://avatars0.githubusercontent.com/u/39099082?v=4","gravatar_id":"","url":"https://api.github.com/users/SkylakeXx","html_url":"https://github.com/SkylakeXx","followers_url":"https://api.github.com/users/SkylakeXx/followers","following_url":"https://api.github.com/users/SkylakeXx/following{/other_user}","gists_url":"https://api.github.com/users/SkylakeXx/gists{/gist_id}","starred_url":"https://api.github.com/users/SkylakeXx/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/SkylakeXx/subscriptions","organizations_url":"https://api.github.com/users/SkylakeXx/orgs","repos_url":"https://api.github.com/users/SkylakeXx/repos","events_url":"https://api.github.com/users/SkylakeXx/events{/privacy}","received_events_url":"https://api.github.com/users/SkylakeXx/received_events","type":"User","site_admin":false},"created_at":"2019-10-29T20:46:19Z","updated_at":"2019-10-29T20:46:19Z","author_association":"NONE","body":"Here you have the log on debug mode, it includes the definition and scrpts I'm using.\r\n\r\n`SET ThousandSep=',';\r\nSET DecimalSep='.';\r\nSET MoneyThousandSep=',';\r\nSET MoneyDecimalSep='.';\r\nSET MoneyFormat='$#,##0.00;-$#,##0.00';\r\nSET TimeFormat='h:mm:ss TT';\r\nSET DateFormat='M/D/YYYY';\r\nSET TimestampFormat='M/D/YYYY h:mm:ss[.fff] TT';\r\nSET FirstWeekDay=6;\r\nSET BrokenWeeks=1;\r\nSET ReferenceDay=0;\r\nSET FirstMonthOfYear=1;\r\nSET CollationLocale='en-US';\r\nSET CreateSearchIndexOnReload=1;\r\nSET MonthNames='Jan;Feb;Mar;Apr;May;Jun;Jul;Aug;Sep;Oct;Nov;Dec';\r\nSET LongMonthNames='January;February;March;April;May;June;July;August;September;October;November;December';\r\nSET DayNames='Mon;Tue;Wed;Thu;Fri;Sat;Sun';\r\nSET LongDayNames='Monday;Tuesday;Wednesday;Thursday;Friday;Saturday;Sunday';\r\nSET NumericalAbbreviation='3:k;6:M;9:G;12:T;15:P;18:E;21:Z;24:Y;-3:m;-6:µ;-9:n;-12:p;-15:f;-18:a;-21:z;-24:y';\r\n\r\n\r\n[Estimators]:\r\nLOAD\r\n\t[Estimators.Model Name] AS [Model Name],\r\n\t[Estimator];\r\nLOAD * INLINE \r\n[\r\nEstimators.Model Name,Estimator\r\nHR-Attrition-LR-GSCV,LinearRegression\r\n](delimiter is ',');\r\n\r\n[Scaler]:\r\nLOAD\r\n\t[Scaler],\r\n\t[Null Value Handling],\r\n\t[Scale Hashed Features],\r\n\t[Hyperparameters] AS [Scaler.Hyperparameters];\r\nLOAD * INLINE \r\n[\r\nScaler;Null Value Handling;Scale Hashed Features;Hyperparameters\r\nStandardScaler;zeros;true;with_mean=True|bool, with_std=True|bool\r\n](delimiter is ';');\r\n\r\n[Feature Definitions]:\r\nload * Inline [\r\n\tName,\t\t\t\t\tVariable_Type,\t\tData_Type,\t\tFeature_Strategy,\t\tHash_Features\r\n    \"Ventas\",\t\t\t\t\ttarget,\t\t\t\tfloat,\t\t\t\"scaling\"\r\n    \"Fecha\",\t\t\t\tfeature,\t\t\tstr,\t\t\t\"hashing\",\t\t\t\t4\r\n \t\"Tipo Cliente\",\t\t\tfeature,\t\t\tstr,\t\t\t\"one hot enconding\"\r\n     \"Zona\",\t\t\t\tfeature,\t\t\tstr,\t\t\t\"one hot enconding\"\r\n     \"Cliente\",\t\t\t\tfeature,\t\t\tstr,\t\t\t\"hashing\",\t\t\t\t4\r\n     \"R.Produccion\",\t\t\tfeature,\t\t\tstr,\t\t\t\"one hot enconding\"\r\n     \"Fam_desc\",\t\t\t\tfeature,\t\t\tstr,\t\t\t\"hashing\",\t\t\t\t4\r\n     \"Articulo\",\t\t\t\tfeature,\t\t\tstr,\t\t\t\"hashing\",\t\t\t\t4\r\n    \"Agente_Pedido\",\t\tfeature,\t\t\tstr,\t\t\t\"hashing\",\t\t\t\t4\r\n    \"Agente Cliente\",\t\tfeature,\t\t\tstr,\t\t\t\"hashing\",\t\t\t\t4\r\n];\r\n\r\n[Train-Test]:\r\n//sample 0.03\r\nFirst 20\r\nLOAD\r\n    Fecha,\r\n    \"Tipo Cliente\",\r\n    Zona,\r\n    Cliente,\r\n    R.Produccion,\r\n    Fam_desc,\r\n    Articulo,\r\n    Agente_Pedido,\r\n    \"Agente Cliente\",\r\n    Ventas\r\nFROM [lib://DATOS/Datos.xlsx]\r\n(ooxml, embedded labels, table is Sheet1)\r\n;\r\n\r\n\r\n\r\n[Optimization]:\r\nLOAD\r\n\t[Optimization.Model Name] AS [Model Name],\r\n\t[Hyperparameters];\r\nLOAD * INLINE \r\n[\r\nOptimization.Model Name:Hyperparameters\r\nHR-Attrition-LR-GSCV:n_jobs=1|list|int\r\n](delimiter is ':');\r\n\r\n\r\n// Set up a variable for the scaler parameters\r\nLET vScalerArgs = 'scaler=' & peek('Scaler', 0, 'Scaler') & ',' & \r\n\t\t\t\t\t'missing=' & peek('Null Value Handling', 0, 'Scaler') & ',' & \r\n                    'scale_hashed=' & peek('Scale Hashed Features', 0, 'Scaler') & ',' & \r\n                    peek('Scaler.Hyperparameters', 0, 'Scaler');\r\n\r\n// Set up a variable for execution parameters\r\nLET vExecutionArgs = 'overwrite=true,cv=5,calculate_importances=true, debug=true';\r\n//LET vExecutionArgs = 'overwrite=true,cv=3,calculate_importances=true';\r\n\r\n// Set up a variable for grid search parameters\r\nLET vGridSearch = 'scoring=f1_micro|str';\r\n\r\nLET i = 0;\r\n\r\n// Create a model for each estimator\r\nFor Each vModel in FieldValueList('Model Name')\r\n    \r\n    // Set up a variable for this estimator's parameters\r\n    LET vEstimatorArgs = 'estimator=' & peek('Estimator', $(i), 'Estimators') & ',' & peek('Hyperparameters', $(i), 'Estimators');\r\n    \r\n    // Set up a temporary table for the model parameters\r\n    [MODEL_INIT]:\r\n    LOAD\r\n        [Model Name] as Model_Name,\r\n        '$(vEstimatorArgs)' as EstimatorArgs,\r\n        '$(vScalerArgs)' as ScalerArgs,\r\n        '$(vExecutionArgs)' as ExecutionArgs    \r\n    RESIDENT Estimators\r\n    WHERE [Model Name] = '$(vModel)';\r\n    \r\n    // Use the LOAD...EXTENSION syntax to call the Setup function\r\n    [Result-Setup]:\r\n    LOAD\r\n        model_name,\r\n        result,\r\n        timestamp\r\n    EXTENSION PyTools.sklearn_Setup(MODEL_INIT{Model_Name, EstimatorArgs, ScalerArgs, ExecutionArgs});\r\n    \r\n    [PARAM_GRID]:\r\n    LOAD\r\n    \t[Model Name] as Model_Name,\r\n        [Hyperparameters],\r\n        '$(vGridSearch)' as GridSearchArgs  \r\n    RESIDENT Optimization\r\n    WHERE [Model Name] = '$(vModel)';\r\n    \r\n    // Use the LOAD...EXTENSION syntax to call the Setup function\r\n    [Result-Setup]:\r\n    LOAD\r\n        model_name,\r\n        result,\r\n        timestamp\r\n    EXTENSION PyTools.sklearn_Set_Param_Grid(PARAM_GRID{Model_Name, Hyperparameters, GridSearchArgs});\r\n    \r\n    [FEATURES]:\r\n    LOAD\r\n    \t'$(vModel)' as Model_Name,\r\n        Name,\r\n        Variable_Type,\r\n        Data_Type,\r\n        Feature_Strategy,\r\n        Hash_Features\r\n    RESIDENT [Feature Definitions];\r\n    \r\n    // Use the LOAD...EXTENSION syntax to call the Set_Features function\r\n    [Result-Setup]:\r\n    LOAD\r\n        model_name,\r\n        result,\r\n        timestamp\r\n    EXTENSION PyTools.sklearn_Set_Features(FEATURES{Model_Name, Name, Variable_Type, Data_Type, Feature_Strategy, Hash_Features});\r\n\r\n    Drop table MODEL_INIT, [PARAM_GRID], FEATURES;   \r\n    \r\n    LET i = $(i) + 1;\r\n  \r\nNext vModel;\r\n\r\n// Set up a temporary table for the training and test dataset\r\n\r\n[TEMP_TRAIN_TEST]:\r\nLOAD\r\n    Fecha & '|' & \r\n    \"Tipo Cliente\" & '|' & \r\n    Zona & '|' & \r\n    Cliente & '|' & \r\n    R.Produccion & '|' & \r\n    Fam_desc & '|' & \r\n    Articulo & '|' & \r\n    Agente_Pedido & '|' & \r\n    \"Agente Cliente\" & '|' & \r\n    Ventas AS N_Features\r\nRESIDENT [Train-Test];\r\n\r\n\r\n\r\n\r\nLET i = 0;\r\n\r\n// Train and Test each model\r\nFor Each vModel in FieldValueList('Model Name')\r\n\r\n    [TEMP_SAMPLES]:\r\n    LOAD\r\n    \t'$(vModel)' as Model_Name,\r\n        N_Features\r\n    RESIDENT [TEMP_TRAIN_TEST];\r\n    \r\n    // Use the LOAD...EXTENSION syntax to call the Fit function\r\n    [Result-Fit]:\r\n    LOAD\r\n        model_name,\r\n        result as fit_result, \r\n        time_stamp as fit_timestamp, \r\n        score_result, \r\n        score\r\n    EXTENSION PyTools.sklearn_Fit(TEMP_SAMPLES{Model_Name, N_Features});\r\n    \r\n    // Use the LOAD...EXTENSION syntax to call the Get_Metrics function\r\n    [Result-Metrics]:\r\n    LOAD\r\n        model_name,\r\n        class,\r\n        accuracy,\r\n        precision,\r\n        precision_std,\r\n        recall,\r\n        recall_std,\r\n        fscore,\r\n        fscore_std\r\n    EXTENSION PyTools.sklearn_Get_Metrics(TEMP_SAMPLES{Model_Name});\r\n    \r\n    // Use the LOAD...EXTENSION syntax to call the Get_Confusion_Matrix function\r\n    [Result-ConfusionMatrix]:\r\n    LOAD\r\n        model_name,\r\n        true_label,\r\n        pred_label,\r\n        count\r\n    EXTENSION PyTools.sklearn_Get_Confusion_Matrix(TEMP_SAMPLES{Model_Name});\r\n    \r\n    // Use the LOAD...EXTENSION syntax to call the Get_Best_Params function\r\n    [Best-Params]:\r\n    LOAD\r\n    \tmodel_name as [Model Name],\r\n        best_params\r\n    EXTENSION PyTools.sklearn_Get_Best_Params(TEMP_SAMPLES{Model_Name});\r\n    \r\n    \r\n    \r\n    // Use the LOAD...EXTENSION syntax to call the Explain_Importances function\r\n    [Result-Importances]:\r\n    LOAD\r\n        model_name,\r\n        feature_name,\r\n        importance\r\n    EXTENSION PyTools.sklearn_Explain_Importances(TEMP_SAMPLES{Model_Name});\r\n    \r\n    Drop table TEMP_SAMPLES;   \r\n    \r\n    LET i = $(i) + 1;\r\n  \r\nNext vModel\r\n\r\nDrop table TEMP_TRAIN_TEST; `"}}
{"actor":{"display_login":"SkylakeXx"}}