{"action":"closed","issue":{"url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/10838","repository_url":"https://api.github.com/repos/scikit-learn/scikit-learn","labels_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/10838/labels{/name}","comments_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/10838/comments","events_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/10838/events","html_url":"https://github.com/scikit-learn/scikit-learn/issues/10838","id":306889567,"number":10838,"title":"Accessing global variables from custom transformers in sklearn pipelines","user":{"login":"belalghanem","id":34351199,"avatar_url":"https://avatars1.githubusercontent.com/u/34351199?v=4","gravatar_id":"","url":"https://api.github.com/users/belalghanem","html_url":"https://github.com/belalghanem","followers_url":"https://api.github.com/users/belalghanem/followers","following_url":"https://api.github.com/users/belalghanem/following{/other_user}","gists_url":"https://api.github.com/users/belalghanem/gists{/gist_id}","starred_url":"https://api.github.com/users/belalghanem/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/belalghanem/subscriptions","organizations_url":"https://api.github.com/users/belalghanem/orgs","repos_url":"https://api.github.com/users/belalghanem/repos","events_url":"https://api.github.com/users/belalghanem/events{/privacy}","received_events_url":"https://api.github.com/users/belalghanem/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2018-03-20T14:28:08Z","updated_at":"2018-03-20T21:34:39Z","closed_at":"2018-03-20T21:34:39Z","author_association":"NONE","body":"In my sklearn classification model, I built a custom transformer which process the input text before the TfidfVectorization process:\r\n\r\n```\r\n# variables --\r\nco = dict() #<-- filled previously\r\n\r\nclass prepTxtStar(BaseEstimator, TransformerMixin):\r\n    def __init__(self, kValue=0):\r\n        self.kValue = kValue\r\n\r\n    def fit(self, X, y=None):\r\n        return self\r\n\r\n    def transform(self, X):\r\n        global co\r\n        print('Size of co from inside:', len(co))\r\n        if len(X[0]) > 3:\r\n            for row in range(len(X)):\r\n                c = str(X[row]).split()\r\n                for i in range(len(c)):\r\n                    if int(co[c[i]]) < int(self.kValue): #<---\r\n                        c[i] = repAllTxt(c[i])\r\n                X[row] = ' '.join(c)\r\n        return X\r\n\r\n\r\n\r\nif __name__ == \"__main__\":\r\n    xOri, yOri = readFiles()\r\n    xOri = [clean_str(i) for i in xOri]\r\n    corpDict(xOri) #<--- co is filled\r\n    print('Size of co before: ', len(co))\r\n\r\n    model = Pipeline(   ('preparingtxt', prepTxtStar(kValue=0)),\r\n                        ('tfidf', TfidfVectorizer(analyzer='char_wb' , lowercase=False)),\r\n                        ('clf', SGDClassifier(tol = None))\r\n                        ])\r\n```\r\n\r\n\r\nThe problem is: when I fit my model without using GridSearch technique, like here:\r\n`model = model.fit(x, y)`\r\n\r\nit works perfectly, but once I used GS technique, the size of \"co\" dictionary that I called in the beginning of my transformer : global co, become 0! as it is empty !?\r\n\r\nthe output with error:\r\n```\r\n> Size of co before:  4361 \r\n> Size of co from inside: 0 \r\n>          ....\r\n>      60         if len(X[0]) > 3:\r\n>      61             for row in range(len(X)):\r\n>      62                 c = str(X[row]).split()\r\n>      63                 # print('2: ' , c)\r\n>      64                 for i in range(len(c)):\r\n> ---> 65                     print('HERE: ', int(co[c[i]]))\r\n>         c = ['You', 'essentially', '.', '.', '.']\r\n>         i = 0\r\n>      66                     if int(co[c[i]]) < int(self.kValue):\r\n>      67                         c[i] = repAllTxt(c[i])\r\n>      68                 X[row] = ' '.join(c)\r\n>      69                 # print(X[row])\r\n> \r\n> KeyError: 'You'\r\n```\r\n\r\nDid I missed something?"}}
{"actor":{"display_login":"belalghanem"}}