{"action":"created","issue":{"url":"https://api.github.com/repos/jpmml/jpmml-sklearn/issues/81","repository_url":"https://api.github.com/repos/jpmml/jpmml-sklearn","labels_url":"https://api.github.com/repos/jpmml/jpmml-sklearn/issues/81/labels{/name}","comments_url":"https://api.github.com/repos/jpmml/jpmml-sklearn/issues/81/comments","events_url":"https://api.github.com/repos/jpmml/jpmml-sklearn/issues/81/events","html_url":"https://github.com/jpmml/jpmml-sklearn/issues/81","id":361278915,"node_id":"MDU6SXNzdWUzNjEyNzg5MTU=","number":81,"title":"Feature Request: Regular Expression Transformers","user":{"login":"tyers","id":43376215,"node_id":"MDQ6VXNlcjQzMzc2MjE1","avatar_url":"https://avatars3.githubusercontent.com/u/43376215?v=4","gravatar_id":"","url":"https://api.github.com/users/tyers","html_url":"https://github.com/tyers","followers_url":"https://api.github.com/users/tyers/followers","following_url":"https://api.github.com/users/tyers/following{/other_user}","gists_url":"https://api.github.com/users/tyers/gists{/gist_id}","starred_url":"https://api.github.com/users/tyers/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tyers/subscriptions","organizations_url":"https://api.github.com/users/tyers/orgs","repos_url":"https://api.github.com/users/tyers/repos","events_url":"https://api.github.com/users/tyers/events{/privacy}","received_events_url":"https://api.github.com/users/tyers/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":8,"created_at":"2018-09-18T12:26:04Z","updated_at":"2018-10-01T10:07:02Z","closed_at":null,"author_association":"NONE","body":"Hi Villu,\r\nI'm looking for a way to apply the two PCRE functions available within the PMML specification within a PMML pipeline / dataframe mapper.\r\nI've not seen anything in the docs here that suggests this is currently possible outside of implementing a custom transformer through the sklearn2pmml-plugin framework, so happy to be corrected on this.\r\nMany thanks\r\nChris\r\n\r\nRef docs.\r\nhttp://dmg.org/pmml/v4-3/BuiltinFunctions.html#replace\r\nhttp://dmg.org/pmml/v4-3/BuiltinFunctions.html#matches"},"comment":{"url":"https://api.github.com/repos/jpmml/jpmml-sklearn/issues/comments/425854928","html_url":"https://github.com/jpmml/jpmml-sklearn/issues/81#issuecomment-425854928","issue_url":"https://api.github.com/repos/jpmml/jpmml-sklearn/issues/81","id":425854928,"node_id":"MDEyOklzc3VlQ29tbWVudDQyNTg1NDkyOA==","user":{"login":"tyers","id":43376215,"node_id":"MDQ6VXNlcjQzMzc2MjE1","avatar_url":"https://avatars3.githubusercontent.com/u/43376215?v=4","gravatar_id":"","url":"https://api.github.com/users/tyers","html_url":"https://github.com/tyers","followers_url":"https://api.github.com/users/tyers/followers","following_url":"https://api.github.com/users/tyers/following{/other_user}","gists_url":"https://api.github.com/users/tyers/gists{/gist_id}","starred_url":"https://api.github.com/users/tyers/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tyers/subscriptions","organizations_url":"https://api.github.com/users/tyers/orgs","repos_url":"https://api.github.com/users/tyers/repos","events_url":"https://api.github.com/users/tyers/events{/privacy}","received_events_url":"https://api.github.com/users/tyers/received_events","type":"User","site_admin":false},"created_at":"2018-10-01T10:07:02Z","updated_at":"2018-10-01T10:07:02Z","author_association":"NONE","body":"Hi Villu,\r\nI've updated again and removed the Pandas requirement. Regular expressions now use PCRE or fallback to re. The string manipulations are now also just using base python operations. \r\n```python\r\nfrom sklearn2pmml.util import eval_rows\r\nfrom sklearn.base import TransformerMixin\r\nimport warnings\r\n\r\ntry:\r\n    import pcre\r\n    PCRE_AVAILABLE = True\r\nexcept ImportError:\r\n    import re\r\n    PCRE_AVAILABLE = False\r\n\r\n\r\nclass ReMatchTransformer(TransformerMixin):\r\n\r\n  def __init__(self, pattern = None):\r\n    if PCRE_AVAILABLE == False:\r\n      warnings.warn(\"PCRE engine not available, results might not be reproducible between Python and PMML when using more complex patterns\", Warning)\r\n\r\n    if pattern is  None:\r\n      raise ValueError(\"Pattern is None\")\r\n    self.pattern = pattern\r\n\r\n  def fit(self, x, y = None):\r\n    return self\r\n\r\n  def transform(self, x, y = None):\r\n    if PCRE_AVAILABLE:\r\n      func = lambda m: bool(pcre.search(self.pattern,m))\r\n    else:\r\n      func = lambda m: bool(re.search(self.pattern,m))\r\n    y = eval_rows(x, func)\r\n    return y\r\n\r\n\r\nclass ReReplaceTransformer(TransformerMixin):\r\n\r\n  def __init__(self, pattern = None, replace = None):\r\n    if PCRE_AVAILABLE == False:\r\n      warnings.warn(\"PCRE engine not available, results might not be reproducible between Python and PMML when using more complex patterns\", Warning)\r\n    if pattern is  None:\r\n      raise ValueError(\"Pattern is None\")\r\n    if replace is  None:\r\n      raise ValueError(\"Replace is None\")\r\n    self.pattern = pattern\r\n    self.replace = replace\r\n\r\n  def fit(self, x):\r\n    return self\r\n\r\n  def transform(self, x):\r\n    if PCRE_AVAILABLE:\r\n      func = lambda m: pcre.sub(self.pattern,self.replace,m)\r\n    else:\r\n      func = lambda m: re.sub(self.pattern,self.replace,m)\r\n    y = eval_rows(x, func)\r\n    return y\r\n\r\n\r\nclass ReContainsTransformer(TransformerMixin):\r\n\r\n  def fit(self, x):\r\n    return self\r\n\r\n  def transform(self, x):\r\n    if PCRE_AVAILABLE:\r\n      func = lambda m: bool(pcre.search(m[1],m[0]))\r\n    else:\r\n      func = lambda m: bool(re.search(m[1],m[0]))\r\n    y = eval_rows(x, func)\r\n    return y.reshape(-1, 1)\r\n\r\n\r\nclass ConcatTransformer(TransformerMixin):\r\n  def __init__(self, separator = None):\r\n    if separator == None:\r\n      self.separator = ''\r\n    else:\r\n      self.separator = separator\r\n\r\n  def fit(self, x):\r\n    return self\r\n\r\n  def transform(self, x):\r\n    func = lambda X: self.separator.join(X)\r\n    y = eval_rows(x, func)\r\n    return y.reshape(-1, 1)\r\n\r\n\r\nclass SubStrTransformer(TransformerMixin):\r\n  def __init__(self, start = None, length = None):\r\n    self.start = start - 1  # convert to 0 index\r\n    self.stop = start - 1 + length # convert from length to stop index\r\n\r\n  def fit(self, x):\r\n    return self\r\n\r\n  def transform(self, x):\r\n    func = lambda X: X[self.start:self.stop]\r\n    y = eval_rows(x, func)\r\n    return y.reshape(-1, 1)\r\n\r\n```"}}
{"actor":{"display_login":"tyers"}}