Caching doesn't work for map (non-deterministic)
Thanks for reporting !

To store the cache file, we compute a hash of the function given in `.map`, using our own hashing function.
The hash doesn't seem to stay the same over sessions for the tokenizer.
Apparently this is because of the regex at `tokenizer.pat` is not well supported by our hashing function.

I'm working on a fix Thanks everyone. Works great now. Hi. I believe the fix was for the nlp library. Is there a solution to handle compiled regex expressions in .map() with the caching. I want to run a simple regex pattern on a big dataset, but I am running into the issue of compiled expression not being cached. 

Instead of opening a new issue, I thought I would put my query here. Let me know if a new issue would be more suitable. Thanks Hi @MaveriQ! This fix is also included in the `datasets` library. Can you provide a reproducer?
The caching functionality doesn't work reliably when tokenizing a dataset. Here's a small example to reproduce it. 

```python
import nlp
import transformers

def main():
    ds = nlp.load_dataset("reddit", split="train[:500]")

    tokenizer = transformers.AutoTokenizer.from_pretrained("gpt2")

    def convert_to_features(example_batch):
        input_str = example_batch["body"]
        encodings = tokenizer(input_str, add_special_tokens=True, truncation=True)
        return encodings

    ds = ds.map(convert_to_features, batched=True)

if __name__ == "__main__":
    main()
```

Roughly 3/10 times, this example recomputes the tokenization.

Is this expected behaviour?