fix tokenizer save_pretrained method

#32

by katuni4ka - opened May 20, 2024

base: refs/heads/main

←

from: refs/pr/32

Discussion Files changed

+175

-1

katuni4ka

May 20, 2024

tokenizer.save_pretrained methos failed due to imposibility serialize into json some config values (add_special_tokens, that is represented by method in tokenizer and present in init kwargs)

fix tokenizer save_pretrained method927e4b7c

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

Ready to merge

This branch is ready to get merged automatically.

Your need to confirm your account before you can post a new comment.

· Sign up or log in to comment