BERTShared-meter2poem-arV01 / added_tokens.json
mareloraby's picture
add tokenizer
d342f99
raw
history blame contribute delete
95 Bytes
{
"[BOS]": 30003,
"[BSEP]": 30000,
"[EOS]": 30004,
"[FSEP]": 30001,
"[SOB]": 30002
}