support user defined prompters, pretokenized datasets in config, local parquet, local arrow files (#348)

d2e7f27

unverified

winglian commited on Aug 20, 2023