How to train BERT model with next sentence prediction?
Hello.

I'm trying to pretrain the BERT model with next sentence prediction. Is there any function that supports next sentence prediction 
like ` TextDatasetForNextSentencePrediction` of `huggingface/transformers` ?

Hi !
We're not using `TextDatasetForNextSentencePrediction` in `datasets`.
Although you can probably use the `TextDatasetForNextSentencePrediction.create_examples_from_document` on a dataset to prepare it for next sentence prediction. Thanks.

Do you mean that `TextDatasetForNextSentencePrediction.create_exapmles_from_document` can be applied to dataset object other than `TextDatasetForNextSentencePrediction`  e.g. a `Dataset` object which is  loaded by  `datasets.load_dataset`? It would probably require a bit of tweaking, but you can apply it to a dataset, yes.
This should give you a new dataset with sentence pairs you can train a model on.

You can find the documentation about dataset processing here:
https://huggingface.co/docs/datasets/processing.html#processing-data-with-map Thank you for detail information.

I'll try to apply `create_examples_from_document` to `Dataset` object.
