About this model: Topical Change Detection in Documents

This network has been fine-tuned for the task described in the paper Topical Change Detection in Documents via Embeddings of Long Sequences and is our best-performing base-transformer model. You can find more detailed information in our GitHub page for the paper here, or read the paper itself. The weights are based on RoBERTa-base.

Load the model

from transformers import AutoModel, AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained('dennlinger/roberta-cls-consec')
model = AutoModel.from_pretrained('dennlinger/roberta-cls-consec')

Training objective

The training task is to determine whether two text segments (paragraphs) belong to the same topical section or not. This can be utilized to create a topical segmentation of a document by consecutively predicting the "togetherness" of two models.

Performance

The results of this model can be found in the paper. We average over models from five different random seeds, which is why the specific results for this model might be different from the exact values in the paper.

Note that this model is not trained to work on classifying single texts, but only works with two (separated) inputs.