| # About this model: Topical Change Detection in Documents | |
| This network has been fine-tuned for the task described in the paper *Topical Change Detection in Documents via Embeddings of Long Sequences* and is our best-performing base-transformer model. You can find more detailed information in our GitHub page for the paper [here](https://github.com/dennlinger/TopicalChange), or read the [paper itself](https://arxiv.org/abs/2012.03619). The weights are based on RoBERTa-base. | |
| # Load the model | |
| ```python | |
| from transformers import AutoModel, AutoTokenizer | |
| tokenizer = AutoTokenizer.from_pretrained('dennlinger/roberta-cls-consec') | |
| model = AutoModel.from_pretrained('dennlinger/roberta-cls-consec') | |
| ``` | |
| # Training objective | |
| The training task is to determine whether two text segments (paragraphs) belong to the same topical section or not. This can be utilized to create a topical segmentation of a document by consecutively predicting the "togetherness" of two models. | |
| # Performance | |
| The results of this model can be found in the paper. We average over models from five different random seeds, which is why the specific results for this model might be different from the exact values in the paper. | |
| Note that this model is *not* trained to work on classifying single texts, but only works with two (separated) inputs. |