README.md

---
language: 
  - "lb"
license: "mit"
tags:
- "luxembourgish"
- "lëtzebuergesch"
- "text generation"
model-index:
- name: "LuxGPT2"
  results:
  - task:
      type: "text-generation"            # Required. Example: automatic-speech-recognition
      name: "Text Generation"             # Optional. Example: Speech Recognition
    dataset:
      type: "LuxembourgishTestDataset"          # Required. Example: common_voice. Use dataset id from https://hf.co/datasets
      name: "Luxembourgish Test Dataset"          # Required. A pretty name for the dataset. Example: Common Voice (French)
    metrics:
      - type: "accuracy"        # Required. Example: wer. Use metric id from https://hf.co/metrics
        value: "0.33"      # Required. Example: 20.90
- name: "LuxGPT2"
  results:
  - task:
      type: "text-generation"            # Required. Example: automatic-speech-recognition
      name: "Text Generation"             # Optional. Example: Speech Recognition
    dataset:
      type: "LuxembourgishTestDataset"          # Required. Example: common_voice. Use dataset id from https://hf.co/datasets
      name: "Luxembourgish Test Dataset"          # Required. A pretty name for the dataset. Example: Common Voice (French)
    metrics:
      - type: "perplexity"        # Required. Example: wer. Use metric id from https://hf.co/metrics
        value: "46.69"      # Required. Example: 20.90
---

GPT-2 model for Text Generation in luxembourgish language, trained on 636.8 MB of text data, consisting of RTL.lu news articles, comments, parlament speeches, the luxembourgish Wikipedia, Newscrawl, Webcrawl and subtitles.
The training took place on a 32 GB Nvidia Tesla V100
with an initial learning rate of 5e-5
with Batch size 4
for 109 hours
for 30 epochs


See the GPT2 model card for considerations on limitations and bias. See the GPT2 documentation for details on GPT2.