---
tags:
- translation
- text2text-generation
model-index:
- name: flan-t5-base-opus-en-id-id-en
  results: []
license: apache-2.0
language:
- en
- id
- multilingual
metrics:
- sacrebleu
widget:
- text: "translate Indonesia to English: Hai, Bagaimana kabarmu?"
  example_title: "tl_id2en_v1"
- text: "translate to English: Hai, Bagaimana kabarmu?"
  example_title: "tl_id2en_v2"
- text: "hey apa yang kamu lakukan terhadapnya ? in English"
  example_title: "tl_id2en_v3"
- text: "translate English to Indonesia: Hello, How are you today?"
  example_title: "tl_en2id_v1"
- text: "translate to Indonesia: Hello, How are you today?"
  example_title: "tl_en2id_v2"
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# flan-t5-base-opus-en-id-id-en

This model consist to be Translator in multimodal Indonesia and English only.

<!---This model was trained from scratch on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 1.3685
- Score: 35.0259
- Counts: [4617, 2627, 1550, 883]
- Totals: [7288, 6288, 5297, 4382]
- Precisions: [63.350713501646545, 41.777989821882954, 29.261846328110252, 20.150616157005935]
- Bp: 0.991
- Sys Len: 7288
- Ref Len: 7354
- Gen Len: 10.556
 Learning Rate: 0.0004-->

# Model Details

## Model Description


- **Model type:** Language model
- **Language(s) (NLP):** English, Indonesian
- **License:** Apache 2.0

# Usage

## Using the Pytorch model

### Running the model on a CPU

<details>
<summary> Click to expand </summary>

```python
from transformers import T5Tokenizer, T5ForConditionalGeneration
tokenizer = T5Tokenizer.from_pretrained("muvazana/flan-t5-base-opus-en-id-id-en")
model = T5ForConditionalGeneration.from_pretrained("muvazana/flan-t5-base-opus-en-id-id-en")
input_text = "translate English to Indonesia: How old are you?"
input_ids = tokenizer(input_text, return_tensors="pt").input_ids
outputs = model.generate(input_ids)
print(tokenizer.decode(outputs[0]))
```

</details>

### Running the model on a GPU

<details>
<summary> Click to expand </summary>

```python
# pip install accelerate
from transformers import T5Tokenizer, T5ForConditionalGeneration
tokenizer = T5Tokenizer.from_pretrained("muvazana/flan-t5-base-opus-en-id-id-en")
model = T5ForConditionalGeneration.from_pretrained("muvazana/flan-t5-base-opus-en-id-id-en", device_map="auto")
input_text = "translate English to Indonesia: How old are you?"
input_ids = tokenizer(input_text, return_tensors="pt").input_ids.to("cuda")
outputs = model.generate(input_ids)
print(tokenizer.decode(outputs[0]))
```

</details>

### Running the model on a GPU using different precisions

#### FP16

<details>
<summary> Click to expand </summary>

```python
# pip install accelerate
import torch
from transformers import T5Tokenizer, T5ForConditionalGeneration
tokenizer = T5Tokenizer.from_pretrained("muvazana/flan-t5-base-opus-en-id-id-ene")
model = T5ForConditionalGeneration.from_pretrained("muvazana/flan-t5-base-opus-en-id-id-en", device_map="auto", torch_dtype=torch.float16)
input_text = "translate English to Indonesia: How old are you?"
input_ids = tokenizer(input_text, return_tensors="pt").input_ids.to("cuda")
outputs = model.generate(input_ids)
print(tokenizer.decode(outputs[0]))
```

</details>

#### INT8

<details>
<summary> Click to expand </summary>

```python
# pip install bitsandbytes accelerate
from transformers import T5Tokenizer, T5ForConditionalGeneration
tokenizer = T5Tokenizer.from_pretrained("muvazana/flan-t5-base-opus-en-id-id-en")
model = T5ForConditionalGeneration.from_pretrained("muvazana/flan-t5-base-opus-en-id-id-en", device_map="auto", load_in_8bit=True)
input_text = "translate English to Indonesia: How old are you?"
input_ids = tokenizer(input_text, return_tensors="pt").input_ids.to("cuda")
outputs = model.generate(input_ids)
print(tokenizer.decode(outputs[0]))
```

</details>

<!-- - **Related Models:** [All FLAN-T5 Checkpoints](https://huggingface.co/models?search=flan-t5)
- **Original Checkpoints:** [All Original FLAN-T5 Checkpoints](https://github.com/google-research/t5x/blob/main/docs/models.md#flan-t5-checkpoints)
- **Resources for more information:**
  - [Research paper](https://arxiv.org/pdf/2210.11416.pdf)
  - [GitHub Repo](https://github.com/google-research/t5x)
  - [Hugging Face FLAN-T5 Docs (Similar to T5) ](https://huggingface.co/docs/transformers/model_doc/t5)

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.005
- train_batch_size: 64
- eval_batch_size: 64
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 128
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 3-->

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Score   | Counts                  | Totals                   | Precisions                                                                       | Bp     | Sys Len | Ref Len | Gen Len |
|:-------------:|:-----:|:-----:|:---------------:|:-------:|:-----------------------:|:------------------------:|:--------------------------------------------------------------------------------:|:------:|:-------:|:-------:|:-------:|
| 1.6959        | 0.55  | 4000  | 1.5776          | 30.6542 | [4414, 2368, 1345, 733] | [7417, 6417, 5426, 4519] | [59.511932047997846, 36.9019791179679, 24.78805750092149, 16.220402743969906]    | 1.0    | 7417    | 7354    | 10.77   |
| 1.4378        | 1.11  | 8000  | 1.4527          | 32.3772 | [4526, 2538, 1483, 834] | [7567, 6567, 5576, 4666] | [59.81234306858729, 38.647784376427595, 26.596126255380202, 17.873981997428203]  | 1.0    | 7567    | 7354    | 10.885  |
| 1.3904        | 1.66  | 12000 | 1.3961          | 33.8978 | [4558, 2559, 1494, 836] | [7286, 6286, 5295, 4383] | [62.55833104584134, 40.70951320394528, 28.21529745042493, 19.073693817020306]    | 0.9907 | 7286    | 7354    | 10.569  | 
| 1.3035        | 2.21  | 16000 | 1.3758          | 34.9471 | [4609, 2628, 1546, 880] | [7297, 6297, 5306, 4392] | [63.16294367548308, 41.73415912339209, 29.136826234451565, 20.036429872495447]   | 0.9922 | 7297    | 7354    | 10.591  |
| 1.2994        | 2.77  | 20000 | 1.3685          | 35.0259 | [4617, 2627, 1550, 883] | [7288, 6288, 5297, 4382] | [63.350713501646545, 41.777989821882954, 29.261846328110252, 20.150616157005935] | 0.991  | 7288    | 7354    | 10.556  |


### Framework versions

- Transformers 4.29.2
- Pytorch 2.0.1
- Datasets 2.13.1
- Tokenizers 0.13.3