Commit
·
2f9ec5a
1
Parent(s):
a75980b
Update README.md
Browse files
README.md
CHANGED
|
@@ -3,10 +3,14 @@ license: apache-2.0
|
|
| 3 |
---
|
| 4 |
|
| 5 |
# OFA-base
|
|
|
|
|
|
|
| 6 |
This is the **base** version of OFA pretrained model. OFA is a unified multimodal pretrained model that unifies modalities (i.e., cross-modality, vision, language) and tasks (e.g., image generation, visual grounding, image captioning, image classification, text generation, etc.) to a simple sequence-to-sequence learning framework.
|
| 7 |
|
| 8 |
The directory includes 4 files, namely `config.json` which consists of model configuration, `vocab.json` and `merge.txt` for our OFA tokenizer, and lastly `pytorch_model.bin` which consists of model weights. There is no need to worry about the mismatch between Fairseq and transformers, since we have addressed the issue yet.
|
| 9 |
|
|
|
|
|
|
|
| 10 |
To use it in transformers, please refer to https://github.com/OFA-Sys/OFA/tree/feature/add_transformers. Install the transformers and download the models as shown below.
|
| 11 |
```
|
| 12 |
git clone --single-branch --branch feature/add_transformers https://github.com/OFA-Sys/OFA.git
|
|
|
|
| 3 |
---
|
| 4 |
|
| 5 |
# OFA-base
|
| 6 |
+
|
| 7 |
+
## Introduction
|
| 8 |
This is the **base** version of OFA pretrained model. OFA is a unified multimodal pretrained model that unifies modalities (i.e., cross-modality, vision, language) and tasks (e.g., image generation, visual grounding, image captioning, image classification, text generation, etc.) to a simple sequence-to-sequence learning framework.
|
| 9 |
|
| 10 |
The directory includes 4 files, namely `config.json` which consists of model configuration, `vocab.json` and `merge.txt` for our OFA tokenizer, and lastly `pytorch_model.bin` which consists of model weights. There is no need to worry about the mismatch between Fairseq and transformers, since we have addressed the issue yet.
|
| 11 |
|
| 12 |
+
|
| 13 |
+
## How to use
|
| 14 |
To use it in transformers, please refer to https://github.com/OFA-Sys/OFA/tree/feature/add_transformers. Install the transformers and download the models as shown below.
|
| 15 |
```
|
| 16 |
git clone --single-branch --branch feature/add_transformers https://github.com/OFA-Sys/OFA.git
|