|
--- |
|
language: |
|
- en |
|
--- |
|
|
|
# FIDO-GPT: Generative AI behind "Fidonet Cybernetic Immortality" Project |
|
|
|
[FIDONet](https://en.wikipedia.org/wiki/FidoNet) is a historic computer network based on nightly mail exchange between servers |
|
via telephone lines, which was popular in 1990-s. In [FIDONet Cybernetic Immortality Project](https://soshnikov.com/art/fidoci) |
|
we are looking to create exhibits that will revive now-almost-dead FIDONet by automatically writing correspondence in |
|
FIDONet style via generative large language models. |
|
|
|
This model is based on [GPT2-large](https://huggingface.co/gpt2-large) model, and was fine-tuned for 2 epochs on archives of |
|
[ExecPC BBS](https://en.wikipedia.org/wiki/ExecPC_BBS), obtained from [here](https://breakintochat.com/collections/messages/fidonet/index.html). |
|
This process took around 9 hours on NVidia A100 compute in Yandex Datasphere service. |
|
|
|
This code can be used for generation: |
|
|
|
```python |
|
from transformers import pipeline, AutoModelForCausalLM,AutoTokenizer |
|
import torch |
|
|
|
model_name = 'estonto/fido-gpt' |
|
|
|
model = AutoModelForCausalLM.from_pretrained(model_name) |
|
tokenizer = AutoTokenizer.from_pretrained(model_name) |
|
pipe = pipeline(model=model,tokenizer=tokenizer,task="text-generation",device="cuda") |
|
result = pipe("<s>Topic: COMPUTING",do_sample=True,max_length=500)[0]['generated_text'].replace('\\n','\n') |
|
``` |
|
|
|
Project idea and model training: [Dmitry Soshnikov](https://soshnikov.com) |
