|
--- |
|
language: en |
|
tags: |
|
- arxiv |
|
- research-papers |
|
- text-generation |
|
license: apache-2.0 |
|
--- |
|
|
|
# KnullAI v2 - Fine-tuned on ArXiver Dataset |
|
|
|
This model is a fine-tuned version of KnullAI v2, specifically trained on the ArXiver dataset containing research paper information. |
|
|
|
## Training Data |
|
The model was fine-tuned on the neuralwork/arxiver dataset, which contains: |
|
- Paper titles |
|
- Abstracts |
|
- Authors |
|
- Publication dates |
|
- Links |
|
|
|
## Model Details |
|
- Base model: Rawkney/knullAi_v2 |
|
- Training type: Causal language modeling |
|
- Hardware: T4 GPU |
|
- Mixed precision: FP16 |
|
|
|
## Usage |
|
```python |
|
from transformers import AutoModelForCausalLM, AutoTokenizer |
|
|
|
# Load model and tokenizer |
|
model = AutoModelForCausalLM.from_pretrained("YOUR_REPO_ID") |
|
tokenizer = AutoTokenizer.from_pretrained("YOUR_REPO_ID") |
|
|
|
# Example usage |
|
title = "Your paper title" |
|
input_text = f"Title: {title}\nAbstract:" |
|
inputs = tokenizer(input_text, return_tensors="pt").to("cuda") |
|
|
|
outputs = model.generate( |
|
inputs["input_ids"], |
|
max_length=256, |
|
temperature=0.7, |
|
top_p=0.9, |
|
pad_token_id=tokenizer.eos_token_id |
|
) |
|
|
|
response = tokenizer.decode(outputs[0], skip_special_tokens=True) |
|
print(response) |
|
``` |
|
|
|
## Training Parameters |
|
- Learning rate: 1e-5 |
|
- Epochs: 1 |
|
- Batch size: 1 |
|
- Gradient accumulation steps: 16 |
|
- Mixed precision training (fp16) |
|
- Max sequence length: 512 |
|
|
