|
|
--- |
|
|
license: apache-2.0 |
|
|
datasets: |
|
|
- Allanatrix/Scientific_Research_Tokenized |
|
|
language: |
|
|
- en |
|
|
base_model: |
|
|
- allenai/OLMo-7B |
|
|
pipeline_tag: text-generation |
|
|
library_name: peft |
|
|
tags: |
|
|
- Olmo |
|
|
- lora |
|
|
- peft |
|
|
- transformers |
|
|
- scientific-ml |
|
|
- fine-tuned |
|
|
- research-assistant |
|
|
- hypothesis-generation |
|
|
- scientific-writing |
|
|
- scientific-reasoning |
|
|
--- |
|
|
|
|
|
# Model Card for nexa-OLMo-sci7b |
|
|
|
|
|
## Model Details |
|
|
**Model Description:** |
|
|
nexa-OLMo-sci7b is a fine-tuned variant of allenai/OLMo-7B, optimized for scientific research generation tasks such as hypothesis generation, abstract writing, and methodology completion. Fine-tuning was performed using PEFT with LoRA in 4-bit quantized mode via bitsandbytes. |
|
|
|
|
|
**Developed by:** Allan (Independent Scientific Intelligence Architect) |
|
|
**Shared by:** Allan (https://huggingface.co/allan-wandia) |
|
|
**Model type:** Decoder-only transformer (causal language model) |
|
|
**Language(s):** English (scientific domain-specific vocabulary) |
|
|
**License:** Apache 2.0 |
|
|
**Fine-tuned from:** allenai/OLMo-7B |
|
|
**Repository:** https://huggingface.co/allan-wandia/nexa-olmo-sci7b |
|
|
|
|
|
## Training Details |
|
|
**Training Data:** |
|
|
- Size: 100 million tokens |
|
|
- Source: Curated scientific literature (Bio, Physics, QST, Astro) |
|
|
|
|
|
**Hyperparameters:** |
|
|
- Sequence length: 1024 |
|
|
- Batch size: 1 |
|
|
- Gradient Accumulation Steps: 64 |
|
|
- Effective Batch Size: 64 |
|
|
- Learning rate: 2e-05 |
|
|
- Epochs: 2 |
|
|
- LoRA: Enabled (PEFT) |
|
|
- Quantization: 4-bit |
|
|
|
|
|
**Results:** |
|
|
Robust performance in scientific prose tasks, with novelty varying by prompt diversity. |
