license: apache-2.0
datasets:
- Allanatrix/Scientific_Research_Tokenized
language:
- en
base_model:
- allenai/OLMo-7B
pipeline_tag: text-generation
library_name: peft
tags:
- Olmo
- lora
- peft
- transformers
- scientific-ml
- fine-tuned
- research-assistant
- hypothesis-generation
- scientific-writing
- scientific-reasoning
Model Card for nexa-OLMo-sci7b
Model Details
Model Description:
nexa-OLMo-sci7b is a fine-tuned variant of allenai/OLMo-7B, optimized for scientific research generation tasks such as hypothesis generation, abstract writing, and methodology completion. Fine-tuning was performed using PEFT with LoRA in 4-bit quantized mode via bitsandbytes.
Developed by: Allan (Independent Scientific Intelligence Architect)
Shared by: Allan (https://huggingface.co/allan-wandia)
Model type: Decoder-only transformer (causal language model)
Language(s): English (scientific domain-specific vocabulary)
License: Apache 2.0
Fine-tuned from: allenai/OLMo-7B
Repository: https://huggingface.co/allan-wandia/nexa-olmo-sci7b
Training Details
Training Data:
- Size: 100 million tokens
- Source: Curated scientific literature (Bio, Physics, QST, Astro)
Hyperparameters:
- Sequence length: 1024
- Batch size: 1
- Gradient Accumulation Steps: 64
- Effective Batch Size: 64
- Learning rate: 2e-05
- Epochs: 2
- LoRA: Enabled (PEFT)
- Quantization: 4-bit
Results:
Robust performance in scientific prose tasks, with novelty varying by prompt diversity.