metadata
license: apache-2.0
language:
- en
library_name: transformers
pipeline_tag: fill-mask
tags:
- climate
- biology
Model Card for nasa-smd-ibm-v0.1
nasa-smd-ibm-v0.1 is a RoBERTa-based, Encoder-only transformer model, domain-adapted for NASA Science Mission Directorate (SMD) applications. It's fine-tuned on scientific journals and articles relevant to NASA SMD, aiming to enhance natural language technologies like information retrieval and intelligent search.
Model Details
- Base Model: RoBERTa
- Tokenizer: Custom
- Parameters: 125M
- Pretraining Strategy: Masked Language Modeling (MLM)
Training Data
- Wikipedia English (Feb 1, 2020)
- AGU Publications
- AMS Publications
- Scientific papers from Astrophysics Data Systems
- PubMed abstracts
- PMC (commercial license subset)
Training Procedure
- Framework: fairseq 0.12.1 with PyTorch 1.9.1
- transformers Version: 4.2.0
- Strategy: Masked Language Modeling (MLM)
Evaluation
- BLURB Benchmark
- Pruned SQuAD2.0 (SQ2) Benchmark
- NASA SMD Experts Benchmark (WIP)
Uses
- Named Entity Recognition (NER)
- Information Retrieval
- Sentence Transformers
Citation
If you find this work useful, please cite using the following bibtex citation:
@misc {nasa-impact_2023,
author = {Masayasu Maraoka and Bishwaranjan Bhattacharjee and Muthukumaran Ramasubramanian and Ikhsa Gurung and Rahul Ramachandran and Manil Maskey and Kaylin Bugbee and Rong Zhang and Yousuf El Kurdis and Mike Little and Elizabeth Fancher and Lauren Sanders and Sylvain Costes and Sergi Blanco-Cuaresma and Kelly Lockhart and Thomas Allen and Felix Grazes and Megan Ansdell and Alberto Accomazzi and Sanaz Vahidinia and Ryan McGranaghan and Armin Mehrabian and Tsendgar Lee},
title = { nasa-smd-ibm-v0.1 (Revision f01d42f) },
year = 2023,
url = { https://huggingface.co/nasa-impact/nasa-smd-ibm-v0.1 },
doi = { 10.57967/hf/1429 },
publisher = { Hugging Face }
}
Attribution
IBM Research
- Masayasu Maraoka
- Bishwaranjan Bhattacharjee
- Rong Zhang
- Yousuf El Kurdis
NASA SMD
- Muthukumaran Ramasubramanian
- Iksha Gurung
- Rahul Ramachandran
- Manil Maskey
- Kaylin Bugbee
- Mike Little
- Elizabeth Fancher
- Lauren Sanders
- Sylvain Costes
- Sergi Blanco-Cuaresma
- Kelly Lockhart
- Thomas Allen
- Felix Grazes
- Megan Ansdell
- Alberto Accomazzi
- Sanaz Vahidinia
- Ryan McGranaghan
- Armin Mehrabian
- Tsendgar Lee