This is an unofficial reupload of microsoft/codebert-base in the SafeTensors
format using transformers
4.40.1
. The goal of this reupload is to prevent older models that are still relevant baselines from becoming stale as a result of changes in HuggingFace. Additionally, I may include minor corrections, such as model max length configuration.
Original model card below:
CodeBERT-base
Pretrained weights for CodeBERT: A Pre-Trained Model for Programming and Natural Languages.
Training Data
The model is trained on bi-modal data (documents & code) of CodeSearchNet
Training Objective
This model is initialized with Roberta-base and trained with MLM+RTD objective (cf. the paper).
Usage
Please see the official repository for scripts that support "code search" and "code-to-document generation".
Reference
- CodeBERT trained with Masked LM objective (suitable for code completion)
- 🤗 Hugging Face's CodeBERTa (small size, 6 layers)
Citation
@misc{feng2020codebert,
title={CodeBERT: A Pre-Trained Model for Programming and Natural Languages},
author={Zhangyin Feng and Daya Guo and Duyu Tang and Nan Duan and Xiaocheng Feng and Ming Gong and Linjun Shou and Bing Qin and Ting Liu and Daxin Jiang and Ming Zhou},
year={2020},
eprint={2002.08155},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
- Downloads last month
- 29
Inference Providers
NEW
This model is not currently available via any of the supported Inference Providers.