# AI for Biomedicine in the Era of Large Language Models

*Zhenyu Bi, *Sajib Acharjee Dip, *Daniel Hajialigol, *Sindhura Kommu,
*Hanwen Liu, *Meng Lu, Xuan Wang  zhenyub, sajibacharjeedip, danielhajialigol, sindhura, liuhwen, menglu<EMAIL_ADDRESS>Virginia Tech, Blacksburg, VA, USA

(2024; 2024)

###### Abstract.

The capabilities of AI for biomedicine span a wide spectrum, from the atomic
level, where it solves partial differential equations for quantum systems, to
the molecular level, predicting chemical or protein structures, and further
extending to societal predictions like infectious disease outbreaks. Recent
advancements in large language models, exemplified by models like ChatGPT,
have showcased significant prowess in natural language tasks, such as
translating languages, constructing chatbots, and answering questions. When we
consider biomedical data, we observe a resemblance to natural language in
terms of sequences – biomedical literature and health records presented as
text, biological sequences or sequencing data arranged in sequences, or sensor
data like brain signals as time series. The question arises: Can we harness
the potential of recent large language models to drive biomedical knowledge
discoveries? In this tutorial, we will explore the application of large
language models to three crucial categories of biomedical data: 1) textual
data, 2) biological sequences, and 3) brain signals. Furthermore, we will
delve into large language models’ challenges in biomedical research, including
ensuring trustworthiness, achieving personalization, and adapting to multi-
modal data representation.

*These authors have made equal contributions to this paper. They are listed in alphabetical order by last name.

††copyright: acmcopyright††journalyear: 2024††doi:
XXXXXXX.XXXXXXX††conference: Make sure to enter the correct conference title
from your rights confirmation email; June 03–05, 2018; Woodstock, NY††isbn:
978-1-4503-XXXX-X/18/06††copyright: acmcopyright††journalyear: 2024††doi:
XXXXXXX.XXXXXXX††conference: The 30th ACM SIGKDD Conference on Knowledge
Discovery and Data Mining; August 25–29, 2024; Barcelona, Spain††booktitle:
KDD ’24: The 30th ACM SIGKDD Conference on Knowledge Discovery and Data
Mining, August 25–29, 2024, Barcelona, Spain††price: 15.00††isbn:
978-1-4503-XXXX-X/18/06

##  1\. Introduction

The impressive capabilities of AI for biomedicine span a wide spectrum, from
the atomic level, where it attempts to solve partial differential equations
for quantum systems, to the molecular level, where it accurately predicts the
structures of chemicals and proteins, and extends even further, encompassing
societal predictions like forecasting infectious disease outbreaks
(zhang2023artificial, ). Amidst this landscape of possibilities, recent
advancements in large language models (LLMs), notably exemplified by models
like ChatGPT111https://chat.openai.com/chat, have risen to the forefront,
demonstrating significant proficiency in tasks tied to natural language. These
tasks include language translation, constructing chatbots, and answering
questions (yang2023harnessing, ).

Interestingly, when we turn our attention to biomedical data, we observe a
striking resemblance to natural language in terms of sequences. Biomedical
literature and health records are laid out as textual narratives, biological
sequences or sequencing data takes the form of molecular or expression
sequences, and sensor data like brain signals is inherently sequential time
series (wang2021pre, ; thirunavukarasu2023large, ). This observation prompts a
compelling question: Can we leverage the advanced LLMs to drive biomedical
knowledge discoveries?

In this survey paper, we embark on a journey to explore precisely this
intersection—the fusion of cutting-edge large language models with biomedical
inquiry. Our exploration zooms in on three pivotal categories of biomedical
data: 1) textual data, 2) biological sequences, and 3) brain signals. By
drawing inspiration from the transformative capabilities of LLMs, we seek to
unravel novel understanding and innovation within each domain.

As we move forward, we further discuss the intricate challenges accompanying
AI infusion into biomedical research. The foundation of trustworthiness stands
tall—how do we ensure the reliability of AI-enhanced biomedical insights? The
concept of personalization emerges as a critical consideration, urging us to
tailor LLMs to the specific contours of biomedical investigation. Furthermore,
the multi-modal nature of biomedical data motivates us to handle data
representations that span across various modalities.

##  2\. LLMs on Diverse Biomedical Data

We comprehensively survey the topics of LLMs for biomedicine based on three
pivotal categories of biomedical data: 1) textual data, 2) biological
sequences, and 3) brain signals.

###  2.1. LLMs on Biomedical Textual Data

First, we introduce LLMs in the realm of biomedical textual data, which
encompasses diverse domains like biomedical literature (beltagy2019scibert, ;
lee2020biobert, ; gu2021domain, ; alrowili2021biom, ; yasunaga2022linkbert, )
and electronic health records (alsentzer2019publicly, ; singhal2022large, ).
This form of biomedical textual data closely mirrors the fundamental structure
of large language models. It finds extensive utility across biomedicine and
healthcare, facilitating tasks such as extracting valuable information and
responding to queries. The applicability spans a multitude of areas,
underpinning biomedical and healthcare information extraction
(wang2021chemner, ; zhong2023reactie, ; wang2022reactclass, ) and question-
answering (krithara2023bioasq, ).

Figure 1. Overview of applications, models, and downstream tasks for
biomedical LLMs on textual data.

##### SciBERT

SciBERT (beltagy2019scibert, ) is a BERT-based model (devlin2018bert, ) pre-
trained on scientific publications. Specifically, SciBERT is pre-trained on
1.14 million scientific publications from Semantic Scholar (ammar-
etal-2018-construction, ) via random sampling. SciBERT’s vocabulary
construction utilizes WordPiece (wu2016google, ), the same unsupervised
tokenization technique that BERT employs, while also maintaining the same
vocabulary size. Only 42% of the vocabulary across SciBERT and BERT contain
the same vocabulary, indicating the difference in the frequency of most common
words in the general domain (BERT) and in the scientific domain (SciBERT).
SciBERT surpasses BERT in all four evaluated downstream tasks, which include
named entity recognition, PICO (Population, Intervention, Comparison, Outcome)
extraction, relation extraction, and dependency parsing. Additionally, SciBERT
was state-of-the-art in all four tasks excluding dependency parsing at the
time of publication.

##### ClinicalBERT

ClinicalBERT (alsentzer2019publicly, ) is the first framework to pre-train
clinical texts using a BERT-based architecture. Approximately 2 million
clinical notes, originating from the MIMIC III v1.4 database
(johnson2016mimic, ), are used for pre-training. The discharge summaries were
used for pre-training to generate Discharge BERT, while the clinical note
types (nursing, radiology, etc.) were used to train Clinical BERT. Experiments
were conducted on natural language inference and named entity recognition
tasks. ClinicalBERT demonstrated state-of-the-art results on natural language
inference and two of the four named entity recognition datasets at the time of
publication. A concern of using ClinicalBERT is the lack of robustness for
patient de-identification, as the framework was not able to obtain
improvements on the de-identification task.

##### BioBERT

BioBERT (lee2020biobert, ) is a BERT-based architecture that is pre-trained on
biomedical data. Precisely, BioBERT pre-trains on Pubmed abstracts (4.5
billion words) and PMC publications (13.5 billion words). At the time of
publication, experimental results indicate that BioBERT achieves state-of-the-
art performance on the majority of biomedical named entity recognition,
biomedical question answering, and biomedical relation extraction datasets.

##### BioMegatron

BioMegatron (shin2020biomegatron, ) is biomedical LLM based on Megatron-LM
(shoeybi2019megatron, ), where model parallelism is applied to both self-
attention and multi-layer perceptron blocks for each transformer layer. While
Megatron-LM is pre-trained on general domain text, BioMegatron is pre-trained
on approximately 6.1 billion words, 4.5 billion of which are original from a
PubMed abstract dataset, and the remaining 1.6 billion derived from all PMC
scientific publications. BioMegatron’s size ranges from 345 million to 1.2
billion and also has the flexibility to use vocabularies of either 30,000 or
50,000 subtokens. Their experiments indicate that BioMegatron performs state-
of-the-art on all tested named entity recognition and relation extraction
benchmarks, mainly with the larger vocabulary.

##### SciFive

SciFive (phan2021scifive, ) uses a T5 (raffel2020exploring, ), a generative
text model that is pre-trained millions of PubMed abstraction and PMC
scientific publications. Unlike BERT-inspired architectures, SciFive utilizes
a Sentence Piece (kudo-richardson-2018-sentencepiece, ) algorithm for its
vocabulary. State-of-the-art results are achieved against baselines such as
BioBERT, general-domain T5, and general-domain BERT on biomedical question
answering, relation extraction, named entity recognition, and natural language
inference.

##### PubMedBERT

A common assumption within the biomedical LLM literature is that the transfer
of general domain knowledge will help domain-specific, in particular
biomedical, downstream tasks; however, PubMedBERT (gu2021domain, ) shows this
transfer of knowledge is detrimental to downstream performance: Pre-training
using biomedical corpora results in better performance than continual pre-
training. This performance improvement is in large part due to PubMedBERT
having an in-domain vocabulary, whereas mixed-domain pre-training is subjected
to model out of vocabulary words using fragmented subwords. PubMedBERT uses
BERT as its backbone and is pre-trained on PubMed articles. Across all 14
datasets stemming from relation extraction, named entity recognition, document
classification, question answering, and PICO extraction for evaluation,
PubMedBERT achieves state-of-the-art on 12 datasets.

##### BioLinkBERT

Contrary to the approach of many biomedical LLMs, which typically pre-train
using only single documents, BioLinkBERT (yasunaga2022linkbert, ) utilizes the
multiple links between documents, resulting in a multi-task self-supervised
pre-training, modeling both document relation extraction and masked language
modeling. BioLinkBERT utilizes PubMed, equipped with citations, as its pre-
training corpus. To facilitate the document relation extraction task,
BioLinkBERT creates a document relation extraction dataset of the pre-training
corpus. Each pair of sections (coming from either the same or different
documents) falls under one of three classes: random, contiguous, or linked,
where ”contiguous” indicates the sections (from the same document) are right
next to each other, and ”linked” indicates two sections (from different
documents) are indeed linked. BioLinkBERT achieves state-of-the-art
performance on 12 of the 13 different datasets used for evaluations that span
6 different tasks (named entity recognition, PICO extraction, relation
extraction, sentence similarity, document classification, and question
answering).

##### Galactica

Galactica (taylor2022galactica, ) employs a Transformer-based architecture but
generates a vocabulary of size 50,000 using byte-pair encoding as well as
learnable positional embeddings. The size of Galactica concerning its
parameters can range from 125 million to 120 billion. Galactica has multiple
tokenization techniques dependent on the modality that’s presented to it. This
mainly comes in the form of special tokens wrapping around a modality’s input.
For example, citation information is wrapped around [START_REF] and [END_REF]
tokens; special tokens [START_AMINO] and [END_AMINO] are wrapped around amino
acid sequences. Across all types of modalities, Galactica was pre-trained on
106 billion tokens from scientific sources such as scholarly papers,
textbooks, and encyclopedias. Galactica either achieves state-of-the-art
performance or is within 0.5% on biomedical question-answering datasets that
are evaluated.

##### BioGPT

BioGPT (luo2022biogpt, ) is one of the first biomedical GPT-inspired
biomedical frameworks. BioGPT is pre-trained on 15 million PubMed articles and
employs byte-pair as their vocabulary-building algorithm, resulting in a
vocabulary of approximately 42,000 in size. BioGPT is trained with the GPT-2
(radford2019language, ) backbone due to the sheer size of GPT-3
(brown2020language, ) (15 billion parameters). As a result, BioGPT has 347
million parameters. Experiments on three biomedical relation extraction
datasets and a biomedical question-answer dataset indicate state-of-the-art
performance at the time of publication.

##### DoT5

DoT5 (liu2023compositional, ) proposed a zero-shot domain transfer framework
that transfer domain knowledge to the biomedical domain. DoT5 continuously
pre-trains using a multi-task setup: combining general-domain task knowledge
and biomedical knowledge. Specifically, DoT5’s pre-training objective is a
linear combination of solving domain-agnostic tasks (natural language
inference, summarization, etc.) and masked biomedical language modeling. DoT5
uses T5 (raffel2020exploring, ) as its backbone and utilizes PubMed abstracts
for its in-domain dataset. Results indicate that DoT5 is competitive with
state-of-the-art biomedical natural language inference and biomedical
summarization tasks.

##### GatorTronGPT

Considerable stigma associated with applying LLMs like ChatGPT to biomedicine
stems from their lack of specialization in the biomedical field. GatorTronGPT
(peng2023study, ) was introduced to alleviate this stigma, as it is pre-
trained on billions of clinical texts. Specifically, GatorTronGPT was pre-
trained on 277 billion words, 82 billion of which originated from de-
identified clinical text (yang2022large, ) and the remaining 195 billion were
obtained from Pile (gao2020pile, ), a diverse dataset specifically for
language modeling. GatorTronGPT was evaluated on downstream tasks such as
biomedical question answering and biomedical relation extraction. To evaluate
the quality of the generated texts, billions of synthetic text were generated
from GatorTronGPT that were used for fine-tuning a BERT-based model,
GatorTronS, on other downstream tasks such as clinical concept extraction,
biomedical relation extraction, semantic textual similarity, natural language
inference, and question answering. Additionally, it was shown that less than
half of 30 synthetic clinical paragraphs generated by GatorTronGPT were
identified to be synthetically generated by two physician evaluators at the
University of Florida. While GatorTronGPT takes steps to de-identify its
clinical data, the authors suggest future studies to validate the robustness
of re-identifying patients further.

##### Med-PaLM 2

Med-PaLM 2 (singhal2023towards, ) is an extension of Med-PaLM
(chung2022scaling, ) with several improvements. First, Med-PaLM 2 utilizes
PaLM 2 (anil2023palm, ) as the backbone LLM, unlike PaLM which was the
backbone of Med-PaLM. Second, to improve reasoning capabilities, Med-PaLM 2
introduces ensemble refinement, a prompting strategy that iteratively feeds in
prior generations, consisting of an answer and explanation, as well as the
given question and prompt to output a refined answer and generation.
Performance on the MedQA medical dataset is state-of-the-art by the time of
publication, beating the previous state-of-the-art (Med-PaLM) by over 19%.
Additionally, Med-PaLM 2 performed state-of-the-art or had comparable
performance to other clinical datasets, such as PubMedQA, MMLU, and MedMCQA.

#####  Applications:

There are multiple downstream applications for Biomedical LLMs. We roughly
divide them into two categories: clinical applications and research
applications.

##### NLP-related Clinic applications.

The most widely-researched directions in Biomedical NLP related to clinical
are Clinical Treatment Planning, Clinical Reports Generation, and Multi-Agents
Collaboration. Clinical Treatment Planning(rajpurkar2022ai, ) involves
diagnosing health issues and determining the suitable treatment course,
requiring healthcare professionals to assess objective medical test data and
subjective patient symptoms. Incorporating Large Language Models (LLMs) could
enhance diagnostic accuracy and broaden access to specialized medical
knowledge(thirunavukarasu2023large, ). In parallel, Clinical Reports
Generation(wu2023generalist, ), crucial for documenting patient medical
histories, benefits significantly from LLMs(wang2023chatcad, ). These models
facilitate the creation of reports by summarizing content, referencing
relevant information, and generating follow-up diagnostic comments and
medication recommendations based on clinical diagnoses(wu2023gpt4vision, ).
Furthermore, Multi-Agents Collaboration(xi2023rise, ), by simulating various
diagnostic scenarios, enables a synergy among agents to interpret data and
perform complex clinical tasks. This collaboration extends to integrating
Electronic Health Records (EHR)(solares2021transfer, ) and medication history,
thus offering real-time management and advisories. Such an approach augments
diagnostic accuracy, optimizes the duration of medical procedures, and
minimizes patient recovery times(tang2024medagents, ), offering a
comprehensive and dynamic support system for healthcare providers.

##### NLP-related Research applications.

The most widely-researched directions in Biomedical NLP are information
extraction (IE) related tasks, such as Named Entity Recognition (NER) and
Relation Extraction (RE), where the model is asked to extract entities (such
as genes, drug names, and diseases) or relations of the entities, from text
data in biomedical literature. Another prominent direction in the field is
Biomedical Question Answering, where the model is asked to answer biomedical-
related questions, given heavily professional background text. Since the
aforementioned tasks are very domain-specific, fine-tuning or pre-training
domain-specific LLMs are most effective when dealing with them (Gao2021APA, ;
Sheikhshab2018IndomainCT, ; Zhu2018ClinicalCE, ). The biomedical domain-
specific datasets are often created based on PubMed articles (jin2019pubmedqa,
; 10.1093/bib/bbac282, ; 10.1093/database/baw068, ), professional medical
examinations (jin2020disease, ), or real-life patient health information
(johnson2018mimic, ), with extensive expert annotation.

###  2.2. LLMs on Biological Sequences

Next, we extend the application of LLMs to the intricate realm of biological
sequence data, where a rich landscape of possibilities emerges. Within this
domain, we shift our focus to four major categories of biological sequences:
1) DNA sequences, 2) RNA sequences, 3) protein sequences, and 4) multi-omics
sequencing data.

Figure 2. Overview of models and applications for Genomic LLMs on biological
sequences.

####  2.2.1. DNA sequences:

The central dogma of molecular biology outlines the flow of genetic
information within living organisms, with DNA serving as the source of this
crucial information. DNA variants play a significant role in determining
heritable traits, influencing aspects related to health and disease. Various
innovative works in genomics, such as Enformer (Avsec2021, ), Nucleotide
Transformer (dalla2023nucleotide, ), GenSLMs (zvyagin2022genslms, ), DNABERT
(ji2021dnabert, ), GENA-LM (fishman2023gena, ), and HyenaDNA
(nguyen2023hyenadna, ), are proposed to decode the genetic blueprint of life.

##### Enformer

Enformer (Avsec2021, ) is designed to predict gene expression and chromatin
states purely from DNA sequences by integrating long-range interactions. It
utilizes transformer modules with self-attention to achieve a larger receptive
field, enabling the detection of sequence elements up to 100 kb (kilobases)
away, compared with 20 kb for Basenji2 (kelley2020cross, ) in the genome. By
increasing this information flow between distal elements and prioritizing
relevant data across the DNA sequence, the model can better capture biological
phenomena, such as enhancers that regulate promoters, despite a large DNA-
sequence distance between the two. Enformer demonstrates superior performance
over the previous model Basenji2 in gene expression prediction across genes
and CAGE experiments for protein-coding genes. It excels in predicting tissue
and cell-type-specific gene expression, enhancing accuracy in enhancer-
promoter prediction and non-coding variant effect prediction.

##### GenSLMs

GenSLMs (zvyagin2022genslms, ), or Genome-scale Language Models, represent a
groundbreaking approach to understanding the evolutionary dynamics of SARS-
CoV-2. These models utilize large language models (LLMs) adapted for genomic
data to learn the evolutionary landscape of SARS-CoV-2 genomes. By pre-
training on a vast number of prokaryotic gene sequences and fine-tuning on
SARS-CoV-2-specific data, GenSLMs can accurately and rapidly identify variants
of concern. They are among the first whole-genome scale foundation models
capable of generalizing to various prediction tasks. The models have been
demonstrated to scale efficiently on GPU-based supercomputers and AI-hardware
accelerators, achieving remarkable computational performance. This innovative
approach aims to enhance public health intervention strategies and inform
vaccine development for emerging variants by providing insights into the
evolutionary dynamics of SARS-CoV-2.

##### DNABERT

DNABERT (ji2021dnabert, ) is a pre-trained BERT model specifically designed
for genomic DNA sequences. It addresses the complexity of the gene regulatory
code by capturing global and transferable understandings of DNA sequences
based on nucleotide contexts. DNABERT excels in data-scarce scenarios,
achieving state-of-the-art performance in predicting splice sites and
transcription factor binding sites. It allows for direct visualization of
nucleotide-level importance and semantic relationships within sequences,
enhancing interpretability and identification of genetic variants.

##### GENA-LM

GENA-LM (fishman2023gena, ) represents a significant advancement in the field
of computational genomics, introducing a suite of open-source, transformer-
based foundational DNA language models specifically designed to handle long
DNA sequences. These models are capable of processing input lengths up to
36,000 base pairs, a substantial increase over previous architectures, which
struggled with the extensive contextual information spread across such long
sequences. Key features of GENA-LM include its use of Byte Pair Encoding (BPE)
tokenization, which differs from the k-mer approach used in earlier models
like DNABERT. This allows for a more efficient representation of DNA
sequences, facilitating the processing of longer inputs. The models have been
pre-trained on the latest T2T human genome assembly (Nurk2022-nq, ),
incorporating a wide range of genomic data to enhance their predictive
capabilities.

##### HyenaDNA

HyenaDNA (nguyen2023hyenadna, ) is a groundbreaking genomic foundation model
designed for long-range sequence modeling at single nucleotide resolution. It
addresses the limitations of previous models by enabling the processing of
ultralong sequences directly at the single nucleotide level without the need
for tokenizers. HyenaDNA is significantly smaller than previous genomic
models, yet it demonstrates exceptional performance, surpassing state-of-the-
art benchmarks on various datasets with fewer parameters and pre-training
data. This model scales sub-quadratically in sequence length, allowing for
efficient training and processing of long DNA sequences. Additionally,
HyenaDNA shows promise in species classification tasks and offers exciting
prospects for synthetic regulatory element design, gene identification, and
protein complex modeling. Despite its early-stage limitations related to
training on a single human genome and focusing solely on DNA, HyenaDNA’s
potential for personalized genomic analysis and innovative applications in
drug discovery is immense.

##### Nucleotide transformer

The Nucleotide Transformer model (Avsec2021, ) is a robust foundational model
pre-trained on raw DNA sequences from diverse human genomes and various
species. This model tokenizes sequences into six-character words (k-mers of
length 6) and utilizes the BERT methodology for training. It is then applied
to 18 downstream tasks covering various predictions like promoter prediction,
splice site donor and acceptor prediction, histone modifications, and more.
Predictions are made through probing or light fine-tuning, providing accurate
molecular phenotype predictions even in low-data settings by generating
context-specific representations of nucleotide sequences. The Nucleotide
Transformer outperforms specialized methods on multiple tasks, showcasing a
focus on key genomic elements such as gene expression regulators. Models
trained on genomes from different species show superior performance compared
to those trained solely on human sequences in various human prediction tasks,
indicating that diverse species training enhances generalization in human-
based predictions.

##### Applications:

Genomic foundation models play a crucial role in various tasks, including
predicting gene expression, deciphering DNA sequences for biological
functions, identifying genomic region structures, and predicting genetic
elements such as promoter regions, enhancers, and transcription factor binding
sites. These models are instrumental in understanding transcriptional
regulation and its connection to non-coding genetic variants linked to human
diseases and traits. For example, Enformer can directly predict enhancer-
promoter interactions from DNA sequences (Avsec2021, ), competing with methods
that rely on experimental data. Additionally, GenSLMs, trained on over 110
million prokaryotic gene sequences and fine-tuned for SARS-CoV-2, can
accurately and rapidly identify variants of concern (zvyagin2022genslms, ),
which also represents the generalization ability of these models to different
prediction tasks. DNABERT, fine-tuned for specific tasks like predicting
promoters, transcription factor binding sites, and splice sites, has been
benchmarked against state-of-the-art tools in the field (ji2021dnabert, ).
These genomic models that can accurately identify promoter regions and splice
sites, aid in understanding gene regulation and RNA transcript processing.
They can also predict epigenetic modifications (Dunham2012, ) and the effects
of DNA variations on genes and protein function (doi:10.1073/pnas.2311219120,
), helping identify genetic risk factors. Moreover, DLM models contribute to
understanding how non-coding DNA influences gene expression (Zhou2015-pp, )
and disease (10.1093/nar/gkz972, ) in different cell types, which is crucial
for human genetics and disease studies.

####  2.2.2. RNA sequences:

Ribonucleic acid (RNA) is another critical component of molecular biology,
playing various essential roles in fundamental biological processes. In recent
years, RNA has emerged as an intriguing target for drug development
(Garner2023-oj, ), underscoring the importance of enhancing our understanding
of its structures and functions. We embrace the innovative contributions
highlighted in foundation models such as RNABERT [(10.1093/nargab/lqac012, )],
RNA-FM (chen2022interpretable, ), and RNA-MSM (zhang2023multiple, ), which are
pivotal in elucidating the intricate mechanisms governing biological processes
mediated by RNA. These advancements empower us to decipher the complex
orchestration of biological processes by RNA molecules.

##### RNABERT

RNABERT (10.1093/nargab/lqac012, ) proposes an effective method for embedding
RNA bases by applying the pre-training approach of BERT to non-coding RNA
(ncRNA). The research aims to develop a robust RNA base embedding technique
for tasks like RNA structural alignment and clustering using deep
representation learning. By leveraging pre-training algorithms, the
researchers aim to create semantically rich representations of RNA bases to
improve the accuracy of structural alignment and clustering tasks. This method
of informative base embedding incorporates contextual information and
secondary structure details of RNA sequences, resulting in superior accuracies
compared to existing methods such as GraphClust (heyne2012graphclust, ),
EnsembleClust (saito2011fast, ), and CNNclust (aoki2018convolutional, ). The
approach combines informative base embedding with a simple Needleman–Wunsch
alignment algorithm to calculate structural alignments with improved time
complexity, making it a significant advancement in RNA sequence analysis.

##### RNA-FM

RNA-FM (chen2022interpretable, ) is a novel interpretable computational model
proposed to predict RNA structure and function accurately using unannotated
data. It leverages self-supervised learning on 23 million non-coding RNA
sequences to infer sequential and evolutionary information without labels. It
has shown effectiveness in predicting secondary/3D structures, SARS-CoV-2
genome structure, protein-RNA binding preferences, and gene expression
regulation. Despite being trained on unlabelled data, RNA-FM significantly
improves RNA structural and functional modeling results, making it a
foundational model for the field.

##### RNA-MSM

RNA-MSM (zhang2023multiple, ) is an RNA language model that leverages
homologous RNA sequences to interpret and predict RNA structures more
effectively than previous methods. It is distinguished by its use of
unsupervised learning on multiple sequence alignments (MSAs) derived from
RNAcmap, a tool that outperforms traditional methods like Rfam in identifying
homologous sequences due to its comprehensive and automatic pipeline. This
approach allows RNA-MSM to capture evolutionary information and structural
nuances of RNA sequences, which are less conserved and thus more challenging
to analyze compared to proteins. The model generates two-dimensional attention
maps and one-dimensional embeddings that contain rich structural information.
These outputs can be directly correlated with RNA structural features, such as
2D base pairing probabilities and 1D solvent accessibilities, with high
accuracy. The study demonstrates that RNA-MSM significantly outperforms
existing techniques, including SPOT-RNA2 and RNAsnap2, in predicting these
structural features after fine-tuning. This research underscores the potential
of RNA-MSM to revolutionize RNA structure prediction and functional analysis
by incorporating evolutionary and structural insights from homologous
sequences.

##### Applications:

RNA language models have a wide range of applications in RNA research and
biotechnology. These models utilize evolutionary information from homologous
sequences to accurately interpret RNA sequences, making them valuable in tasks
such as RNA structure (yamada2022prediction, ), function (zhang2023multiple,
), and RNA-protein interaction prediction (chen2022interpretable, ). They can
predict crucial aspects of RNA structures, such as 2D base pairing
probabilities and 1D solvent accessibilities (zhang2023multiple, ), which are
essential for understanding RNA function and cellular interactions.
Additionally, RNA language models facilitate the design of RNA-based
therapeutics (Li2023.09.09.556981, ). Understanding RNA structures helps
identify novel drug targets and design RNA-based drugs, speeding up drug
discovery processes. These pre-trained models can be fine-tuned for various
downstream tasks related to RNA structure and function, including RNA 2D/3D
structure prediction (zhang2023multiple, ), RNA structural alignment and
family clustering (10.1093/nargab/lqac012, ), and RNA splice site prediction
(Chen2023.01.31.526427, ) from RNA sequences.

####  2.2.3. Protein sequences:

Venturing into the complex realm of proteins, we embrace luminous works such
as (rives2021biological, ), (bepler2021learning, ), (brandes2022proteinbert,
), (madani2023large, ), (lin2023evolutionary, ), (zheng2023structure, ), and
(xu2023protst, ). These endeavors illuminate the path to unraveling the
intricate choreography of molecular functions and interactions. The protein
LLMs have a wide application in functional protein generation
(leinonen2004uniprot, ) and protein structure prediction (suzek2015uniref, ).

Protein language models can be broadly classified into three main categories
based on their functionalities: 1) structure and function understanding:
interpreting protein sequences to reveal their structures and functions
through evolutionary data or textual enhancements, 2) protein sequence
generation: creating novel protein sequences using generative modeling for
unexplored spaces or precise design, and 3) multi-modal integration for
enhanced understanding and generation: combining various data types to
generate protein functions or achieve a unified understanding and generation
of protein sequences.

##### Structure and Function Understanding models

These models are primarily concerned with interpreting existing protein
sequences to deduce their structures, functions, or both, using vast datasets
of known proteins. ProteinBERT (brandes2022proteinbert, ) is a deep-learning
model designed for understanding protein functions and sequences. It combines
language modeling with Gene Ontology (GO) (ashburner2000gene, ) annotation
predictions in a self-supervised training approach. Trained on over 106
million proteins from the UniProtKB/UniRef90 database (boutet2007uniprotkb, ;
suzek2007uniref, ), it efficiently captures detailed and broad protein
features using a transformer-like architecture. ESM models are noted for their
ability to capture deep evolutionary signals and predict protein structures
with high accuracy, contributing significantly to structural biology. ESM-1b
(rives2021biological, ), utilizing a Transformer architecture, harnesses
unsupervised learning from 250 million protein sequences to predict biological
properties. It offers a versatile and comprehensive approach to protein
analysis, with applications ranging from structure prediction to functional
annotation, by capturing deep evolutionary signals across various biological
research tasks. ESM-2 (lin2023evolutionary, ) significantly advances over
ESM-1b by utilizing a scaled-up model with 15 billion parameters (compared to
650 million) and a richer UniRef90 dataset to predict atomic-level protein
structures directly from sequences. This enhancement not only allows for high-
resolution structure predictions and the construction of the ESM Metagenomic
Atlas but also achieves a notable speed improvement and wider applicability in
structural genomics and protein design, thus enhancing diversity and accuracy
in structural predictions. ProtST (xu2023protst, ) enhances protein language
models by merging protein sequences with biomedical texts from the
ProtDescribe dataset, sourced from Swiss-Prot (bairoch2000swiss, ), to provide
a comprehensive understanding of protein functions and properties. This multi-
modality learning not only addresses gaps in sequence-only models but also
enhances protein representations for a variety of applications, including
function annotation and localization prediction. xTrimoPGLM
(chen2024xtrimopglm, ) revolutionizes protein sequence analysis and generation
by merging autoencoding and autoregressive objectives within a massive 100
billion parameter framework, trained on about 940 million unique protein
sequences.

##### Sequence Generation based models

In the quest to generate novel protein sequences with the potential to fold
into functional proteins, several key players have emerged, notably ProGen
(madani2023large, ), ProGen2 (nijkamp2023progen2, ), ProtGPT2
(ferruz2022protgpt2, ), and LM-Design (zheng2023structure, ). ProGen starts
the journey by training on a vast array of 280 million protein sequences,
using special tags to make proteins with specific functions (madani2023large,
). ProGen2 takes this further by using a much larger model to capture more
complex protein patterns and making it possible to predict protein fitness
without extra steps (nijkamp2023progen2, ). ProtGPT2, inspired by language
models, generates proteins that could exist in nature, trained on 50 million
sequences for wide exploration. While ProGen series focus on making diverse
and functional proteins, ProtGPT2 pushes the limits of designing completely
new proteins, showing the power of deep learning in understanding and creating
life’s building blocks (ferruz2022protgpt2, ). LM-Design innovates in protein
design by infusing language models with structural adapters, enhancing the
generation of sequences. This addresses both the creativity seen in ProtGPT2
and the functional specificity of the ProGen series with an added focus on
structural viability (zheng2023structure, ).

##### Multi-modal Integration models

ProtST (xu2023protst, ), Prot2Text (abdine2023prot2text, ), and xTrimoPGLM
(chen2024xtrimopglm, ) exemplify the fusion of sequence, structure, and
textual data to redefine protein function understanding and design. ProtST
integrates dual language models, specifically the ESM series and ProtBert for
protein sequences, alongside PubmedBERT (gu2021domain, ) for biomedical texts,
to synergize textual descriptions with sequence data (xu2023protst, ).
Prot2Text utilizes Graph Neural Networks and Transformers to generate
descriptive texts of protein functions, integrating structural and sequence
data with textual annotations. This approach not only enhances function
annotation and drug discovery efforts but also enriches databases with
contextually rich protein descriptions, setting a new standard for how protein
functions are predicted and understood (abdine2023prot2text, ). xTrimoPGLM,
while its primary category is understanding and generation, its methodological
approach of combining masked language modeling (MLM) and general language
modeling (GLM) objectives allows it to handle multimodal data for enhanced
protein sequence analysis (chen2024xtrimopglm, ).

#####  Applications:

In the rapidly evolving field of computational biology, all these protein
sequence based large language methods (rives2021biological, ;
lin2023evolutionary, ; abdine2023prot2text, ; brandes2022proteinbert, ;
xu2023protst, ; chen2024xtrimopglm, ) are pushing the boundaries of how we
understand, design, and utilize proteins. The application of these models
spans from detailed protein function annotation (brandes2022proteinbert, ) and
structure prediction (rives2021biological, ; lin2023evolutionary, ) to the
high-throughput design of novel proteins with specific functionalities
(ferruz2022protgpt2, ; madani2023large, ; nijkamp2023progen2, ), significantly
impacting drug discovery, synthetic biology, and therapeutic development
(chen2024xtrimopglm, ; lin2023evolutionary, ). Looking forward, the
integration of even more diverse datasets, the refinement of models to enhance
accuracy and efficiency, and the exploration of uncharted areas of the protein
universe remain pivotal directions.

####  2.2.4. Multi-omics sequencing data:

Within these domains, the transformative capabilities of LLMs manifest in
high-impact downstream applications. From predicting molecular structures to
forecasting molecule interactions, and from unraveling molecule functions to
drawing poignant associations with disease progression processes, LLMs guides
us towards a deeper comprehension of life’s building blocks. In multi-omic
sequencing, models such as scBERT(yang2022scbert, ),
Geneformer(theodoris2023transfer, ), scFoundation(hao2023large, ) and
scGPT(cui2024scgpt, ) use advanced pre-trained language models to deeply
analyze cellular biology and genetics, showcasing the fusion of cutting-edge
computation with biological understanding.

##### scBERT

scBERT (yang2022scbert, ), a transformer-based deep neural network model,
leverages extensive unlabeled single-cell RNA-seq data for pre-training,
addressing the challenges of cell type annotation by capturing latent gene-
gene interactions and overcoming limitations such as batch effects and the
absence of curated marker genes. Through self-supervised learning and
supervised fine-tuning, scBERT significantly enhances cell type annotation
accuracy, robustness against batch effects, and interpretability over existing
methods. Its versatility extends to discovering novel cell types and
understanding complex gene expression patterns, offering a comprehensive tool
for single-cell omics analysis.

##### Geneformer

Geneformer (theodoris2023transfer, ), pre-trained on about 30 million single-
cell transcriptomes, employs a context-aware, attention-based framework for
versatile applications in network biology, including disease modeling and
therapeutic target identification, especially in data-scarce areas. It
differentiates itself with a self-supervised learning strategy for fundamental
insights into network dynamics. This emphasizes its utility in a broader range
of downstream tasks such as chromatin dynamics predictions. In comparison,
scBERT, focusing on cell type annotation, demonstrates its strength in
handling batch effects and discovering novel cell types through its
transformer architecture. Both models offer significant advancements over
traditional methods, with Geneformer offering a broader application scope and
scBERT excelling in cell type specificity and interpretability.

##### scFoundation

scFoundation (hao2023large, ) redefines the landscape of single-cell analysis
with its large-scale pre-trained model, trained on an unprecedented dataset of
over 50 million human single-cell transcriptomes. Through its unique
asymmetric architecture and the introduction of a read-depth-aware pre-
training task, the model efficiently addresses the complexities of single-cell
RNA-seq data, including its sparsity and high dimensionality. Distinguished by
its vast scale of trainable parameters and extensive data coverage,
scFoundation delivers superior performance across a spectrum of downstream
applications, from enhancing gene expression to predicting drug responses.

##### scGPT

scGPT (cui2024scgpt, ), trained on 33 million cells, significantly advances
cell biology and genetics research with its large-scale data analysis. It
differentiates itself by applying generative pre-training on extensive multi-
omics data for a comprehensive analysis. Using a self-attention transformer,
scGPT excels in analyzing complex cell data, enabling it to perform
exceptionally well in identifying cell types, predicting genetic changes, and
integrating multi-omics data. Its ability to derive insights into gene-gene
interactions and demonstrate a scaling effect of improved performance with
larger datasets highlights its potential for single-cell omics research.

#####  Applications:

The integration of advanced computational models like scBERT (yang2022scbert,
), scFoundation (hao2023large, ), scGPT (cui2024scgpt, ), and Geneformer
(theodoris2023transfer, ) into single-cell and multi-omics analysis heralds a
new era in precision biology. These models, each with its unique approach
ranging from cell type annotation(yang2022scbert, ; cui2024scgpt, ) to the
elucidation of complex gene networks, offer comprehensive tools for dissecting
the intricacies of cellular functions. Future research directions anticipate
the convergence of these models into a cohesive analytical framework,
enhancing our capability to predict disease mechanisms and responses to
treatment based on sophisticated cell-level predictions (hao2023large, ;
theodoris2023transfer, ). As the pool of omics data expands, these models will
continue to evolve, driving forward the discovery of novel biological
mechanisms and therapeutic targets (theodoris2023transfer, ).

###  2.3. LLMs on Brain Signals

Last, we delve into the fascinating realm of applying LLMs to brain signals.
In this section, we focus on two topics: brain LLM building and downstream
applications in brain-to-text translation. We start with introducing various
pre-training techniques for brain EEG signal representation learning,
including BrainBERT (self-supervised representation learning for intracranial
recordings) (wang2022brainbert, ), MMM (topology-agnostic EEG representation
learning) (yi2023learning, ), and LaBraM (generic representation learning for
tremendous EEG data from various sources) (jiang2024large, ).

Figure 3. Overview of models, downstream tasks and challenges for brain signal
LLMs.

##### BrainBERT

BrainBERT (wang2022brainbert, ) is a Transformer-based model designed for
decoding intracranial field potential recordings. Pretrained unsupervised on
large datasets of neural activities recorded while subjects watched movies,
BrainBERT facilitates significant improvements in decoding neural signals,
addressing the challenges posed by the scarcity and variability of neural
data. It excels in generalizing across different subjects and electrode
locations, a notable advantage over traditional methods. Employing techniques
like super-resolution spectrograms and contextual representation, BrainBERT
outperforms existing models, offering enhanced accuracy with less data.
BrainBERT demonstrates superior performance across various neural decoding
tasks, establishing a new benchmark in the field compared to standard linear
decoders and deeper neural networks. The model not only improves brain-
computer interface technologies but also enables novel analyses, such as
exploring the intrinsic dimensionality of brain computations.

##### MMM

MMM (yi2023learning, ) is a novel framework for EEG pre-training. MMM achieves
this by mapping different EEG channel selections onto a unified topology and
employing strategies such as Multi-dimensional Positional Encoding, Multi-
level Channel Hierarchy, and Multi-stage Pre-training. This approach
facilitates topology-agnostic EEG representation learning, enhancing cross-
dataset generalizability. Evaluations on SEED (seed, ) and SEED-IV (seediv, )
datasets demonstrate MMM’s superior performance in emotion recognition tasks,
outperforming traditional and deep learning baselines. Specifically, MMM
surpasses previous state-of-the-art methods in subject-dependent
classification, showcasing its robust transferability across different EEG
datasets and sensor configurations. The results validate MMM’s effectiveness
in leveraging larger datasets for extracting more generalized representations.

##### LaBraM

LaBraM (jiang2024large, ) is a novel Large Brain Model, which marks a
significant advancement in EEG-based deep learning through its pre-training on
over 2500 hours of diverse EEG data. LaBraM overcomes the limitations of EEG
data variability and volume by transforming EEG signals into unified channel
patches and employing vector-quantized neural spectrum prediction for
efficient learning. The model’s architecture enables the effective learning of
EEG signal representations, addressing both temporal and spatial features.
LaBraM is evaluated on various downstream tasks such as abnormal detection,
event type classification, emotion recognition, and gait prediction. In these
evaluations, LaBraM consistently outperforms state-of-the-art methods across
multiple metrics, underscoring its superior ability to generalize from large-
scale EEG data. Moreover, the model exhibits scalability with increasing
dataset sizes, suggesting potential for further performance improvements with
even larger datasets. This research not only sets new benchmarks for EEG-based
analyses but also opens up new avenues for deep learning applications in
neuroscience and medical diagnostics.

#####  Applications:

We further discuss a broad range of brain LLM applications in brain-computer
interfaces (BCIs).

##### Brain-to-Text Translation:

Previous research on converting brain signals to text, as described in
papers(herff2015brain, ; sun2019towards, ; anumanchipalli2019speech, ;
makin2020machine, ; panachakel2021decoding, ; moses2021neuroprosthesis, ;
nieto2022thinking, ), has shown notable success with small and restricted
vocabularies. However, these studies have struggled to achieve similar
accuracy levels with larger and unrestricted vocabularies. Building upon this
foundation, we further introduce an exciting topic of open-vocabulary brain-
to-text translation (wang2022open, ), including recent work DeWave (discrete
encoding for EEG-to-text translation) (duan2023dewave, ) and continuous
language reconstruction from fMRI images (tang2023semantic, ). Starting with
EEG2TEXT (wang2022open, ), this paper presents a method of open vocabulary
EEG-To-Text decoding and zero-shot sentence sentiment classification . Tested
on the ZuCo dataset (zuco, ), the author utilizes pre-trained language models
like BART (bart, ), achieving significant advancements with a 40.1% BLEU-1
score for EEG-To-Text decoding and a 55.6% F1 score for zero-shot EEG-based
ternary sentiment classification, which notably surpass the supervised
baselines. Further advancing the field, DeWave (duan2023dewave, ) is a novel
framework for converting EEG signals into text by leveraging a discrete codex
encoding. DeWave utilizes a quantized variational encoder to transform EEG
waves into discrete representations, aligning them with pre-trained language
models for enhanced translation accuracy. The result surpassing previous
baselines including EEG2TEXT. In a parallel vein, the paper (tang2023semantic,
) explores a non-invasive brain-computer interface decoding continuous natural
language from fMRI brain recordings. Employing a unique approach, the model
addresses fMRI’s low temporal resolution by generating candidate word
sequences, scoring them against brain responses to identify the most likely
stimuli being heard or imagined.

##### Brain-to-Image Translation:

Image Translation is also a mainstream Downstream Tasks of brain signals. Some
recent interesting work includes nature image decoding (song2024decoding, )
NeuroGAN (NeuroGAN, ) and EEG2IMAGE (EEG2IMAGE, ). Specifically, nature image
decoding (song2024decoding, ) introduces a self-supervised framework, which
leverages a large and diverse EEG-image dataset (imageeeg, ). This dataset is
used in conjunction with a novel approach that applies contrastive learning to
align features extracted from image stimuli and corresponding EEG responses,
significantly advancing the field of non-invasive brain-computer interfaces.
The framework employs self-attention and graph attention modules within the
EEG encoder to enhance spatial feature extraction, reflecting the spatial
dynamics of brain activity related to object recognition. NeuroGAN (NeuroGAN,
) introduces a sophisticated method for transforming EEG signals into visual
images, leveraging a specialized architecture to enhance image synthesis from
brain activity data. Central to NeuroGAN is a cross-modality encoder-decoder
structure, which effectively compresses EEG features into a latent space and
reconstructs corresponding images, focusing on capturing the complex
relationship between neural signals and visual stimuli. The method also
incorporates a perceptual loss function, utilizing a pre-trained image
classifier to measure the perceptual similarity between generated and real
images. Similarly, EEG2IMAGE (EEG2IMAGE, ) introduces an innovative framework
for synthesizing images from EEG signals. The framework uses small EEG
datasets to learn features via a contrastive learning approach and synthesizes
images using a modified conditional Generative Adversarial Network.
Specifically, the framework employs semi-hard triplet loss for feature
extraction from EEG signals, ensuring that signals from similar images are
closer to the learned feature space, leading to more accurate image
reconstruction.

##### Other Brain EEG Applications:

In addition to text translation and image translation, brain signal decoding
also performs well in other downstream tasks. In the field of voice decoding,
NeuroTalk (voicere, ) presents a novel model, for transforming non-invasive
EEG signals of imagined speech into audible voice outputs. The model combines
multi-receptive residual modules with recurrent neural networks to process
brain signals effectively. Notably, the model could generate words not
included in the training dataset, indicating its potential for broader
vocabulary coverage. In the motion decoding domain, Motor Recognition
(motoreeg, ) explores using EEG data for recognizing motor activities. The
research employs an innovative ensemble approach combining stacked
bidirectional long short-term memory (BiLSTM) with long short-term memory
(LSTM) networks alongside a newly proposed EEG-transformer network to classify
17 different everyday motor activities. Their ensemble achieves a
classification accuracy of 98.5%, which has a substantial advancement over
existing state-of-the-art methodologies.

##  3\. Conclusions and Future Directions

As a conclusion, this is a comprehensive survey of large language models for
biomedicine, focusing on three pivotal data types: 1) textual data, 2)
biological sequences, and 3) brain signals. There are also significant new
challenges that come with using AI in biomedical research. One big challenge
is making sure that the biomedical insights enhanced by AI are reliable and
trustworthy, including model explainability and interpretability, model
robustness to adversarial attacks, model bias towards different populations,
and data privacy issues. Another challenge is the personalization of LLMs,
which means adjusting LLMs to fit the specific needs of different personalized
data. For example, there is a large individual variance in brain signals when
different people are thinking of the same word under the same context. Instead
of using one LLM to fit everyone, can we construct personalized LLMs based on
different brain patterns for different people? The last challenge is the
multi-modality data. Since biomedical information can be very varied, we learn
how to handle different types of data in a skillful and effective way. For
example, Google has announced Med-PaLM-2 (singhal2023towards, ) that
integrates image, text, and genome data in the electronic health record,
declaring an expert-level ability for medical question answering. Can we
develop more effective and efficient methods to integrate multi-modal and
multi-omic LLMs into one powerful unified LLM? This survey paper serves as a
foundation step to bridge the gap and encourages both theoretical researchers
and practitioners in LLMs to look into real-world scientific applications.

## Acknowledgement

This work is sponsored by the Commonwealth Cyber Initiative, Children’s
National Hospital, Fralin Biomedical Research Institute (Virginia Tech),
Sanghani Center for AI and Data Analytics (Virginia Tech), Virginia Tech
Innovation Campus, and a generous gift from the Amazon + Virginia Tech Center
for Efficient and Robust Machine Learning.

## References

  * [1] Hadi Abdine, Michail Chatzianastasis, Costas Bouyioukos, and Michalis Vazirgiannis.  Prot2text: Multimodal protein’s function generation with gnns and transformers.  arXiv preprint arXiv:2307.14367, 2023.
  * [2] Manato Akiyama and Yasubumi Sakakibara.  Informative RNA base embedding for RNA structural alignment and clustering by deep representation learning.  NAR Genomics and Bioinformatics, 4(1):lqac012, 02 2022.
  * [3] Sultan Alrowili and K Vijay-Shanker.  Biom-transformers: building large biomedical language models with bert, albert and electra.  In Proceedings of the 20th workshop on biomedical language processing, pages 221–227, 2021.
  * [4] Emily Alsentzer, John R Murphy, Willie Boag, Wei-Hung Weng, Di Jin, Tristan Naumann, WA Redmond, and Matthew BA McDermott.  Publicly available clinical bert embeddings.  NAACL HLT 2019, page 72, 2019.
  * [5] Waleed Ammar, Dirk Groeneveld, Chandra Bhagavatula, Iz Beltagy, Miles Crawford, Doug Downey, Jason Dunkelberger, Ahmed Elgohary, Sergey Feldman, Vu Ha, Rodney Kinney, Sebastian Kohlmeier, Kyle Lo, Tyler Murray, Hsu-Han Ooi, Matthew Peters, Joanna Power, Sam Skjonsberg, Lucy Lu Wang, Chris Wilhelm, Zheng Yuan, Madeleine van Zuylen, and Oren Etzioni.  Construction of the literature graph in semantic scholar.  pages 84–91, June 2018.
  * [6] Rohan Anil, Andrew M Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, et al.  Palm 2 technical report.  arXiv preprint arXiv:2305.10403, 2023.
  * [7] Gopala K Anumanchipalli, Josh Chartier, and Edward F Chang.  Speech synthesis from neural decoding of spoken sentences.  Nature, 568(7753):493–498, 2019.
  * [8] Genta Aoki and Yasubumi Sakakibara.  Convolutional neural networks for classification of alignments of non-coding rna sequences.  Bioinformatics, 34(13):i237–i244, 2018.
  * [9] Michael Ashburner, Catherine A Ball, Judith A Blake, David Botstein, Heather Butler, J Michael Cherry, Allan P Davis, Kara Dolinski, Selina S Dwight, Janan T Eppig, et al.  Gene ontology: tool for the unification of biology.  Nature genetics, 25(1):25–29, 2000.
  * [10] Žiga Avsec, Vikram Agarwal, Daniel Visentin, Joseph R. Ledsam, Agnieszka Grabska-Barwinska, Kyle R. Taylor, Yannis Assael, John Jumper, Pushmeet Kohli, and David R. Kelley.  Effective gene expression prediction from sequence by integrating long-range interactions.  Nature Methods, 18(10):1196–1203, Oct 2021.
  * [11] Amos Bairoch and Rolf Apweiler.  The swiss-prot protein sequence database and its supplement trembl in 2000.  Nucleic acids research, 28(1):45–48, 2000.
  * [12] Iz Beltagy, Kyle Lo, and Arman Cohan.  Scibert: A pretrained language model for scientific text.  In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3615–3620, 2019.
  * [13] Gonzalo Benegas, Sanjit Singh Batra, and Yun S. Song.  Dna language models are powerful predictors of genome-wide variant effects.  Proceedings of the National Academy of Sciences, 120(44):e2311219120, 2023.
  * [14] Tristan Bepler and Bonnie Berger.  Learning the protein language: Evolution, structure, and function.  Cell systems, 12(6):654–669, 2021.
  * [15] Emmanuel Boutet, Damien Lieberherr, Michael Tognolli, Michel Schneider, and Amos Bairoch.  Uniprotkb/swiss-prot: the manually annotated section of the uniprot knowledgebase.  In Plant bioinformatics: methods and protocols, pages 89–112. Springer, 2007.
  * [16] Nadav Brandes, Dan Ofer, Yam Peleg, Nadav Rappoport, and Michal Linial.  Proteinbert: a universal deep-learning model of protein sequence and function.  Bioinformatics, 38(8):2102–2110, 2022.
  * [17] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al.  Language models are few-shot learners.  Advances in neural information processing systems, 33:1877–1901, 2020.
  * [18] Bo Chen, Xingyi Cheng, Pan Li, Yangli-ao Geng, Jing Gong, Shen Li, Zhilei Bei, Xu Tan, Boyan Wang, Xin Zeng, et al.  xtrimopglm: unified 100b-scale pre-trained transformer for deciphering the language of protein.  arXiv preprint arXiv:2401.06199, 2024.
  * [19] Jiayang Chen, Zhihang Hu, Siqi Sun, Qingxiong Tan, Yixuan Wang, Qinze Yu, Licheng Zong, Liang Hong, Jin Xiao, Tao Shen, et al.  Interpretable rna foundation model from unannotated data for highly accurate rna structure and function predictions.  bioRxiv, pages 2022–08, 2022.
  * [20] Ken Chen, Yue Zhou, Maolin Ding, Yu Wang, Zhixiang Ren, and Yuedong Yang.  Self-supervised learning on millions of pre-mrna sequences improves sequence-based rna splicing prediction.  bioRxiv, 2023.
  * [21] Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al.  Scaling instruction-finetuned language models.  arXiv preprint arXiv:2210.11416, 2022.
  * [22] Haotian Cui, Chloe Wang, Hassaan Maan, Kuan Pang, Fengning Luo, Nan Duan, and Bo Wang.  scgpt: toward building a foundation model for single-cell multi-omics using generative ai.  Nature Methods, pages 1–11, 2024.
  * [23] Hugo Dalla-Torre, Liam Gonzalez, Javier Mendoza-Revilla, Nicolas Lopez Carranza, Adam Henryk Grzywaczewski, Francesco Oteri, Christian Dallago, Evan Trop, Hassan Sirelkhatim, Guillaume Richard, et al.  The nucleotide transformer: Building and evaluating robust foundation models for human genomics.  bioRxiv, pages 2023–01, 2023.
  * [24] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.  Bert: Pre-training of deep bidirectional transformers for language understanding.  arXiv preprint arXiv:1810.04805, 2018.
  * [25] Yiqun Duan, Charles Zhou, Zhen Wang, Yu-Kai Wang, and Chin-teng Lin.  Dewave: Discrete encoding of eeg waves for eeg to text translation.  In Thirty-seventh Conference on Neural Information Processing Systems, 2023.
  * [26] Ian Dunham, Anshul Kundaje, Shelley F. Aldred, Patrick J. Collins, Carrie A. Davis, Francis Doyle, Charles B. Epstein, Seth Frietze, Jennifer Harrow, Rajinder Kaul, and et al.  An integrated encyclopedia of dna elements in the human genome.  Nature, 489(7414):57–74, Sep 2012.
  * [27] Noelia Ferruz, Steffen Schmidt, and Birte Höcker.  Protgpt2 is a deep unsupervised language model for protein design.  Nature communications, 13(1):4348, 2022.
  * [28] Veniamin Fishman, Yuri Kuratov, Maxim Petrov, Aleksei Shmelev, Denis Shepelin, Nikolay Chekanov, Olga Kardymon, and Mikhail Burtsev.  Gena-lm: A family of open-source foundational models for long dna sequences.  bioRxiv, pages 2023–06, 2023.
  * [29] Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, et al.  The pile: An 800gb dataset of diverse text for language modeling.  arXiv preprint arXiv:2101.00027, 2020.
  * [30] Shang Gao, Olivera Kotevska, Alexandre Sorokine, and James Blair Christian.  A pre-training and self-training approach for biomedical named entity recognition.  PloS one, 16 2:e0246310, 2021.
  * [31] Amanda L Garner.  Contemporary progress and opportunities in RNA-targeted drug discovery.  ACS Med. Chem. Lett., 14(3):251–259, March 2023.
  * [32] Alessandro T. Gifford, Kshitij Dwivedi, Gemma Roig, and Radoslaw M. Cichy.  A large and rich eeg dataset for modeling human visual object recognition.  NeuroImage, 264:119754, 2022.
  * [33] Yu Gu, Robert Tinn, Hao Cheng, Michael Lucas, Naoto Usuyama, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, and Hoifung Poon.  Domain-specific language model pretraining for biomedical natural language processing.  ACM Transactions on Computing for Healthcare (HEALTH), 3(1):1–23, 2021.
  * [34] Minsheng Hao, Jing Gong, Xin Zeng, Chiming Liu, Yucheng Guo, Xingyi Cheng, Taifeng Wang, Jianzhu Ma, Le Song, and Xuegong Zhang.  Large scale foundation model on single-cell transcriptomics.  bioRxiv, pages 2023–05, 2023.
  * [35] Christian Herff, Dominic Heger, Adriana De Pesters, Dominic Telaar, Peter Brunner, Gerwin Schalk, and Tanja Schultz.  Brain-to-text: decoding spoken phrases from phone representations in the brain.  Frontiers in neuroscience, 9:217, 2015.
  * [36] Steffen Heyne, Fabrizio Costa, Dominic Rose, and Rolf Backofen.  Graphclust: alignment-free structural clustering of local rna secondary structures.  Bioinformatics, 28(12):i224–i232, 2012.
  * [37] Nora Hollenstein, Jonathan Rotsztejn, Marius Troendle, Andreas Pedroni, Ce Zhang, and Nicolas Langer.  Zuco, a simultaneous eeg and eye-tracking resource for natural sentence reading.  Scientific data, 5(1):1–13, 2018.
  * [38] Yanrong Ji, Zhihan Zhou, Han Liu, and Ramana V Davuluri.  Dnabert: pre-trained bidirectional encoder representations from transformers model for dna-language in genome.  Bioinformatics, 37(15):2112–2120, 2021.
  * [39] Wei-Bang Jiang, Li-Ming Zhao, and Lu Bao-Liang.  Large brain model for learning generic representations with tremendous eeg data in bci.  In The Thriteenth International Conference on Learning Representations, 2024.
  * [40] Di Jin, Eileen Pan, Nassim Oufattole, Wei-Hung Weng, Hanyi Fang, and Peter Szolovits.  What disease does this patient have? a large-scale open domain question answering dataset from medical exams.  arXiv preprint arXiv:2009.13081, 2020.
  * [41] Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William Cohen, and Xinghua Lu.  Pubmedqa: A dataset for biomedical research question answering.  In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2567–2577, 2019.
  * [42] Alistair E W Johnson, David J Stone, Leo A Celi, and Tom J Pollard.  The mimic code repository: enabling reproducibility in critical care research.  Journal of the American Medical Informatics Association, 25(1):32–39, 2018.
  * [43] Alistair EW Johnson, Tom J Pollard, Lu Shen, Li-wei H Lehman, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G Mark.  Mimic-iii, a freely accessible critical care database.  Scientific data, 3(1):1–9, 2016.
  * [44] Pallavi Kaushik, Ilina Tripathi, and Partha Pratim Roy.  Motor activity recognition using eeg data and ensemble of stacked BLSTM-LSTM network and transformer model.  In IEEE International Conference on Acoustics, Speech and Signal Processing ICASSP 2023, Rhodes Island, Greece, June 4-10, 2023, pages 1–5. IEEE, 2023.
  * [45] David R Kelley.  Cross-species regulatory sequence activity prediction.  PLoS computational biology, 16(7):e1008050, 2020.
  * [46] Anastasia Krithara, Anastasios Nentidis, Konstantinos Bougiatiotis, and Georgios Paliouras.  Bioasq-qa: A manually curated corpus for biomedical question answering.  Scientific Data, 10(1):170, 2023.
  * [47] Taku Kudo and John Richardson.  SentencePiece: A simple and language independent subword tokenizer and detokenizer for neural text processing.  In Eduardo Blanco and Wei Lu, editors, Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 66–71, Brussels, Belgium, November 2018. Association for Computational Linguistics.
  * [48] Melissa J Landrum, Shanmuga Chitipiralla, Garth R Brown, Chao Chen, Baoshan Gu, Jennifer Hart, Douglas Hoffman, Wonhee Jang, Kuljeet Kaur, Chunlei Liu, Vitaly Lyoshin, Zenith Maddipatla, Rama Maiti, Joseph Mitchell, Nuala O’Leary, George R Riley, Wenyao Shi, George Zhou, Valerie Schneider, Donna Maglott, J Bradley Holmes, and Brandi L Kattman.  ClinVar: improvements to accessing data.  Nucleic Acids Research, 48(D1):D835–D844, 11 2019.
  * [49] Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, and Jaewoo Kang.  Biobert: a pre-trained biomedical language representation model for biomedical text mining.  Bioinformatics, 36(4):1234–1240, 2020.
  * [50] Young-Eun Lee, Seo-Hyun Lee, Sang-Ho Kim, and Seong-Whan Lee.  Towards voice reconstruction from EEG during imagined speech.  In Brian Williams, Yiling Chen, and Jennifer Neville, editors, Thirty-Seventh AAAI Conference on Artificial Intelligence, AAAI 2023, Thirty-Fifth Conference on Innovative Applications of Artificial Intelligence, IAAI 2023, Thirteenth Symposium on Educational Advances in Artificial Intelligence, EAAI 2023, Washington, DC, USA, February 7-14, 2023, pages 6030–6038. AAAI Press, 2023.
  * [51] Rasko Leinonen, Federico Garcia Diez, David Binns, Wolfgang Fleischmann, Rodrigo Lopez, and Rolf Apweiler.  Uniprot archive.  Bioinformatics, 20(17):3236–3237, 2004.
  * [52] Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zettlemoyer.  Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension, 2019.
  * [53] Jiao Li, Yueping Sun, Robin J. Johnson, Daniela Sciaky, Chih-Hsuan Wei, Robert Leaman, Allan Peter Davis, Carolyn J. Mattingly, Thomas C. Wiegers, and Zhiyong Lu.  BioCreative V CDR task corpus: a resource for chemical disease relation extraction.  Database, 2016:baw068, 05 2016.
  * [54] Sizhen Li, Saeed Moayedpour, Ruijiang Li, Michael Bailey, Saleh Riahi, Lorenzo Kogler-Anele, Milad Miladi, Jacob Miner, Dinghai Zheng, Jun Wang, Akshay Balsubramani, Khang Tran, Minnie Zacharia, Monica Wu, Xiaobo Gu, Ryan Clinton, Carla Asquith, Joseph Skaleski, Lianne Boeglin, Sudha Chivukula, Anusha Dias, Fernando Ulloa Montoya, Vikram Agarwal, Ziv Bar-Joseph, and Sven Jager.  Codonbert: Large language models for mrna design and optimization.  bioRxiv, 2023.
  * [55] Zeming Lin, Halil Akin, Roshan Rao, Brian Hie, Zhongkai Zhu, Wenting Lu, Nikita Smetanin, Robert Verkuil, Ori Kabeli, Yaniv Shmueli, et al.  Evolutionary-scale prediction of atomic-level protein structure with a language model.  Science, 379(6637):1123–1130, 2023.
  * [56] Fangyu Liu, Qianchu Liu, Shruthi Bannur, Fernando Pérez-García, Naoto Usuyama, Sheng Zhang, Tristan Naumann, Aditya Nori, Hoifung Poon, Javier Alvarez-Valle, et al.  Compositional zero-shot domain transfer with text-to-text models.  arXiv preprint arXiv:2303.13386, 2023.
  * [57] Ling Luo, Po-Ting Lai, Chih-Hsuan Wei, Cecilia N Arighi, and Zhiyong Lu.  BioRED: a rich biomedical relation extraction dataset.  Briefings in Bioinformatics, 23(5):bbac282, 07 2022.
  * [58] Renqian Luo, Liai Sun, Yingce Xia, Tao Qin, Sheng Zhang, Hoifung Poon, and Tie-Yan Liu.  Biogpt: generative pre-trained transformer for biomedical text generation and mining.  Briefings in Bioinformatics, 23(6):bbac409, 2022.
  * [59] Ali Madani, Ben Krause, Eric R Greene, Subu Subramanian, Benjamin P Mohr, James M Holton, Jose Luis Olmos Jr, Caiming Xiong, Zachary Z Sun, Richard Socher, et al.  Large language models generate functional protein sequences across diverse families.  Nature Biotechnology, pages 1–8, 2023.
  * [60] Joseph G Makin, David A Moses, and Edward F Chang.  Machine translation of cortical activity to text with an encoder–decoder framework.  Nature neuroscience, 23(4):575–582, 2020.
  * [61] Rahul Mishra, Krishan Sharma, Ranjeet Ranjan Jha, and Arnav Bhavsar.  Neurogan: image reconstruction from EEG signals via an attention-based GAN.  Neural Comput. Appl., 35(12):9181–9192, 2023.
  * [62] David A Moses, Sean L Metzger, Jessie R Liu, Gopala K Anumanchipalli, Joseph G Makin, Pengfei F Sun, Josh Chartier, Maximilian E Dougherty, Patricia M Liu, Gary M Abrams, et al.  Neuroprosthesis for decoding speech in a paralyzed person with anarthria.  New England Journal of Medicine, 385(3):217–227, 2021.
  * [63] Eric Nguyen, Michael Poli, Marjan Faizi, Armin Thomas, Callum Birch-Sykes, Michael Wornow, Aman Patel, Clayton Rabideau, Stefano Massaroli, Yoshua Bengio, et al.  Hyenadna: Long-range genomic sequence modeling at single nucleotide resolution.  arXiv preprint arXiv:2306.15794, 2023.
  * [64] Nicolás Nieto, Victoria Peterson, Hugo Leonardo Rufiner, Juan Esteban Kamienkowski, and Ruben Spies.  Thinking out loud, an open-access eeg-based bci dataset for inner speech recognition.  Scientific Data, 9(1):52, 2022.
  * [65] Erik Nijkamp, Jeffrey A Ruffolo, Eli N Weinstein, Nikhil Naik, and Ali Madani.  Progen2: exploring the boundaries of protein language models.  Cell systems, 14(11):968–978, 2023.
  * [66] Sergey Nurk, Sergey Koren, Arang Rhie, Mikko Rautiainen, Andrey V Bzikadze, Alla Mikheenko, Mitchell R Vollger, Nicolas Altemose, Lev Uralsky, Ariel Gershman, and et al.  The complete sequence of a human genome.  Science, 376(6588):44–53, April 2022.
  * [67] Jerrin Thomas Panachakel and Angarai Ganesan Ramakrishnan.  Decoding covert speech from eeg-a comprehensive review.  Frontiers in Neuroscience, 15:392, 2021.
  * [68] Cheng Peng, Xi Yang, Aokun Chen, Kaleb E Smith, Nima PourNejatian, Anthony B Costa, Cheryl Martin, Mona G Flores, Ying Zhang, Tanja Magoc, et al.  A study of generative large language model for medical research and healthcare.  arXiv preprint arXiv:2305.13523, 2023.
  * [69] Long N Phan, James T Anibal, Hieu Tran, Shaurya Chanana, Erol Bahadroglu, Alec Peltekian, and Grégoire Altan-Bonnet.  Scifive: a text-to-text transformer model for biomedical literature.  arXiv preprint arXiv:2106.03598, 2021.
  * [70] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al.  Language models are unsupervised multitask learners.  OpenAI blog, 1(8):9, 2019.
  * [71] Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu.  Exploring the limits of transfer learning with a unified text-to-text transformer.  The Journal of Machine Learning Research, 21(1):5485–5551, 2020.
  * [72] Pranav Rajpurkar, Ewen Chen, Oishi Banerjee, and Eric J Topol.  Ai in health and medicine.  Nat Med, 28(1):31–38, 2022.
  * [73] Alexander Rives, Joshua Meier, Tom Sercu, Siddharth Goyal, Zeming Lin, Jason Liu, Demi Guo, Myle Ott, C Lawrence Zitnick, Jerry Ma, et al.  Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences.  Proceedings of the National Academy of Sciences, 118(15):e2016239118, 2021.
  * [74] Yutaka Saito, Kengo Sato, and Yasubumi Sakakibara.  Fast and accurate clustering of noncoding rnas using ensembles of sequence alignments and secondary structures.  BMC bioinformatics, 12:1–8, 2011.
  * [75] Golnar Sheikhshab, Inanç Birol, and Anoop Sarkar.  In-domain context-aware token embeddings improve biomedical named entity recognition.  In Louhi@EMNLP, 2018.
  * [76] Hoo-Chang Shin, Yang Zhang, Evelina Bakhturina, Raul Puri, Mostofa Patwary, Mohammad Shoeybi, and Raghav Mani.  Biomegatron: Larger biomedical domain language model.  In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4700–4706, 2020.
  * [77] Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper, and Bryan Catanzaro.  Megatron-lm: Training multi-billion parameter language models using model parallelism.  arXiv preprint arXiv:1909.08053, 2019.
  * [78] Prajwal Singh, Pankaj Pandey, Krishna P. Miyapuram, and Shanmuganathan Raman.  EEG2IMAGE: image reconstruction from EEG brain signals.  CoRR, abs/2302.10121, 2023.
  * [79] Karan Singhal, Shekoofeh Azizi, Tao Tu, S Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, et al.  Large language models encode clinical knowledge.  arXiv preprint arXiv:2212.13138, 2022.
  * [80] Karan Singhal, Tao Tu, Juraj Gottweis, Rory Sayres, Ellery Wulczyn, Le Hou, Kevin Clark, Stephen Pfohl, Heather Cole-Lewis, Darlene Neal, et al.  Towards expert-level medical question answering with large language models.  arXiv preprint arXiv:2305.09617, 2023.
  * [81] Jose Roberto Ayala Solares, Yajie Zhu, Abdelaali Hassaine, Shishir Rao, Yikuan Li, Mohammad Mamouei, Dexter Canoy, Kazem Rahimi, and Gholamreza Salimi-Khorshidi.  Transfer learning in electronic health records through clinical concept embedding, 2021.
  * [82] Yonghao Song, Bingchuan Liu, Xiang Li, Nanlin Shi, Yijun Wang, and Xiaorong Gao.  Decoding natural images from EEG for object recognition.  In The Twelfth International Conference on Learning Representations, 2024.
  * [83] Jingyuan Sun, Shaonan Wang, Jiajun Zhang, and Chengqing Zong.  Towards sentence-level brain decoding with distributed representations.  In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 7047–7054, 2019.
  * [84] Baris E Suzek, Hongzhan Huang, Peter McGarvey, Raja Mazumder, and Cathy H Wu.  Uniref: comprehensive and non-redundant uniprot reference clusters.  Bioinformatics, 23(10):1282–1288, 2007.
  * [85] Baris E Suzek, Yuqi Wang, Hongzhan Huang, Peter B McGarvey, Cathy H Wu, and UniProt Consortium.  Uniref clusters: a comprehensive and scalable alternative for improving sequence similarity searches.  Bioinformatics, 31(6):926–932, 2015.
  * [86] Jerry Tang, Amanda LeBel, Shailee Jain, and Alexander G Huth.  Semantic reconstruction of continuous language from non-invasive brain recordings.  Nature Neuroscience, pages 1–9, 2023.
  * [87] Xiangru Tang, Anni Zou, Zhuosheng Zhang, Ziming Li, Yilun Zhao, Xingyao Zhang, Arman Cohan, and Mark Gerstein.  Medagents: Large language models as collaborators for zero-shot medical reasoning, 2024.
  * [88] Ross Taylor, Marcin Kardas, Guillem Cucurull, Thomas Scialom, Anthony Hartshorn, Elvis Saravia, Andrew Poulton, Viktor Kerkez, and Robert Stojnic.  Galactica: A large language model for science.  arXiv preprint arXiv:2211.09085, 2022.
  * [89] Christina V Theodoris, Ling Xiao, Anant Chopra, Mark D Chaffin, Zeina R Al Sayed, Matthew C Hill, Helene Mantineo, Elizabeth M Brydon, Zexian Zeng, X Shirley Liu, et al.  Transfer learning enables predictions in network biology.  Nature, 618(7965):616–624, 2023.
  * [90] Ajan J Thirunavukarasu, Daniel S J Ting, Karthikeyan Elangovan, Livia Gutierrez, Tien F Tan, and Daniel S W Ting.  Large language models in medicine.  Nat Med, 29(8):1930–1940, Aug 2023.  Epub 2023 Jul 17.
  * [91] Benyou Wang, Qianqian Xie, Jiahuan Pei, Zhihong Chen, Prayag Tiwari, Zhao Li, and Jie Fu.  Pre-trained language models in biomedical domain: A systematic survey.  ACM Computing Surveys, 2021.
  * [92] Christopher Wang, Vighnesh Subramaniam, Adam Uri Yaari, Gabriel Kreiman, Boris Katz, Ignacio Cases, and Andrei Barbu.  Brainbert: Self-supervised representation learning for intracranial recordings.  In The Eleventh International Conference on Learning Representations, 2022.
  * [93] Sheng Wang, Zihao Zhao, Xi Ouyang, Qian Wang, and Dinggang Shen.  Chatcad: Interactive computer-aided diagnosis on medical image using large language models, 2023.
  * [94] Xuan Wang, Vivian Hu, Minhao Jiang, Yu Zhang, Jinfeng Xiao, Danielle Cherrice Loving, Heng Ji, Martin Burke, and Jiawei Han.  Reactclass: Cross-modal supervision for subword-guided reactant entity classification.  In 2022 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pages 844–847. IEEE, 2022.
  * [95] Xuan Wang, Vivian Hu, Xiangchen Song, Shweta Garg, Jinfeng Xiao, and Jiawei Han.  Chemner: fine-grained chemistry named entity recognition with ontology-guided distant supervision.  In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 2021.
  * [96] Zhenhailong Wang and Heng Ji.  Open vocabulary electroencephalography-to-text decoding and zero-shot sentiment classification.  In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 5350–5358, 2022.
  * [97] Chaoyi Wu, Jiayu Lei, Qiaoyu Zheng, Weike Zhao, Weixiong Lin, Xiaoman Zhang, Xiao Zhou, Ziheng Zhao, Ya Zhang, Yanfeng Wang, and Weidi Xie.  Can gpt-4v(ision) serve medical applications? case studies on gpt-4v for multimodal medical diagnosis, 2023.
  * [98] Chaoyi Wu, Xiaoman Zhang, Ya Zhang, Yanfeng Wang, and Weidi Xie.  Towards generalist foundation model for radiology by leveraging web-scale 2d&3d medical data, 2023.
  * [99] Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, et al.  Google’s neural machine translation system: Bridging the gap between human and machine translation.  arXiv preprint arXiv:1609.08144, 2016.
  * [100] Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong, Ming Zhang, Junzhe Wang, Senjie Jin, Enyu Zhou, Rui Zheng, Xiaoran Fan, Xiao Wang, Limao Xiong, Yuhao Zhou, Weiran Wang, Changhao Jiang, Yicheng Zou, Xiangyang Liu, Zhangyue Yin, Shihan Dou, Rongxiang Weng, Wensen Cheng, Qi Zhang, Wenjuan Qin, Yongyan Zheng, Xipeng Qiu, Xuanjing Huang, and Tao Gui.  The rise and potential of large language model based agents: A survey, 2023.
  * [101] Minghao Xu, Xinyu Yuan, Santiago Miret, and Jian Tang.  Protst: Multi-modality learning of protein sequences and biomedical texts.  arXiv preprint arXiv:2301.12040, 2023.
  * [102] Keisuke Yamada and Michiaki Hamada.  Prediction of rna–protein interactions using a nucleotide language model.  Bioinformatics Advances, 2(1):vbac023, 2022.
  * [103] Fan Yang, Wenchuan Wang, Fang Wang, Yuan Fang, Duyu Tang, Junzhou Huang, Hui Lu, and Jianhua Yao.  scbert as a large-scale pretrained deep language model for cell type annotation of single-cell rna-seq data.  Nature Machine Intelligence, 4(10):852–866, 2022.
  * [104] Jingfeng Yang, Hongye Jin, Ruixiang Tang, Xiaotian Han, Qizhang Feng, Haoming Jiang, Bing Yin, and Xia Hu.  Harnessing the power of llms in practice: A survey on chatgpt and beyond.  arXiv preprint arXiv:2304.13712, 2023.
  * [105] Xi Yang, Aokun Chen, Nima PourNejatian, Hoo Chang Shin, Kaleb E Smith, Christopher Parisien, Colin Compas, Cheryl Martin, Anthony B Costa, Mona G Flores, et al.  A large language model for electronic health records.  NPJ digital medicine, 5(1):194, 2022.
  * [106] Michihiro Yasunaga, Jure Leskovec, and Percy Liang.  Linkbert: Pretraining language models with document links.  In Association for Computational Linguistics (ACL), 2022.
  * [107] Ke Yi, Yansen Wang, Kan Ren, and Dongsheng Li.  Learning topology-agnostic eeg representations with geometry-aware modeling.  In Thirty-seventh Conference on Neural Information Processing Systems, 2023.
  * [108] Xuan Zhang, Limei Wang, Jacob Helwig, Youzhi Luo, Cong Fu, Yaochen Xie, Meng Liu, Yuchao Lin, Zhao Xu, Keqiang Yan, et al.  Artificial intelligence for science in quantum, atomistic, and continuum systems.  arXiv preprint arXiv:2307.08423, 2023.
  * [109] Yikun Zhang, Mei Lang, Jiuhong Jiang, Zhiqiang Gao, Fan Xu, Thomas Litfin, Ke Chen, Jaswinder Singh, Xiansong Huang, Guoli Song, et al.  Multiple sequence-alignment-based rna language model and its application to structural inference.  bioRxiv, pages 2023–03, 2023.
  * [110] Wei-Long Zheng, Wei Liu, Yifei Lu, Bao-Liang Lu, and Andrzej Cichocki.  Emotionmeter: A multimodal framework for recognizing human emotions.  IEEE transactions on cybernetics, 49(3):1110–1122, 2018.
  * [111] Wei-Long Zheng and Bao-Liang Lu.  Investigating critical frequency bands and channels for eeg-based emotion recognition with deep neural networks.  IEEE Transactions on autonomous mental development, 7(3):162–175, 2015.
  * [112] Zaixiang Zheng, Yifan Deng, Dongyu Xue, Yi Zhou, Fei Ye, and Quanquan Gu.  Structure-informed language models are protein designers.  bioRxiv, pages 2023–02, 2023.
  * [113] Ming Zhong, Siru Ouyang, Minhao Jiang, Vivian Hu, Yizhu Jiao, Xuan Wang, and Jiawei Han.  Reactie: Enhancing chemical reaction extraction with weak supervision.  In Findings of the Association for Computational Linguistics: ACL 2023, pages 12120–12130, 2023.
  * [114] Jian Zhou and Olga G Troyanskaya.  Predicting effects of noncoding variants with deep learning-based sequence model.  Nat. Methods, 12(10):931–934, October 2015.
  * [115] Henghui Zhu, Ioannis Ch. Paschalidis, and Amir M. Tahmasebi.  Clinical concept extraction with contextual word embedding.  ArXiv, abs/1810.10566, 2018.
  * [116] Maxim Zvyagin, Alexander Brace, Kyle Hippe, Yuntian Deng, Bin Zhang, Cindy Orozco Bohorquez, Austin Clyde, Bharat Kale, Danilo Perez-Rivera, Heng Ma, et al.  Genslms: Genome-scale language models reveal sars-cov-2 evolutionary dynamics.  bioRxiv, pages 2022–10, 2022.


* [72] Bernhard, J. E., Moreland, J. S., Bass, S. A.  “Alternative ansatz to wounded nucleon and binary collision scaling in high-energy nuclear collisions”  In _Phys. Rev. C_ 92 American Physical Society, 2015, pp. 011901  DOI: 10.1103/PhysRevC.92.011901
  * [73] Bernhard, J. E., Moreland, J. S., Bass, S. A.  “Reduced Thickness Event-by-event Nuclear Topology (TRENTo) official website”  URL: http://qcd.phy.duke.edu/trento/
  * [74] S. Acharya  “Charged-particle multiplicity distributions over a wide pseudorapidity range in proton-proton collisions at $\sqrt{s}=$ 0.9, 7, and 8 TeV”  In _Eur. Phys. J. C_ 77.12, 2017, pp. 852  DOI: 10.1140/epjc/s10052-017-5412-6
  * [75] Aleksi Kurkela et al.  “Effective kinetic description of event-by-event pre-equilibrium dynamics in high-energy heavy-ion collisions”  In _Phys. Rev._ C99.3, 2019, pp. 034910  DOI: 10.1103/PhysRevC.99.034910
  * [76] Aleksi Kurkela et al.  “2+1D simulations of pre-equilibrium stage with QCD kinetic theory”  In _PoS_ HardProbes2018, 2018, pp. 111  DOI: 10.22323/1.345.0111
  * [77] Peter Brockway Arnold, Guy D. Moore and Laurence G. Yaffe  “Effective kinetic theory for high temperature gauge theories”  In _JHEP_ 01, 2003, pp. 030  DOI: 10.1088/1126-6708/2003/01/030
  * [78] Ulrich W. Heinz  “Early collective expansion: Relativistic hydrodynamics and the transport properties of QCD matter”  In _Landolt-Bornstein_ 23, 2010, pp. 240  DOI: 10.1007/978-3-642-01539-7˙9
  * [79] Charles Gale, Sangyong Jeon and Bjoern Schenke  “Hydrodynamic Modeling of Heavy-Ion Collisions”  In _Int. J. Mod. Phys. A_ 28, 2013, pp. 1340011  DOI: 10.1142/S0217751X13400113
  * [80] Gabriel Denicol al.  “A (3+1)D hydrodynamic code for heavy-ion collisions (MUSIC) official website” http://www.physics.mcgill.ca/music/
  * [81] G.. Denicol, H. Niemi, E. Molnar and D.. Rischke  “Derivation of transient relativistic fluid dynamics from the Boltzmann equation” [Erratum: Phys.Rev.D 91, 039902 (2015)]  In _Phys. Rev. D_ 85, 2012, pp. 114047  DOI: 10.1103/PhysRevD.85.114047
  * [82] E. Molnár, H. Niemi, G.. Denicol and D.. Rischke  “Relative importance of second-order terms in relativistic dissipative fluid dynamics”  In _Phys. Rev. D_ 89.7, 2014, pp. 074010  DOI: 10.1103/PhysRevD.89.074010
  * [83] Pasi Huovinen and Hannah Petersen  “Particlization in hybrid models”  In _Eur. Phys. J. A_ 48, 2012, pp. 171  DOI: 10.1140/epja/i2012-12171-9
  * [84] Fred Cooper and Graham Frye  “Single-particle distribution in the hydrodynamic and statistical thermodynamic models of multiparticle production”  In _Phys. Rev. D_ 10 American Physical Society, 1974, pp. 186–189  DOI: 10.1103/PhysRevD.10.186
  * [85] J.L. Anderson and H.R. Witting  “A relativistic relaxation-time model for the Boltzmann equation”  In _Physica_ 74.3, 1974, pp. 466–488  DOI: https://doi.org/10.1016/0031-8914(74)90355-3
  * [86] Amaresh Jaiswal, Radoslaw Ryblewski and Michael Strickland  “Transport coefficients for bulk viscous evolution in the relaxation-time approximation”  In _Phys. Rev. C_ 90 American Physical Society, 2014, pp. 044908  DOI: 10.1103/PhysRevC.90.044908
  * [87] Scott Pratt and Giorgio Torrieri  “Coupling relativistic viscous hydrodynamics to Boltzmann descriptions”  In _Phys. Rev. C_ 82 American Physical Society, 2010, pp. 044901  DOI: 10.1103/PhysRevC.82.044901
  * [88] Piotr Bozek  “Bulk and shear viscosities of matter created in relativistic heavy-ion collisions”  In _Phys. Rev. C_ 81, 2010, pp. 034909  DOI: 10.1103/PhysRevC.81.034909
  * [89] Kevin Dusling and Thomas Schäfer  “Bulk viscosity, particle spectra and flow in heavy-ion collisions”  In _Phys. Rev. C_ 85, 2012, pp. 044909  DOI: 10.1103/PhysRevC.85.044909
  * [90] Derek Teaney  “The Effects of viscosity on spectra, elliptic flow, and HBT radii”  In _Phys. Rev. C_ 68, 2003, pp. 034913  DOI: 10.1103/PhysRevC.68.034913
  * [91] Chun Shen et al.  “The iEBE-VISHNU code package for relativistic heavy-ion collisions”  In _Computer Physics Communications_ 199, 2016, pp. 61–85  DOI: https://doi.org/10.1016/j.cpc.2015.08.039
  * [92] Chun Shen  “iSS, Monte Carlo sampler for particle distribution from Cooper-Frye freeze-out procedure” https://github.com/chunshen1987/iSS
  * [93] S.. Bass  “Microscopic models for ultrarelativistic heavy ion collisions”  In _Prog. Part. Nucl. Phys._ 41, 1998, pp. 255–369  DOI: 10.1016/S0146-6410(98)00058-1
  * [94] M. Bleicher  “Relativistic hadron hadron collisions in the ultrarelativistic quantum molecular dynamics model”  In _J. Phys. G_ 25, 1999, pp. 1859–1896  DOI: 10.1088/0954-3899/25/9/308
  * [95] Frankfurt Institute for Advanced Studies  “Ultrarelativistic Quantum Molecular Dynamics (UrQMD) official website”  URL: https://urqmd.org/
  * [96] R.. McAllister and R. Hofstadter  “Elastic Scattering of 188-Mev Electrons from the Proton and the Alpha Particle”  In _Phys. Rev._ 102 American Physical Society, 1956, pp. 851–856  DOI: 10.1103/PhysRev.102.851
  * [97] R. Hofstadter  “Nuclear and nucleon scattering of high-energy electrons”  In _Ann. Rev. Nucl. Part. Sci._ 7, 1957, pp. 231–316  DOI: 10.1146/annurev.ns.07.120157.001311
  * [98] Robert Karplus, Abraham Klein and Julian Schwinger  “Electrodynamic Displacement of Atomic Energy Levels. II. Lamb Shift”  In _Phys. Rev._ 86 American Physical Society, 1952, pp. 288–301  DOI: 10.1103/PhysRev.86.288
  * [99] Hans-Werner Hammer and Ulf-G Meißner  “The proton radius: From a puzzle to precision”  In _Sci. Bull._ 65, 2020, pp. 257–258  DOI: 10.1016/j.scib.2019.12.012
  * [100] Randolf Pohl, Ronald Gilman, Gerald A. Miller and Krzysztof Pachucki  “Muonic hydrogen and the proton radius puzzle”  In _Ann. Rev. Nucl. Part. Sci._ 63, 2013, pp. 175–204  DOI: 10.1146/annurev-nucl-102212-170627
  * [101] CODATA  “The NIST Reference on Constants, Units, and Uncertainty”  URL: https://physics.nist.gov/cgi-bin/cuu/Value?rp
  * [102] Hélène Fleurbaey et al.  “New Measurement of the $1S-3S$ Transition Frequency of Hydrogen: Contribution to the Proton Charge Radius Puzzle”  In _Phys. Rev. Lett._ 120.18, 2018, pp. 183001  DOI: 10.1103/PhysRevLett.120.183001
  * [103] Axel Beyer  “The Rydberg constant and proton size from atomic hydrogen”  In _Science_ 358.6359, 2017, pp. 79–85  DOI: 10.1126/science.aah6677
  * [104] N. Bezginov et al.  “A measurement of the atomic hydrogen Lamb shift and the proton charge radius”  In _Science_ 365.6457, 2019, pp. 1007–1012  DOI: 10.1126/science.aau7807
  * [105] W. Xiong  “A small proton charge radius from an electron–proton scattering experiment”  In _Nature_ 575.7781, 2019, pp. 147–150  DOI: 10.1038/s41586-019-1721-2
  * [106] A. Caldwell and H. Kowalski  “Investigating the gluonic structure of nuclei via $J/\psi$ scattering”  In _Phys. Rev. C_ 81 American Physical Society, 2010, pp. 025203  DOI: 10.1103/PhysRevC.81.025203
  * [107] Dmitri E. Kharzeev  “Mass radius of the proton”  In _Phys. Rev. D_ 104.5, 2021, pp. 054015  DOI: 10.1103/PhysRevD.104.054015
  * [108] “First Measurement of Near-Threshold $J/\psi$ Exclusive Photoproduction off the Proton”  In _Phys. Rev. Lett._ 123 American Physical Society, 2019, pp. 072001  DOI: 10.1103/PhysRevLett.123.072001
  * [109] W.. Hastings  “Monte Carlo sampling methods using Markov chains and their applications”  In _Biometrika_ 57.1, 1970, pp. 97–109  DOI: 10.1093/biomet/57.1.97
  * [110] Jonah E. Bernhard, J. Moreland and Steffen A. Bass  “Bayesian estimation of the specific shear and bulk viscosity of quark–gluon plasma”  In _Nature Phys._ 15.11, 2019, pp. 1113–1117  DOI: 10.1038/s41567-019-0611-8
  * [111] B. Alver  “System size, energy, pseudorapidity, and centrality dependence of elliptic flow”  In _Phys. Rev. Lett._ 98, 2007, pp. 242302  DOI: 10.1103/PhysRevLett.98.242302
  * [112] Jonah E. Bernhard et al.  “Applying Bayesian parameter estimation to relativistic heavy-ion collisions: simultaneous characterization of the initial state and quark-gluon plasma medium”  In _Phys. Rev. C_ 94.2, 2016, pp. 024907  DOI: 10.1103/PhysRevC.94.024907
  * [113] Bjoern Schenke, Chun Shen and Prithwish Tribedy  “Running the gamut of high energy nuclear collisions”  In _Phys. Rev. C_ 102.4, 2020, pp. 044905  DOI: 10.1103/PhysRevC.102.044905
  * [114] D. Everett  “Multisystem Bayesian constraints on the transport coefficients of QCD matter”  In _Phys. Rev. C_ 103.5, 2021, pp. 054904  DOI: 10.1103/PhysRevC.103.054904
  * [115] Govert Nijs, Wilke Schee, Umut Gürsoy and Raimond Snellings  “Transverse Momentum Differential Global Analysis of Heavy-Ion Collisions”  In _Phys. Rev. Lett._ 126.20, 2021, pp. 202301  DOI: 10.1103/PhysRevLett.126.202301
  * [116] Govert Nijs, Wilke Schee, Umut Gürsoy and Raimond Snellings  “Bayesian analysis of heavy ion collisions with the heavy ion computational framework Trajectum”  In _Phys. Rev. C_ 103.5, 2021, pp. 054909  DOI: 10.1103/PhysRevC.103.054909
  * [117] Govert Nijs and Wilke Schee  “Predictions and postdictions for relativistic lead and oxygen collisions with $Trajectum$”, 2021  arXiv:2110.13153 [nucl-th]
  * [118] R… Andrade et al.  “Fluctuations and initial state granularity in heavy ion collisions and their effects on observables from hydrodynamics”  In _Acta Phys. Polon. B_ 40, 2009, pp. 993–998  arXiv:0812.4143 [nucl-th]
  * [119] Tiago Nunes da Silva et al.  “Pre-hydrodynamic evolution and its signatures in final-state heavy-ion observables”  In _Phys. Rev. C_ 103, 2021, pp. 054906  DOI: 10.1103/PhysRevC.103.054906
  * [120] Charles Gale, Jean-François Paquet, Björn Schenke and Chun Shen  “Multimessenger heavy-ion collision physics”  In _Phys. Rev. C_ 105.1, 2022, pp. 014909  DOI: 10.1103/PhysRevC.105.014909
  * [121] Nicolas Borghini, Phuong Mai Dinh and Jean-Yves Ollitrault  “A New method for measuring azimuthal distributions in nucleus-nucleus collisions”  In _Phys. Rev. C_ 63, 2001, pp. 054906  DOI: 10.1103/PhysRevC.63.054906
  * [122] Nicolas Borghini, Phuong Mai Dinh and Jean-Yves Ollitrault  “Flow analysis from multiparticle azimuthal correlations”  In _Phys. Rev. C_ 64, 2001, pp. 054901  DOI: 10.1103/PhysRevC.64.054901
  * [123] Ante Bilandzic, Raimond Snellings and Sergei Voloshin  “Flow analysis with cumulants: Direct calculations”  In _Phys. Rev. C_ 83, 2011, pp. 044913  DOI: 10.1103/PhysRevC.83.044913
  * [124] Giuliano Giacalone, Björn Schenke and Chun Shen  “Constraining the Nucleon Size with Relativistic Nuclear Collisions”  In _Phys. Rev. Lett._ 128.4, 2022, pp. 042301  DOI: 10.1103/PhysRevLett.128.042301
  * [125] Govert Nijs and Wilke Schee  “The hadronic nucleus-nucleus cross section and the nucleon size”, 2022  arXiv:2206.13522 [nucl-th]