tcb@breakable

# Large Language Model (LLM) for Telecommunications: A Comprehensive Survey on
Principles, Key Techniques, and Opportunities ††thanks:  Hao Zhou, Chengming
Hu, Ye Yuan, Yufei Cui, Yili Jin, Can Chen, Haolun Wu, Dun Yuan, Li Jiang, and
Xue Liu are with the School of Computer Science, McGill University, Montreal,
QC H3A 0E9, Canada. (emails:{hao.zhou4, chengming.hu, ye.yuan3, yufei.cui,
yili.jin, can.chen, haolun.wu, dun.yuan<EMAIL_ADDRESS>xue.liu@cs.mcgill.ca); Di Wu is with the School of Electrical and Computer
Engineering, McGill University, Montreal, QC H3A 0E9, Canada. (email:
di.wu5@mcgill.ca); Charlie Zhang is with Samsung Research America, Plano,
Texas, TX 75023, USA. (email: jianzhong.z@samsung.com); Xianbin Wang is with
the Department of Electrical and Computer Engineering, Western University,
London, ON N6A 3K7, Canada. (e-mail: xianbin.wang@uwo.ca); Jiangchuan Liu is
with the School of Computing Science, Simon Fraser University, Burnaby, BC V5A
1S6, Canada. (e-mail: jcliu@sfu.ca).

Hao Zhou, Chengming Hu, Ye Yuan, Yufei Cui, Yili Jin,
Can Chen, Haolun Wu, Dun Yuan, Li Jiang, Di Wu, Xue Liu, ,
Charlie Zhang, , Xianbin Wang, , Jiangchuan Liu

###### Abstract

Large language models (LLMs) have received considerable attention recently due
to their outstanding comprehension and reasoning capabilities, leading to
great progress in many fields. The advancement of LLM techniques also offers
promising opportunities to automate many tasks in the telecommunication
(telecom) field. After pre-training and fine-tuning, LLMs can perform diverse
downstream tasks based on human instructions, paving the way to artificial
general intelligence (AGI)-enabled 6G. Given the great potential of LLM
technologies, this work aims to provide a comprehensive overview of LLM-
enabled telecom networks. In particular, we first present LLM fundamentals,
including model architecture, pre-training, fine-tuning, inference and
utilization, model evaluation, and telecom deployment. Then, we introduce LLM-
enabled key techniques and telecom applications in terms of generation,
classification, optimization, and prediction problems. Specifically, the LLM-
enabled generation applications include telecom domain knowledge, code, and
network configuration generation. After that, the LLM-based classification
applications involve network security, text, image, and traffic classification
problems. Moreover, multiple LLM-enabled optimization techniques are
introduced, such as automated reward function design for reinforcement
learning and verbal reinforcement learning. Furthermore, for LLM-aided
prediction problems, we discussed time-series prediction models and multi-
modality prediction problems for telecom. Finally, we highlight the challenges
and identify the future directions of LLM-enabled telecom networks.

###### Index Terms:

Large language model, telecommunications, generation, classification,
prediction, optimization.

Figure 1: Organization and key topics covered in this work.

##  I Introduction

While 5G networks have entered the commercial deployment stage, the academic
community has started the exploration of envisioned 6G networks. In
particular, 6G networks are expected to achieve terabits per second (Tbps)
level data rates, $10^{7}/km^{2}$ connection densities, and lower than 0.1
$ms$ latency[1]. To achieve these goals, many novel communication techniques
are proposed, such as reconfigurable intelligence surfaces (RISs), integrated
sensing and communication, mmWave and THz networks, vehicle-to-everything
networks, and so on[2]. These novel techniques have shown satisfactory
performance towards 6G requirements, but the complexity of network management
also significantly increased. From 3G, 4G LTE to 5G and envisioned 6G
networks, telecommunication (telecom) networks have become a complicated
large-scale system, including core networks, transport networks, network edge,
and radio access networks[3], and the telecom-domain tasks involve network
configuration and troubleshooting, product design and coding, standard
specification development, performance optimization and prediction, etc. To
handle such complexity, machine learning (ML) has become one of the most
promising solutions, and there have been a large number of studies on
artificial intelligence (AI)/ML-enabled wireless networks, e.g., reinforcement
learning-based network management[4], deep neural network-enabled channel
state information (CSI) prediction[5], and federated learning for distributed
model training in wireless environments[6]. For example, convex optimization
has been applied to optimize network performance, but it requires problem-
specific transformation for convexity. By contrast, reinforcement learning
will transform the problem into a unified Markov decision process (MDP), and
then interact with the environment to explore optimal policies. Compared with
conventional optimization algorithms[2], reinforcement learning overcomes the
complexity of dedicated problem reformulation, and can better handle
environmental uncertainties, e.g., the growing diversity of user preferences,
and more distributed and heterogeneous resources in future telecom networks.
These studies have demonstrated the importance of incorporating ML to improve
the efficiency, reliability, and quality of telecom services.

Recently, large language model (LLM) techniques have attracted considerable
interest from both academia and industry. Unlike previous ML algorithms, these
large-scale models with a huge amount of parameters have shown versatile
comprehension and reasoning capabilities in various fields such as health
care[7], law[8], finance[9], education, and so on [10]. For instance, Wu et
al. introduced a BloombergGPT model that is trained on a wide range of
financial data with 50 billion parameters, and the Med-PaLM2 developed by
Google achieves 86.5% correct rate on the medical question answering
dataset[7]. Existing studies have demonstrated that LLM technologies have many
promising features such as in-context learning (ICL), step-by-step learning,
and instruction following[11]. For instance, ICL means that the model can
learn from natural language instruction or task demonstration [12], and then
generate expected output rapidly without any extra model training. Such an
efficient learning approach can be attractive for the telecom domain, since
many wireless applications require rapid or even real-time responses.

Despite the great potential, LLM’s real-world application is still at a very
early stage, especially for domain-specific scenarios. For instance, telecom
is a broad field that includes various knowledge domains, e.g., signal
transmissions, protocols, network architectures, devices, and different
standards. LLM is expected to properly understand and generate content that
aligns with real-world details and specific requirements of telecom
applications[13]. However, such specific telecom-related requirements are rare
in the existing knowledge base of general-domain LLMs. Therefore, applying a
general-domain LLM directly to telecom tasks may lead to poor performance.
Meanwhile, fine-tuning LLM models on telecom datasets may improve LLM’s
performance of domain-specific tasks, but the telecom-specific dataset
collection and filtering still require careful design and evaluation. In
addition, many telecom tasks require multi-step planning and thinking, e.g., a
simple coding task can include multiple steps, indicating dedicated prompting
and analyses from the telecom perspective[14].

Given the above opportunities and challenges, this work presents a
comprehensive survey of LLM-enabled telecom networks. Different from existing
studies that focus on one specific aspect such as edge intelligence [15, 16],
grounding and alignment[17], this work provides a comprehensive survey on
fundamentals, key techniques, and applications of LLM-enabled telecom. To be
specific, this work focuses on generative models that were originally
developed for language tasks, i.e., language models, and it also involves more
diverse techniques and broad application scenarios such as optimization and
prediction problems. Although LLM development is originally motivated by
natural language tasks, it is worth noting that there have been diverse state-
of-the-art explorations that are beyond the conventional language processing
tasks, e.g., coding and debugging [18], recommendation [19], LLM-enabled
agents [20], instruction-based optimization [21], network time-series
prediction and decision making [22], etc. These LLM-inspired techniques have
become crucial pillars of LLM studies, and exploring these techniques is
crucial to take full advantage of LLM capabilities. Fig.1 presents the
organization of this work, in which the left side indicates the telecom
scenarios and demand, and the right side shows the LLM-enabled techniques111
On the high level, the terms “LLM” and “foundation model” can be used
interchangeably. In this survey, the term ”foundation models” refers to models
specifically developed from scratch for applications that are beyond pure
language-related tasks, such as the prediction foundation models in Section
VII-B, while ”LLM-enabled” or ”LLM-aided” approaches denote methods that
repurpose existing pre-trained language models for telecommunication tasks..
To better present the detailed application scenarios, the bottom of Fig.1
shows telecom environments that include radio access networks, network edge,
central cloud, and other network elements such as regular users, malicious
users, mmWave beam, environment image sensing, RISs, backhaul traffic, etc. In
particular, we focus on the following topics:

1) LLM fundamentals: Understanding LLM fundamentals is the prerequisite for
developing advanced applications in telecom networks[10]. Compared with
existing studies[15, 16, 17], this work presents a more comprehensive overview
of the model architecture, pre-training, fine-tuning, inference and
utilization, and evaluation. Additionally, it presents different approaches to
deploy LLMs in telecom networks, such as central cloud, network edge, and
mobile LLM [16]. It further analyzes LLM fundamentals from the telecom
application perspective, e.g., training or fine-tuning telecom-specific LLM
models, and the importance of prompting and multi-step planning techniques for
telecom tasks.

2) LLM for generation problems in telecom: Generating desired content is the
most common usage of LLM, and here we investigate the applications to specific
telecom scenarios. In particular, it involves answering telecom-domain
questions, generating troubleshooting reports, project coding, and network
configuration. It shows that LLM’s generation capabilities are particularly
useful in text and language-related telecom tasks to save human effort, e.g.,
automated code refactoring and design[14], recommending troubleshooting
solutions[23], and generating network configurations[24].

3) LLM-based classification for telecom: Classification is a common task in
the telecom field, and we present LLM-enabled network attack classification
and detection, telecom text, image, and traffic classification problems. For
instance, there have been many studies on visioned-aided blockage prediction
and beamforming for 6G networks [25], and some LLM can provide zero-shoot
image classification capabilities, overcoming the training difficulties of
conventional algorithms in complicated signal transmission environments [26].

4) LLM-enabled optimization techniques: Optimization techniques are of great
importance to telecom networks, e.g., resource allocation and load
balancing[2], and LLM offers new opportunities [2]. In particular, we
introduce LLM-aided automated reward function design for reinforcement
learning, verbal reinforcement learning, LLM-enabled black-box optimizer, end-
to-end convex optimization, and LLM-aided heuristic algorithm design. For
example, reinforcement learning has been widely used for network optimization,
but the reward functions are usually manually designed with a trial-and-error
approach[27]. LLM can provide automated reward function designs, and such an
improvement can significantly promote reinforcement learning applications in
the telecom field.

TABLE I: Comparison of this work with existing surveys

Ref. |  LLM fundamental techniques |  Generation applications |  Classification applications |  Optimization techniques |  Prediction techniques
---|---|---|---|---|---
|  Archit- ecture |  Pre-training |  Fine-tuning |  Infe- rence |  Evalu- ation |  Deploy- ment |  Question answering |  Troubles- hooting |  Coding |  Network config. |  Network attacks |  Text |  Image |  Network traffic |  LLM -aided RL |  Black- box |  Convex |  Heuristic |  Time series LLM |  Multi- modality
[15] |  |  ✓ |  |  |  |  ✓ |  |  |  |  |  |  |  |  |  |  |  |  |  |
[16] |  |  ✓ |  |  |  |  ✓ |  |  |  |  |  |  |  |  |  |  |  |  |  |
[17] |  |  |  |  |  |  ✓ |  |  |  |  |  |  |  |  |  |  |  |  |  |  ✓
[28] |  |  ✓ |  ✓ |  |  |  ✓ |  |  |  ✓ |  ✓ |  |  |  |  ✓ |  |  |  |  |  |
[29] |  |  ✓ |  |  |  |  ✓ |  |  |  |  |  |  |  |  |  |  |  |  |  |  ✓
[30] |  |  |  |  |  |  |  |  |  |  |  |  |  ✓ |  |  |  |  |  |  |  ✓
[31] |  |  ✓ |  ✓ |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  ✓
Our work |  ✓ |  ✓ |  ✓ |  ✓ |  ✓ |  ✓ |  ✓ |  ✓ |  ✓ |  ✓ |  ✓ |  ✓ |  ✓ |  ✓ |  ✓ |  ✓ |  ✓ |  ✓ |  ✓ |  ✓

  * 1

Multi-modality is discussed in several existing studies but not from the
prediction perspective. Table I divides key topics from LLM fundamentals to
optimization and prediction to better align with the organization of our work.

5) LLM-aided prediction in telecom: Prediction techniques are crucial for
telecom networks, such as CSI prediction[5], prediction-based beamforming[25],
and traffic load prediction[32]. Existing studies have started exploring one-
model-for-all time-series models. After pre-training on a large corpus of
diverse time-series data, such a model can learn the hidden temporal patterns,
and then generalize well across different prediction tasks without extra
training. We will first introduce how to pre-train foundation models, and then
present frozen pre-trained and fine-tune-based LLM models. In addition, the
potential of multi-modal LLM is discussed for telecom prediction tasks.

6) Challenges and future directions: Finally, we identify the challenges and
future directions of LLM-empowered telecom. The challenges focus on telecom-
domain LLM model training, practical LLM deployment, and prompt engineering
for telecom applications. The future directions include LLM-enabled planning,
model compression and fast inference, overcoming hallucination problems,
retrieval augmented-LLM, and economic and affordable LLM models.

In summary, the main contribution of this work is that we provide a
comprehensive survey of the principles, key techniques, and applications for
LLM-enabled telecom networks, ranging from LLM fundamentals to novel LLM-
inspired generation, classification, optimization and prediction techniques
along with telecom applications. This work covers nearly 20 telecom
application scenarios and LLM-inspired novel techniques, aiming to be a
roadmap for researchers to use LLMs to solve various telecom tasks. The rest
of this paper is organized as Fig. 1. Section II discusses related surveys,
and Section III presents LLM fundamentals. Section IV, V, VI, and VII focus on
generation, classification, optimization, and prediction problems and telecom
applications, respectively. Finally, Section VIII identifies the challenges
and future directions, and Section IX concludes this work.

##  II Related Surveys

Table I compares this work with existing studies [15, 16, 17, 28, 29, 30, 31],
including LLM fundamental techniques such as pre-training and fine-tuning, and
other key topics ranging from question answering to multi-modality. Firstly,
Table I shows that most existing studies focus on the fundamental techniques
of LLM models, e.g., pre-training LLM models for telecom tasks in general[15,
16]. Meanwhile, LLM deployment is also discussed in many existing studies,
including central cloud [15, 31], network edge [16], and mobile execution[17].
Due to the storage and computational resources constraint at the network edge,
Lin et al. also summarized various techniques in [16] that can be used to
improve the LLM training efficiency at the network edge, such as parameter-
efficient fine-tuning, split edge learning, and quantized training. In
addition, sensing has become an important part of future 6G networks, and the
multi-modal LLM are discussed in several existing studies, e.g., integrated
sensing and communication with LLM[17, 29], multi-modal input to LLM models
for intelligent sensing and communication[30], and multi-modal sensing[31].
These studies are very valuable explorations of LLM-enabled telecom networks
by focusing on model training and deployment. However, LLM techniques are
rapidly progressing and many LLM-inspired novel techniques and applications
have been recently proposed. This work is different from existing studies in
the following aspects:

1) In terms of LLM fundamentals, we provide comprehensive overviews and
analyses, ranging from model architecture and pre-training to LLM evaluation
and deployment. For instance, prompt engineering is of great importance for
using LLM technology, but some crucial techniques such as chain-of-
thought(CoT)[33] and step-by-step planning are not discussed in many existing
studies[15, 16, 17, 28, 29, 30, 31]. Understanding these prompt design skills
is the prerequisite for advanced telecom applications. By contrast, this work
provides detailed analyses of chain-of-thought along with telecom
applications, e.g., LLM-aided automated wireless project coding with multi-
step prompting and thinking[14]. Meanwhile, we also systemically analyzed the
features of different LLM deployment strategies in telecom, while existing
studies usually involve one single deployment[15, 31, 16, 17].

2) In terms of LLM-inspired techniques, this work presents the most state-of-
the-art novel algorithms and designs. For instance, reinforcement learning has
been widely applied to telecom optimization problems, but the reward function
design requires considerable human effort[27]. Existing studies have shown
that LLM can be used for automated reward function design, achieving a
comparable performance as human manual designs[34, 35, 36]. Such a technique
may bring revolutionary changes to reinforcement learning techniques, which
have great potential for telecom applications. In addition, time-series LLM is
also a promising technique for telecom, enabling one-model-for-all
prediction[37]. However, these novel techniques are not mentioned in most
existing studies.

TABLE II: Summary of existing general and domain-specific LLM models.

Domain | Model |  Size | Pre-train | Latest Update
---|---|---|---|---
General | GPT-4-Turbo |  - | - | Mar 2024
| Claude-3 Opus |  - |  - | Mar 2024
| Gemini-1 Ultra |  - | - | Dec 2023
| Mistral-Large |  - | - | Feb 2024
| Llama-2 |  70B | 10T tokens | Jul 2023
| Qwen-1.5 |  72B | - | Feb 2024
| DeepSeek |  67B | 2T tokens | Jan 2024
| Baichuan-2 Turbo |  13B | - | Sep 2023
Healthcare | MedGPT |  - | - | Jul 2021
| ChatDoctor |  7B | 100K | Jun 2023
| Med-PaLM |  540B | 760B | Dec 2022
Finance | finBERT |  110B | 1.8M | Aug 2019
| FinMA |  30B | 1T tokens | Jun 2023
| BloombergGPT |  50B | 569-770B tokens | Dec 2023
Time Series | TabLLM |  3B |  50,000 rows | Mar 2023
| LLMTime |  70B | - | Oct 2023
| TIME-LLM |  7B | - | Jan 2024
Autonomous |  |  |  |
driving | Driving with LLMs |  7B | 110k | Oct 2023
| Dilu |  - | - | Feb 2024
| DriveGPT4 |  70B | 112k | Mar 2024
Law | LexiLaw |  6B | - | May 2023
| JurisLMs |  13B | - | July 2023
| ChatLaw |  13B | 980k | July 2023
Recommen- |  |  |  |
dation | M6-Rec |  300M | 1G | May 2022
| TallRec |  7B |  100 samples | Oct 2023
| AgentCF |  175B | 20k samples | Oct 2023

3) In terms of telecom applications, we systematically summarize various LLM
application scenarios, including question answering, network troubleshooting,
coding, network configuration, network attack classification and security,
text and image classification, etc. Compared with existing studies, we
presented more comprehensive overviews and analyses of using LLM techniques to
solve various problems in the telecom domain. For each application, this work
provides technical details such as framework, pre-training steps, and prompt
designs, which are more informative than existing studies that focus on
general system-level designs.

Moreover, Table II summarizes some existing LLM models, including general-
domain LLM such as GPT-4 and Llama-2, and domain-specific LLM for
healthcare[38], finance[9], time series, autonomous driving[39], and
recommendation system[40]. Table II demonstrates that LLM techniques have
received considerable attention in many fields, and researchers have trained
various domain-specific LLM models for their application scenarios. Therefore,
given the rapid progress and great potential of LLM technologies, it calls for
a comprehensive investigation of how to apply LLM to the telecom field. To
this end, this work answers such a question: What are the most state-of-the-
art techniques inspired by LLM models, and how these techniques can be used to
solve telecom domain problems? The answer to this question is crucial for
building intelligent next-generation telecom networks in the 6G era.

##  III LLM Fundamentals

This Section will introduce LLM fundamentals, and the overall organization is
shown in Fig.2. It presents a thorough overview of LLM fundamentals, including
the model architecture, pre-training, fine-tuning, inference and utilization,
and model evaluation. We further discuss how LLM models can be deployed in
telecom networks such as central cloud, network edge, and mobile devices.
Finally, we analyze LLM fundamentals from the telecom application perspective,
e.g., training or fine-tuning LLM models for the telecom domain.

###  III-A Model Architecture

The fundamental component of contemporary LLM models is the transformer scheme
[41], which leverages an attention mechanism to capture global dependencies
between inputs and outputs. Transformers process raw inputs by tokenizing them
and applying embeddings and positional encodings. The vanilla transformer
architecture comprises two main components: the encoder and the decoder. The
encoder’s role is to extract features and understand the relationships among
all input tokens. It employs self-attention, also referred to as bidirectional
attention, allowing each token to attend to every other token in both
directions. Conversely, the decoder is responsible for producing the output
sequence while taking into account the input sequence and previously generated
tokens. It initially applies a masked attention mechanism, known as causal
attention, ensuring that the current token only attends to previously
generated tokens. Additionally, the decoder employs cross-attention, where the
query comes from the decoder, and the key and value are from the encoder,
enabling the decoder to integrate information from both the input sequence and
the already generated tokens. Beyond the basic version of the attention
mechanism, various variants are developed to capture the different
relationships among tokens, such as multi-head attention [41], multi-query
attention [42], and grouped-query attention [43]. Current architectures can be
classified into three distinct categories: encoder-only architecture, encoder-
decoder architecture, and decoder-only architecture.

####  III-A1 Encoder-only architecture

Models with an encoder-only structure solely comprise an encoder. These models
are tailored for language understanding tasks, where they extract language
features for downstream applications such as classification. A prominent
example is bidirectional encoder representations from transformers (BERT)
[44]. BERT is pre-trained with two main objectives: the masked language model
objective, which aims to reconstruct randomly masked tokens, and the next
sentence prediction objective, designed to ascertain if one sentence logically
follows another. There have been many variants of this model, such as RoBERTa
[45], which enhances the performance on downstream tasks222Here downstream
tasks refer to a series of target tasks that can be solved by the pre-trained
model, e.g., text classification, natural language inference., and ALBERT [46]
that introduces two parameter-reduction techniques to accelerate BERT’s
training process.

Figure 2: Organization and key topics of Section III.

####  III-A2 Encoder-decoder architecture

The foundational transformer block employs an encoder-decoder architecture,
wherein the encoder relays keys and values generated by its self-attention
module to the decoder for cross-attention processing. For example, the study
in [47] introduces the text-to-text transfer transformer, a unified framework
that reformulates all text-based language tasks into a text-to-text format,
thereby facilitating the exploration of transfer learning within natural
language processing (NLP). BART is another well-known model with standard
transformer architecture[48], which employs a denoising autoencoder approach
for pre-training sequence-to-sequence models. It introduces arbitrary noise
into text and is trained to reconstruct the original content, effectively
combining elements of BERT’s bidirectional encoding and GPT’s causal decoding
methodologies.

####  III-A3 Decoder-only architecture

Decoder-only architectures specialize in unidirectional attention, allowing
each output token to attend only to its past tokens and itself. Both prefix
and output tokens undergo identical processing within the decoder. Decoders
are further distinguished based on their attention mechanisms into causal and
non-causal decoders. In causal decoders, every token is restricted to
attending to its past tokens and itself; in non-causal decoders, prefix tokens
can attend to all tokens within the prefix. Causal decoders are predominantly
adopted in popular LLM models, such as the GPT series [12], PaLM [49], and
LLaMA [50]. Non-causal decoders [51] resemble encoder-decoder frameworks in
their ability to bidirectionally process the prefix sequence and
autoregressively generate output tokens sequentially.

###  III-B LLM Pre-training

The aim of pre-training language models is to predict the next word within a
sentence. After being trained on extensive datasets, LLM models exhibit
emergent capabilities in comprehension and reasoning. This subsection will
introduce dataset collection, preprocessing, and model training techniques.

####  III-B1 Dataset collection and preprocessing

Datasets for training language models fall into two primary categories:
general and specialized. General datasets comprise a diverse range of sources,
such as web pages, literature, and conversational corpora. For instance, web
pages like Wikipedia[52] can contribute to a language model’s broad linguistic
understanding. Meanwhile, literary works also serve as a rich reservoir of
formal and lengthy texts[53]. These materials are crucial for teaching LLM
models complex linguistic constructs, facilitating the modelling of long-range
dependencies. Specialized data involves scientific texts and programming-
related data. For example, scientific literature comprises a wealth of formal
writing imbued with domain-specific knowledge, encompassing academic papers
and textbooks. On the other hand, programming data drawn from online question-
answering platforms like Stack Exchange [54], along with public software
repositories such as GitHub, provide raw material rich with code snippets,
comments, and documentation. Incorporating these specialized texts into the
training of LLM models can significantly improve LLM’s performance in
reasoning and domain-specific knowledge applications. However, before pre-
training, it is critical to preprocess the collected datasets, which often
contain noisy, redundant, irrelevant, and potentially harmful data. The
preprocessing procedure may include quality filtering, de-duplication[49],
privacy redaction[55], and tokenization [56].

####  III-B2 Model training

In the model training process, two pivotal hyperparameters are the batch size
and the learning rate. For the pre-training of LLM models, a substantial batch
size is required, and recent studies suggest incrementally enlarging the batch
size to bolster the stability of the training process [49]. In terms of
learning rate adjustments, a widely used strategy is to start with a warm-up
phase and then succeed with a cosine decay pattern. This approach helps in
achieving a more controllable learning rate schedule. To enhance the training
scalability, several key techniques are proposed. For instance, 3D parallelism
encompasses data parallelism, pipeline parallelism, and tensor parallelism.
Data parallelism involves the replication of the model’s parameters and
optimizer states across multiple GPUs[57], allocating to each GPU a subset of
data to process and subsequently aggregate the computed gradients. Pipeline
parallelism, as detailed in [58], assigns distinct layers of an LLM to various
GPUs, allowing the accommodation of larger models within the confines of GPU
memory. Tensor parallelism operates on a similar premise by decomposing the
tensors[59], especially for the parameter matrices of LLM, facilitating the
distribution and computation across multiple GPUs. Meanwhile, ZeRO is also a
useful technique [60], which conserves memory by retaining only a portion of
the model’s data on each GPU. The remainder of the data is accessible across
the GPU network, as needed, effectively addressing memory redundancy concerns.

###  III-C LLM Fine-tuning

Fine-tuning refers to the process of updating the parameters of pre-trained
LLM models to adapt to domain-specific tasks. Although the pre-trained LLM
already has vast language knowledge, they lack specialization in specific
areas. Fine-tuning overcomes this limitation by allowing the model to learn
from domain-specific datasets, making the LLM more effective on specific
applications. This subsection will introduce two fine-tuning strategies:
instruction and alignment tuning.

####  III-C1 Instruction tuning

Instruction tuning is a method for fine-tuning pre-trained LLM models using a
collection of natural language-formatted instances. This technique aligns
closely with supervised fine-tuning and multi-task prompted training,
enhancing the LLM’s ability to generalize to unseen tasks, even in
multilingual contexts [61]. The process involves collecting or constructing
instruction-formatted instances and employing these to fine-tune LLM models in
a supervised manner, typically using sequence-to-sequence loss for training.
Models like InstructGPT and GPT-4 have demonstrated the effectiveness of
instruction tuning in meeting real user needs and improving task
generalization [62, 63]. Instruction-formatted instances usually consist of a
task description, an optional input, a corresponding output, and possibly a
few examples as demonstrations. These instances can originate from various
sources, such as traditional NLP task datasets, daily chat data, and synthetic
data. Existing research has reformatted traditional NLP datasets with natural
language task descriptions to aid LLM models in understanding tasks, proving
particularly effective in enhancing task generalization capabilities [61]. The
design and quality of instruction instances significantly will impact the
model’s performance. Scaling the instructions, for instance, tends to improve
generalization ability up to a certain point, beyond which additional tasks
may not yield further gains [64]. Diversity in task descriptions and the
number of instances per task are also critical, with a smaller number of high-
quality instances often sufficing for significant performance improvements
[61].

####  III-C2 Alignment tuning

Alignment tuning aims to ensure LLM models adhere to human values, preventing
outputs that could be harmful, biased, or misleading. This concept emerges
from the realization that while the LLM excels in various NLP tasks, they may
inadvertently generate content that deviates from ethical norms or human
expectations [12]. Collecting human feedback is central to the alignment-
tuning process. In particular, it involves curating responses from diverse
human labellers to guide the LLM toward generating outputs that align with the
predefined criteria. Approaches to collecting this feedback include ranking-
based methods, where labellers evaluate the quality of model-generated
outputs, and question-based methods, where labellers provide insights on
specific aspects of the outputs, such as their ethical implications [65].

A prominent technique in alignment tuning is reinforcement learning from human
feedback (RLHF), where the model is fine-tuned using reinforcement learning
algorithms based on human feedback. This process typically starts with
supervised fine-tuning using human-annotated data, followed by training a
reward model that reflects human preferences, and finally, fine-tuning the LLM
using this reward model. Despite its effectiveness, RLHF can be
computationally intensive and complex, necessitating alternative approaches
for practical applications [66]. An alternative method for RLHF is direct
optimization through supervised learning, which bypasses the complexities of
reinforcement learning. This method relies on constructing a high-quality
alignment dataset and directly fine-tuning LLM models to adhere to alignment
criteria. Although less resource-intensive than RLHF, this approach requires
careful dataset construction and may not capture the full range of human
values and preferences as effectively as RLHF [67].

###  III-D LLM Inference and Utilization by Prompting

Prompt engineering is the process in which users design various inputs for AI
models to generate desired outputs. Compared with fine-tuning, prompting has
no requirements for extra training, producing output instantly based on user
inputs. It indicates a straightforward approach to using LLM models, and the
rapid response and training-free features make it a promising method for
telecom applications. This subsection will introduce key techniques in prompt
engineering, including ICL, CoT prompting, and using LLM for complex planning.

####  III-D1 In-context learning (ICL)

ICL, first introduced with GPT-3 [12], utilizes formatted natural language
prompts and integrates task descriptions and examples to guide LLM models in
task execution. This approach allows the LLM to recognize and perform new
tasks by leveraging contextual information. The design of demonstrations is
critical for ICL, encompassing the selection, format, and order of examples.
The format of demonstrations involves converting selected examples into a
structured prompt, integrating task-specific information and possibly
incorporating reasoning enhancements like CoT [64, 68]. The ordering addresses
LLM biases, arranging demonstrations based on similarity to the query or
employing information-theoretic methods to optimize information conveyance
[69, 70].

ICL’s underlying mechanisms include task recognition and task learning, and
then LLM models can use pre-trained knowledge and structured prompts to infer
and solve new tasks. Task recognition involves LLM models identifying the task
type from the provided examples, leveraging pre-existing knowledge from pre-
training data [71]. Task learning, on the other hand, refers to LLMs acquiring
new task-solving strategies through the given demonstrations, a capability
that becomes more pronounced with increasing model size [72]. Recent studies
suggest that larger LLM models exhibit an enhanced ability to surpass prior
knowledge and learn from the demonstrations provided in ICL settings [73].

####  III-D2 Chain-of-thought (CoT) prompting

CoT prompting is an advanced strategy to enhance LLM’s performance on complex
reasoning tasks, such as arithmetic, commonsense, and symbolic reasoning, by
incorporating intermediate reasoning steps into prompts [68]. Differing from
ICL’s input-output pairing, CoT prompting enriches prompts with sequences of
reasoning steps, guiding LLM models to bridge between questions and answers
more effectively. Initially proposed as an ICL extension, CoT augments
demonstrations from mere input-output pairs to sequences comprising inputs,
intermediate reasoning steps, and outputs [68]. These steps help LLM models
navigate complex problem-solving more transparently and logically, though they
typically require manual annotation. However, creative phrasings such as
“Let’s think step by step” can trigger LLM models to generate CoTs
autonomously, which significantly simplifies the CoT implementation.

Despite improvements, CoT prompting faces challenges such as incorrect
reasoning and instability. The enhancement strategies include better prompt
design by utilizing diverse and complex reasoning paths, advanced generation
strategies, and verification-based methods. These methods address generation
issues by exploring multiple paths or validating reasoning steps, thus
improving result accuracy and stability. Furthermore, extending beyond linear
reasoning chains, recent studies propose tree- and graph-structured reasoning
to accommodate more complex problem-solving processes[74]. In addition, CoT
prompting significantly benefits large-scale LLM models (over 10B parameters)
and tasks requiring detailed step-by-step solutions. However, it may
underperform in simpler tasks or when traditional prompting is already
effective [68].

####  III-D3 Planning for complex task solving

While ICL and CoT prompting provide a straightforward approach for task
solving, they often fall short in complex scenarios like mathematical
reasoning and multi-hop question answering [75]. To this end, prompt-based
planning has emerged, breaking down intricate tasks into smaller and
manageable sub-tasks and outlining action sequences for their resolution. The
planning framework for LLM models encompasses three main components: the task
planner, the plan executor, and the environment. The task planner devises a
comprehensive plan to address the target task, which could be represented as a
sequence of actions or an executable program [76]. This plan is then carried
out by the plan executor, which can range from text-based models to code
interpreters, within an environment that provides feedback on the execution
results [77, 74]. In plan generation, LLM models can utilize text-based
approaches to produce natural language sequences or code-based methods for
generating executable programs, enhancing the verifiability and precision of
the planned actions [77]. Feedback acquisition follows, where the LLM
evaluates the plan’s efficacy through internal assessments or external
signals, refining the strategy based on outcomes from different environments
[74]. In addition, the refinement process is crucial for optimizing the plan
based on received feedback, and the corresponding methods include reasoning,
backtracking, memorization, etc [74].

###  III-E Evaluation metrics of LLM

Evaluating the performance of LLM models is a multifaceted task and receives
increasing attention. This subsection focuses on the evaluation metrics that
encompass various dimensions, including accuracy, hallucination, efficiency,
and human alignment etc. Each of these aspects plays a crucial role in
determining the overall applicability of LLM models in real-world scenarios
such as telecom networks. Firstly, accuracy is paramount in evaluating LLM
technologies as it directly impacts the model’s reliability and
trustworthiness. It measures how well an LLM can understand and process
natural language queries, generate relevant and correct responses, and perform
specific tasks like translation, summarization, or question-answering.
Benchmarks and standardized datasets are often used to quantitatively evaluate
the model’s accuracy.

Secondly, hallucination refers to instances where the LLM generates incorrect
or factual inconsistent information, often presenting it with a high degree of
confidence. This phenomenon can significantly undermine the credibility of
LLM-generated content. Evaluating an LLM’s tendency to hallucinate involves
analyzing the model’s responses for factual accuracy, consistency, and
relevance to the input prompt. Recent studies show that traditional automatic
metrics for summarization such as ROUGE [78] and BERTScore [79] show sub-
optimal performance on factual consistency measurement [80]. Thereafter, some
novel metrics have been proposed to detect hallucination errors, such as
AlignScore in [81].

Then, the efficiency of LLMs indicates the computational resources required
for training and inference, as well as the speed at which these models can
generate responses. As the size of LLM expands, this expansion leads to
significant issues regarding environmental sustainability and the ease of
access to these technologies [82, 83]. Evaluating an LLM’s efficiency involves
a detailed assessment of its performance relative to the consumed resources.
Key metrics for this assessment include the energy usage during operations,
the time it takes to process information, and the financial burden associated
with acquiring and maintaining the necessary hardware infrastructure.
Additionally, it’s important to consider the efficiency of data usage during
training, as optimizing data can reduce computational requirements [84].

The last metric is human alignment. Manual evaluation for LLM alignment to
human values generally offers a more holistic and precise assessment compared
to automated evaluation [85]. This is supported by numerous studies, such as
[86, 87], which incorporate human alignment evaluation to provide a more in-
depth analysis of their methods’ performance. Human alignment assesses the
degree to which the language model’s output aligns with human values,
preferences, and expectations. It also considers the ethical implications of
the generated content, ensuring that the language model produces text that
respects societal norms and user expectations, promoting a positive
interaction with human users.

TABLE III: Summary of LLM deployment strategies

LLM deployment strategies |  Main features & Advantages |  Potential issues & Difficulties
---|---|---
Cloud deployment |  Cloud deployment is the most straightforward method for LLM deployment. LLM models are usually computationally demanding, and cloud servers can provide abundant computational and storage resources for model training, fine-tuning, and inference. |  Cloud deployment indicates higher end-to-end latency for implementing user requests, since the inquiries have to be first uploaded and then processed and downloaded. It may prevent the application of some latency-critical applications such as robot control, vehicle-to-vehicle communications, unmanned aerial vehicle (UAV) control, etc.
Network edge deployment |  Network edge deployment can be an appealing approach to shorten the response time and save backhaul bandwidth to the central cloud. It enables rapid user request processing at edge servers or cloud, achieving shorter end-to-end delay than the central cloud-based approach. |  Network edge servers are usually resources-constrained, indicating limited computational and storage resources for LLM model fine-tuning and inference. Therefore, some techniques may be exploited, e.g., efficient parameter-efficient fine-tuning[88], split edge learning[89], and quantized training[90]. In addition, model compression is also a promising direction for edge LLM deployment.
On-device deployment |  On-device LLM is considered a very promising direction to deploy LLM models directly at user devices. It enabled customized LLM models based on specific user requests. Meanwhile, on-device LLMs have the lowest service latency by processing tasks locally. Therefore, it has great potential for implementing real-time tasks. |  Despite the great advantages, on-device LLMs are still in the very early stages, and the main challenge is to overcome the very limited computational and storage resources at user devices. Apple has proposed a technique to store LLM model parameters on flash memory[91] and achieve a 20 times faster inference speed. Qualcomm also announced a new mobile platform to support popular small-scale LLMs[92]. Therefore, how to utilize limited computation resources to achieve faster inference is the key to on-device LLM deployment.
Cache-based deployment |  Cached-based approach is proposed by [16] based on mobile edge computing architecture. Specifically, the authors propose to store the full-precision parameters in the central cloud, quantized parameters in the edge cloud, and the frozen parameters at the user devices, enabling more flexible model training and migration. |  Such a distributed deployment approach is promising to save the model store and migration cost. However, compared with on-device deployment, the cache-based method also requires complicated coordination strategies for model update and synchronization, e.g., model update and synchronization frequency and the quantization bit version selection.
Cooperative deployment |  Cooperative deployment is proposed in [93], which involves the interactions between local small models and cloud-based large models. In particular, it assumes that the local model can collect and submit sensor data selectively to the large model, and the large model will update the small-scale local models based on its domain-specific knowledge. |  The cooperative deployment is a feasible solution to connect small-scale local LLM models to large cloud models. However, the local model updating frequency should also be carefully determined to reduce the burden on cloud LLM models. In addition, note that the inference is still updated locally, and therefore the required computational resources are still challenges. To this end, it may be combined with on-device LLMs to address the resource issues.

###  III-F LLM Deployment in Telecom Networks

Practical deployment is the prerequisite for advanced applications of LLM
technologies in telecom networks. In particular, it indicates how LLM models
can be deployed within the current telecom network architecture, e.g., central
cloud, network edge or even user devices. The LLM has great demands for
computational and storage resources. For instance, GPT-4 has 1.76 trillion
parameters and the model size is 45 GB[63], posing a heavy burden on network
storage capacities. Fine-tuning an LLM model with 7 billion parameters, such
as GPT-4-LLM[94], could take nearly 3 hours on an 8$\times$80GB A100 machine,
which is extremely time-consuming [95]. In addition, the inference time of LLM
models will also contribute to overall network latency, which is related to
hardware support, batch size, parallelism, model pruning, etc[96]. Therefore,
it is of great importance to deploy LLM models appropriately to better serve
the telecom network demand. We summarize the existing deployment schemes in
Table III, including cloud, network edge, on-device, cache-based, and
cooperative deployment. We present the details of each strategy as the
following:

####  III-F1 Cloud deployment

: Considering LLM’s high demand for computational and storage resources,
deploying LLM models in the central cloud is a straightforward solution, which
can provide substantial computational resources to support the fine-tuning and
inference of LLM models[15]. Shen et al. investigate LLM-enabled autonomous
edge AI[15], in which the network edge devices can send the user request and
datasets feature to LLM models in the cloud, and then the LLM can send back
the task planning and AI model configuration to network edge devices through
the backhaul. After that, the network edge and user devices can collaborate to
make edge inferences. Cloud deployment can easily adapt to existing telecom
network architecture, and a few pieces of extra hardware are needed since the
LLM is deployed in the virtual cloud. However, cloud deployment suffers from
long response time and high bandwidth costs since all data has to be
transmitted to the cloud, and then LLM will process the request and finally
download the LLM’s output[16]. The long response time may prevent the
applications on latency-critical tasks, e.g., vehicle-to-vehicle networks and
unmanned aerial vehicle control. In addition, the frequent multimodal
information exchange, such as images and videos, between end users and cloud
LLM will lead to extra bandwidth costs.

Figure 3: Illustration of different LLM deployment strategies.

####  III-F2 Network edge deployment:

Here network edge refers to edge cloud or BSs that are closer to users than
central cloud. Network edge deployment can be an appealing approach to shorten
the response time and save bandwidth. However, compared with the central
cloud, network edge devices usually have limited computational and storage
capacities. To this end, multiple techniques can be exploited. For the storage
capacity challenge, parameter sharing and model compression may be applied. In
particular, LLM models for different downstream tasks may share the same
parameters, which can be exploited to save the storage capacity. On the other
hand, other technologies may be applied to reduce the computational resources
demand in fine-tuning and inference, including parameter-efficient fine-
tuning[88], split edge learning[89], and quantized training[90]. With these
techniques, deploying LLM models at the network edge becomes a practical
strategy.

####  III-F3 On-device deployment:

There are multiple benefits of deploying LLM models on user-side mobile
devices, e.g., fast responses and local customization based on the user’s
specific requirements. However, such a deployment is also challenging since
LLM models are usually storage- and computation-intensive. Xu et al.
introduced a split learning approach based on collaborative end-edge-cloud
computing, aiming to deploy LLM agents at mobile devices and network edge
[17]. Specifically, the authors assume that LLM models with less than 10B
parameters such as LLAMA-7B can operate on mobile devices, providing real-time
inference services. Meanwhile, LLM models with more than 10B parameters such
as GPT-4 are deployed on network edge servers, using global information and
historical memory to assist the mobile LLM in processing complex tasks. Such a
collaboration enables higher flexibility by exploiting mobile LLM models.
However, the study of on-device LLM is still in a very early stage, and it
requires considerable efforts to prove the feasibility of such a design[97].
For instance, Apple has proposed a technique to store LLM model parameters on
flash memory[91], achieving a 20 times faster inference speed than using GPU
with limited dynamic random-access memory (DRAM) capacity. Similarly, Qualcomm
has recently announced the Snapdragon 8s Gen 3 mobile platform, which supports
popular small-scale LLM such as Llama 2 and Gemini Nano[92]. These studies may
pave the way to effective inference of on-device LLMs.

####  III-F4 Cache-based deployment

Lin et al. proposed a cache-based method in [16], which utilizes the mobile
edge computing architecture to store, cache, and migrate models in edge
networks. Specifically, they propose to store the full-precision parameters in
the central cloud, quantized parameters in the edge cloud, and finally the
frozen parameters at the user devices. Such a separate model caching enables
more flexible model training and migration. For instance, edge clouds or
servers can apply low-precision computation by using quantized training,
improving the edge training speed with limited computational resources. In
addition, storing the frozen parameters on user devices can save the storage
capacities of the edge cloud, reducing the latency caused by full model
migration. However, the cache-based method may require complicated
coordination strategies for model update and synchronization, e.g., model
update and synchronization frequency and the quantization bit version
selection.

####  III-F5 Cooperative deployment

Lin et al. proposed a novel EdgeFM approach in [93]. In particular, the edge
devices will collect the sensor data from the environment, and then the local
model can evaluate the uncertainty features of the collected data and the
real-time network dynamics. After that, the local EdgeFM model will
selectively upload the unseen data classes to query large models in the cloud,
and the large models can periodically update a customized small-scale model at
the network edge. Therefore, when the network environment changes, at the
early stage, the local model can frequently query large models in the cloud,
and then it can execute customized small models on edge devices at the late
stage. Such a cooperative deployment can reduce the system overhead, and
enable dynamic customization of local small models for edge devices. The
experiment in [93] shows 3.2x lower end-to-end latency and achieve 34.3%
accuracy improvement than the baseline.

Finally, Fig. 3 illustrates different LLM deployment strategies. Note that
these approaches may be combined, e.g., deploying small-scale on-device LLMs
and then using larger cloud models to update the local models periodically.
Given these deployment methods, many critical problems can then be
investigated, e.g., service delay evaluation and task offloading, which still
require more research efforts. For example, Chen et al. proposed a NETGPT
scheme in [98], involving offload architecture, splitting architecture, and
synergy architecture for cloud-edge collaboration.

###  III-G Analyses of LLM Fundamentals in the Telecom Domain

Previous Sections III-A to III-F have covered the key techniques of LLM
fundamentals, ranging from model architecture and pre-training to evaluation
and deployment in telecom networks. This subsection will analyze how these
fundamental techniques can be applied to the telecom domain.

For telecom applications, pre-training an LLM model from scratch can be time-
consuming. It first requires extensive dataset collection, and the dataset
preprocessing has to consider the format of complicated telecom equations and
theories. Meanwhile, it also requires considerable computational resources to
pre-train LLM models, leading to heavy burdens for telecom networks. By
contrast, a more efficient approach is to fine-tune a general-domain LLM model
for specific telecom-domain tasks. Applying LLM technologies to the telecom
domain requires an in-depth understanding of these fine-tuning techniques,
such as instruction and alignment tuning methods. In particular, instruction
tuning involves carefully constructing and selecting instruction datasets,
employing strategic tuning methodologies, and considering practical
implementation aspects. These strategies will significantly improve the
performance, generalization, and user alignment of LLM technologies in the
telecom domain. On the other hand, alignment tuning is a multifaceted process
involving the setting of ethical guidelines, collection of human feedback, and
application of advanced fine-tuning techniques such as RLHF. However, adapting
these state-of-the-art fine-tuning techniques to telecom environments is still
an open question. The fine-tuning process is usually task-specific, which
requires professional knowledge of various telecom domain tasks. Instruction
tuning can be a promising method for building a telecom-LLM by using existing
telecom knowledge, but the dataset collection can be difficult due to privacy
issues.

Prompting techniques are especially useful for solving real-time telecom tasks
with stringent delay requirements, e.g., resource allocation and user
association. It means that LLM models can directly learn from the inputs and
generate desired outputs without extra training, avoiding the tedious model
training process in conventional ML algorithms. For instance, ICL provides a
framework for leveraging the LLM in new task domains without explicit
retraining, with its effectiveness heavily influenced by the design and
structure of demonstrations. Meanwhile, CoT prompting has emerged as a potent
method for eliciting deeper reasoning capabilities in LLM models, applicable
to a range of complex reasoning tasks. While still evolving, this approach
opens new avenues for LLM application across diverse problem domains such as
the telecom field. In addition, prompt-based planning represents a
sophisticated approach to navigating complex tasks, enhancing LLM’s problem-
solving capabilities through structured action sequences, feedback
integration, and continuous plan refinement. Such planning capabilities are
very important for telecom applications since many telecom tasks involve
multi-step thinking with complicated procedures. For instance, the resource
allocation may include multi-layer controllers[4], and optimization problems
can involve several agents and elements[99]. Therefore, multi-step planning
and thinking should be carefully designed for LLM-enabled telecom
applications.

Evaluation metrics are critical to assess the LLM model’s performance in
telecom environments. For instance, efficiency is one of the most important
metrics that should be considered in telecom applications since many tasks
require rapid or even real-time responses. Therefore, LLM models with long
inference times may be inappropriate for these mission-critical applications,
e.g., Ultra-Reliable Low Latency Communications (URLLC). In addition,
evaluating the performance of LLM models should also include their proneness
to hallucination and ethical standards, e.g., LLM may make misleading or even
wrong decisions in network management. As LLM design and models continue to
evolve and integrate more deeply into various aspects of society, the criteria
for their evaluation will likely expand and become more sophisticated.
Ensuring that LLM models are accurate, reliable, efficient, and ethically
responsible is essential for their sustainable and beneficial integration into
human-centric applications.

Finally, practical deployment is the prerequisite for applying LLM to telecom
networks. Compared with other domains such as education or healthcare, many
telecom tasks have stringent requirements for delay and reliability, which
require more efficient and reliable model output. Meanwhile, telecom devices
usually have limited computational and storage resources. Therefore, efficient
model training, fine-tuning, inference and storage techniques should be
explored[16]. With previous knowledge and analyses, we will present detailed
LLM-inspired techniques and applications in telecom tasks in terms of
generation, classification, optimization, and prediction problems in the
following sections.

##  IV LLM for Generation Problems in Wireless Networks

The outstanding generation capability is one of the most attractive features
of LLMs. This section first introduces the motivations for applying the LLM
technique to telecom-related generation tasks, and then it presents detailed
application scenarios, including telecom domain knowledge generation, code
generation, and network configuration generation.

TABLE IV: Summary of LLM-aided generation-related studies in the network
field.

Topics |  Refer- ences |  Proposed LLM-aided generation schemes |  Key findings & Conclusion |  Telecom application opportunities
---|---|---|---|---
Domain knowledge generation |  [100] |  Adapting a BERT-like model to the telecom domain and testing the model performance by question answering downstream task in the target domain. |  The proposed technique achieved $F1$ score of 61.20 and $EM$ score of 36.48 on question answering in a small-scale telecom question answering dataset. |  Techniques such as customizing LLM models to understand and apply telecom-specific language, evaluating their genuine understanding of domain knowledge generation can contribute to more efficient, reliable, and secure telecom service applications. Existing studies have demonstrated the capability of LLM techniques to be applied in telecom, including question answering, literature review, generating troubleshooting report. It shows great promises to build next generation communication networks.
|  [101] |  It proposed a multi-stage BERT-based approach to understand the textual data of telecom trouble reports, and then generate a ranked solution list for new troubles based on previously solved troubleshooting tickets. |  1) Presenting more information in the query can produce a better list of recommended solutions; 2) Creating a small candidate list is the key to reducing the model latency. |
|  [23] |  It combines a BERT-like method with transfer learning for trouble report retrieval, leveraging non-task-specific telecom data and generalizing the model to unseen scenarios. |  The experiment includes nearly 18500 trouble reports, showing that combining pre-trained telecom-specific language models with fine-tuning strategies outperforms pure domain adaptation fine-tuning. |
|  [102] |  Question answering test on various LLM models, e.g., GPT 3.5, GPT 4, and Bard, including telecom knowledge and product questions. |  Bard and GPT4 show promise with respect to accuracy and could be useful for telecom domain question and answering. LLM’s summarization requires reliability tests. |
|  [103] |  Integrating domain-specific grammars into LLM models to guide the generation of structured language outputs, enhancing performance in domain-specific tasks. |  Demonstrating the efficacy of integrating domain-specific grammars with LLM models in enhancing their ability to generate structured language outputs tailored to specific domains. It emphasizes the potential of this approach to significantly improve LLM performance in domain-specific tasks. |
Code generation |  [14] |  Using LLM models to generate Verilog code for wireless communication system development in FPGA. The experiment was implemented in the OpenWiFi project. |  The LLM is capable of refactoring, reusing, and validating existing code. With proper design and prompting, LLM models can generate more complicated projects with multi-step scheduling. LLM greatly reduced the coding time of undergraduate and graduate students by 65.16% and 68.44%, respectively. |  Code is the cornerstone of modern communication networks, and the LLM provide promising opportunities to improve the efficiency and reliability of codes, and meanwhile greatly save human effort. a) The LLM can refactor and validate existing code. This is very useful in telecom filed, since the network architecture is constantly evolving and updated; b) With proper prompting, the LLM can generate complicated projects with multi-step scheduling requirements, which is very common in telecom filed due to complicated network elements with diverse functions.
|  [104] |  It proposed a framework to use LLM to generate task-specific code for traffic analyses and network life-cycle management. |  Combining the LLM with proper libraries, such as GPT-4 and NetworkX, can achieve 88% and 78% coding accuracy for traffic analysis and network lifecycle management tasks, respectively. |
|  [18] |  Employing four students to reproduce the results of existing network studies with the assistance of LLM models. |  The students successfully reproduced networking systems by prompting engineering ChatGPT. They also achieve much lower lines of code by using ChatGPT, e.g., one of them is only 20% of the open-source existing version. |
|  [105] |  Using LLM techniques for automated program repair of introductory level Python projects. |  The proposed scheme successfully repaired a larger fraction of programs (86.71%) compared to the baseline (67.13%), and adding few-shot examples will raise the ratio to 96.50%. |
|  [106] |  Fine-tuning pre-trained LLM models on Verilog datasets collected from GitHub and Verilog textbooks and then generating Verilog projects. |  Fine-tuning LLM models over a specific language can improve the coding correct rate by 26%. |
Network configuration generation |  [107] |  It proposed a three-stage LLM-aided progressive policy generation pipeline for intent decomposition. |  Through evaluating a service chain use case, the paper found LLM models could generalize to new intents through few-shot learning and concluded leveraging LLM models for policy generation is promising for automatic intent-based application management. |  Telecom network operators can leverage the LLM for network configuration generation in various ways. This includes automatic network provisioning, optimization and performance tuning, security and compliance configuration, fault diagnosis and troubleshooting, and network virtualization. The LLM enables efficient, reliable, and secure generation of network configurations, reducing manual effort and improving network management in telecom environments.
|  [108] |  It proposed a multi-stage framework that utilizes LLM models to automate network configuration by taking in natural language requirements and translating them into formal specification, high-level configurations, and low-level device configurations. |  The results showed that state-of-the-art LLM technologies like GPT-4 are capable of generating fully working configurations from natural language requirements without any fine-tuning. |
|  [109] |  It proposed a framework that combines LLM models with verifiers, using localized feedback from verifiers to automatically correct errors in configurations generated by the LLM. |  The proposed scheme is able to synthesize reasonable though imperfect configurations with significantly reduced human effort, and coupling LLM models with verifiers providing localized feedback is necessary for real-world use configurations despite requiring more testing. |

###  IV-A Motivations of Using LLM-based Generation for Telecom

This subsection will introduce the key motivations of using LLM-enabled
generation for telecom applications. Firstly, LLM can make telecom knowledge
more accessible. LLM models have been pre-trained on many real-world datasets
and equipped with considerable knowledge from various fields. Therefore,
question-answering has become the most well-known application of LLMs. With
domain-specific datasets from websites and textbooks, the LLM can extract
professional knowledge from existing publications and then generate
appropriate answers based on users’ requests. For instance, Maatouk et al.
build a telecom knowledge dataset in [110], including 25,000 pages from
research publications, overview, and standards specifications. With proper
training and fine-tuning, such a dataset can greatly contribute to a Telecom-
GPT, providing a systematic overview of hundreds of publications and
standards. With reasoning and comprehension capabilities, professional telecom
knowledge will become much more accessible to all researchers and even benefit
the whole society.

Meanwhile, LLM’s generation capabilities can also automate many tasks that are
usually time-consuming. For instance, developing new standard specifications
usually requires considerable writing, discussions, and reviews. By contrast,
given enough historical reports and proper prompts, the LLM can produce a
draft standard instantly, and then the experts can review it accordingly.
Moreover, the experts’ comments can be fed directly to LLM models, and then
the LLM can produce a new version efficiently, significantly saving human
efforts on writing and revising paper works. Similarly, LLM technologies have
been used to generate code in many existing studies, which is one of the most
time-consuming tasks of modern industry[18, 105]. LLM models can refactor and
improve existing codes, contributing to developing telecom projects.

In addition, LLM models can easily learn from the provided existing examples,
which is known as ICL. This capability is particularly useful in generation
tasks, and LLM models can quickly generalize the given examples to related
unseen scenarios. Meanwhile, if the initial generated output can not satisfy
the requirements, users can also send the feedback directly to the LLM input,
and then the LLM agent will revise the generation accordingly. This user-
friendly generation approach will lower the difficulty of applying LLM
techniques to generation tasks in telecom, which usually requires considerable
professional knowledge and experience.

Given the above motivations and advantages, it is crucial to exploit LLM’s
generation capabilities and apply them to telecom networks. Table IV
summarized LLM-aided generation-related studies and telecom application
opportunities. In the following, we will introduce domain knowledge
generation, code generation, and network configuration generation.

###  IV-B Domain Knowledge Generation

Generating domain-specific knowledge is an important application of LLM
technologies in telecom. In particular, it refers to creating comprehensive
summaries, overviews, and interpretations of telecom standards, technologies,
and research findings. By leveraging vast datasets of technical documents,
research papers, and standards specifications, LLM agents can produce detailed
explanations and summaries that are tailored to the user’s level of expertise
and interest. This not only democratizes access to telecom knowledge but also
serves as a bridge to fill the gap between experts and non-expert users in the
telecom field.

####  IV-B1  Understanding telecom domain knowledge

Telecom is a broad field, and there are various domains of knowledge such as
signal transmissions, network architectures, communication protocols, and
industry standards. For instance, signal transmission is fundamental telecom
knowledge, involving the differences between amplitude, frequency, and phase
modulation, as well as the distinctions between digital and analog signals.
Meanwhile, communication protocols refer to sets of rules that ensure
standardized data transmission, allowing for interoperability among diverse
systems. Knowledge of these protocols is fundamental for the development and
maintenance of robust communication networks. Additionally, telecom standards
are equally important. Standards such as 3G, 4G, and the emerging 5G for
mobile communications, as well as IEEE 802.11 for Wi-Fi, play a critical role
in global telecom networks[111]. They facilitate the seamless operation of
devices and services across different networks.

A thorough understanding of the above telecom knowledge is not only vital for
the development of new technologies and services, but also for ensuring that
systems are interoperable and secure. The depth of understanding in telecom
knowledge directly impacts the ability to innovate, secure, and solve problems
within the telecom field. The integration of LLM models, trained with domain-
specific datasets, offers promising avenues for automating knowledge
generation and facilitating access to complex telecom content, thus bridging
the gap between experts and general users.

Figure 4: Using language models for automated troubleshooting in telecom
fields[23].

####  IV-B2 Training LLM models with telecom-specific data

Training LLM models with telecom-specific data involves curating and
preprocessing vast amounts of domain-specific information to fine-tune the
models, aiming to generate accurate and relevant content within the telecom
field. This process is crucial as it tailors the LLM’s capabilities to
understand and generate content that aligns with specific telecom
requirements. It can be summarized by following steps:

  * •

The first step in training the LLM with telecom-specific data is the
collection of datasets. These datasets may include technical documents,
research papers, standards specifications, and other forms of professional
literature prevalent in the telecom sector. For example, Holm et al. [100]
created a small-scale TeleQuAD to train the question-answering capabilities of
the build Bert-based model. Similarly, 185,000 trouble reports[23] are
included to train a Bert-like model to generate automated troubleshooting
tickets. However, these datasets are usually inaccessible to the public. By
contrast, Maatouk et al. [110] introduced a large dataset of telecom knowledge
to provide systematic overviews and detailed explanations of standards and
research findings.

  * •

Following dataset collection, the preprocessing stage involves cleaning and
organizing the data to make it suitable for training. This step may include
removing irrelevant information, correcting errors, and converting the data
into a format that is compatible with the ML model. The study [112] shows that
preprocessing large-scale datasets for LLM training can improve the model’s
learning efficiency and output quality.

  * •

Finally, it is worth noting that there are two main approaches to train LLM
models, which are training the model from scratch or fine-tuning a general-
domain LLM. In particular, training the model from scratch may produce better
performance since the model can specialize in telecom language, but it is also
time-consuming. On the other hand, the fine-tuning process adapts the pre-
trained LLM to the telecom domain. This step involves training the model on
the collected telecom-specific dataset, allowing it to adjust its parameters
to better understand and generate telecom content. Fine-tuning enables the
model to grasp the unique terminologies, concepts, and contexts of the telecom
field, significantly enhancing its generation capabilities. Although fine-
tuning a pre-trained LLM is much more efficient than training from scratch,
the experiment in [100] proves that training the model on telecom-domain text
from scratch can achieve better performance than fine-tuning a general-domain
model.

The integration of telecom-specific data into LLM training is not just about
enhancing the model’s knowledge base; it’s about equipping the LLM with the
ability to understand the nuances and complexities of the telecom field. This
tailored training approach ensures that the LLM can generate content that is
not only informative but also practical and applicable to real-world telecom
challenges.

####  IV-B3 Using LLM to telecom knowledge-related generation tasks

After proper training or fine-tuning, using LLM models to generate telecom
domain knowledge is a transformative approach that leverages the model’s
ability to process and synthesize vast amounts of information into coherent,
accessible content tailored to the needs of various stakeholders in the
telecom field. This capability extends from generating summaries of complex
technical documents to answering specific queries with detailed explanations,
thereby facilitating a deeper understanding of telecom technologies,
standards, and practices. In the following, we present some existing
applications of telecom knowledge-related generation tasks.

Telecom-domain question answering: Question answering is one of the most well-
known applications of LLM technologies. Using LLM models to answer domain-
specific questions is grounded in the model’s ability to interpret and
articulate complex information in a manner that is both comprehensive and
understandable. For example, Soman et al. evaluated the capabilities and
limitations of existing pre-trained general domain LLMs in [102], including
GPT-3.5, GPT-4, Bard, and LLaMA. For instance, one telecom-domain question is
”What are the different 5G spectrum layers?” GPT-4 identifies the bands as
below 1 GHz, 1-6 GHz and above 6 GHz, while LLaMA identifies the frequency
bands as below 600 MHz, 600 MHz-24 GHz and above 24 GHz. These differences
could be caused by different data sources of GPT-4 and LLaMA in the pre-
training period. However, this could easily confuse or even mislead users
without professional knowledge, which shows the importance of training a
telecom-domain LLM specifically. Holm et al. [100] further investigate how
various training methods can affect the model performance, e.g., pre-training
a model using telecom knowledge from scratch or fine-tuning an existing
general-domain model. In summary, LLM-enabled question answering democratizes
access to advanced telecom knowledge, making it accessible to a broader
audience, including researchers, practitioners, and the general public. In
addition, LLM agents can also tailor the generated content based on the user’s
level of expertise and specific interests. There is an increasing number of
commercial LLM products for generative question answering over business
documents, e.g., nexocode and Caryon. By leveraging the comprehensive
understanding and generation capabilities of LLM technologies, the telecom
industry can enhance the accessibility of complex information, support
educational endeavours, and streamline development processes.

Figure 5: Using ChatGPT to improve the code quality of OpenWiFi project[14]
(The pink fonts show the main changes).

Generating troubleshooting solutions for telecom trouble reports: Telecom
networks are complicated large-scale systems, and it is critical to identify,
analyze and then resolve both software and hardware faults, which are known as
trouble reports. The authors in [101] and [23] investigated using language
models to understand previous trouble reports and then generate recommended
solutions. Grimalt applied a BERT-based model to generate and rank multiple
possible solutions for a given system fault in [101], which archives a nearly
55% correct rate. Then, Bosch [23] improved the model in [101] by including
transfer learning and non-task-specific telecom data to improve the
generalization capabilities on handling unseen trouble reports. Fig.4
summarizes the proposed scheme in [101] and [23]. One can observe that the
analysis and correction phases can be time- and effort-consuming, which
usually requires professional knowledge of telecom networks and devices. To
this end, a language model-enabled method is proposed. It considers trouble
report observation, headings, and fault areas as input and generates the top-K
possible solutions. Then, the generated candidate solutions are sent back for
verification. In particular, the fine-tuning process of the language model
consists of three main steps, including the telecom language dataset, MS MARCO
document ranking dataset[113], and trouble report dataset. Here, the MS MARCO
dataset is included to train question-answering and ranking models, in which a
large number of question-answer pairs are collected from search engines[113].
Fig.4 proves that using language models to generate solutions for automated
troubleshooting can significantly improve overall efficiency, enabling faster
response and repair for telecom.

Finally, it is worth noting that these models may generate misleading or even
wrong solutions, which can be caused by different data sources, training
strategies, and so on[102]. For instance, the best correct rate in [23] is
around 60%, and therefore, verification is crucial before real-world
implementation.

###  IV-C Code Generation

Efficient and reliable code is of paramount importance to intelligent
communication networks. Recent studies have demonstrated the strong coding
capability of LLMs, including commonly-used languages (e.g., Python[114, 105])
and hardware description languages (e.g., Verilog [14, 106]). For instance,
Zhang et al. [105] apply the LLM to build an automatic program repair system
for introductory Python programming assignments, and the experiment on 286
real student programs achieves a repair rate of 86.71%. For hardware
description languages like Verilog for FPGA development, Du et al. [14] show
that LLM can reduce nearly 50% of the coding time for undergraduate and
postgraduate students and improve the quality by 44.22% for undergraduates and
28.38% for postgraduates. Existing studies[114, 105, 14, 106] have shown that
LLM can refactor and improve existing codes. In addition, well-crafted prompts
and designs can tackle complex, multi-step coding challenges encompassing
multiple sub-tasks. Given these potentials, introducing LLM-aided coding into
telecom can greatly save human effort in coding, validating, and debugging
while providing more efficient and reliable codes for telecom network
scheduling and management projects.

####  IV-C1 LLM for code refactoring

Code refactoring is a common task that is frequently involved when developing
wireless communication systems. Code refactoring aims to improve the
readability, efficiency, and reliability of existing code [115]. For instance,
good readabilities can lower the difficulty of long-term maintenance and reuse
of existing code modules. Readability is also a critical requirement for
wireless networks since the network architectures and protocols are constantly
evolving and updated, e.g., from WiFi 6 to 6E and WiFi 7, and from RAN to
cloud RAN and Open RAN. However, real-world projects usually include multiple
contributors with different coding styles and mixed qualities. Such an issue
could be very common in telecom, which are considered as complicated large-
scale systems that include multiple modules with diverse functions. Therefore,
improving code readability, efficiency, and reliability becomes more important
for the telecom field.

Fig. 5 shows an example from [14], which applies ChatGPT to revise the
original code of an open-source FPGA-based project OpenWiFi[116]. The pink
fonts indicate the changes made by ChatGPT. In particular, ChatGPT suggests
using meaningful names for modules and variables, e.g., replacing the name
“DelayT” with “DelayBuffer”. Meanwhile, four comments are added to improve the
readability of the input and output. The input and output data type
specification “wire” is added from line 2 to line 5, providing more explicit
definitions and higher reliability. ChatGPT also recommends adding the
negative edge of active-low reset signals in the “always” block in line 9 of
the revised code. Du et al. [14] explained that such an asynchronous reset is
more reliable and the system can make instant responses when detecting errors,
without waiting for the rising edge of the clock signal.

In addition, code validation is also an important task for telecom project
development. Du et al. [14] utilized ChatGPT to generate an error-free
testbench for effective OpenWiFi project validation. However, the fine-tuning
process is not investigated, which can be a prerequisite for effectively
generating hardware description languages. Different from the aforementioned
studies, Thakur et al. [106] fine-tuned a pre-trained LLM on Verilog datasets
collected from GitHub and textbooks, demonstrating that fine-tuned LLMs can
improve the coding correct rate by 26% on a specific language.

####  IV-C2 LLM-aided code generation with multi-step scheduling

Previous sections have shown that LLM can be used for fundamental coding
tasks. However, real-world telecom project development is usually much more
complicated by including multi-step scheduling and several sub-tasks. Xiang et
al. applied LLM to regenerate the code of existing studies in [18], and the
authors suggested that ChatGPT does not respond well to monolithic prompts
like ”implement this technique in the following steps”. Instead, a more
practical method is to send a detailed modular prompt each time. Such a step-
by-step approach is also investigated in [104] and [14]. Specifically, Mani et
al. [104] applied LLM models to network graph manipulation, and the prompt
design is decoupled into the application prompt and code generation prompt.
Specifically, the application prompt can provide task-specific prompts based
on templates and user queries, and then the code generation prompt can use
plugins and libraries to instruct LLM models. The experiment shows that
combining the LLM with proper libraries, such as GPT-4 and NetworkX, can
achieve 88% and 78% coding accuracy for traffic analysis and network lifecycle
management tasks, respectively. Du et al. investigated a more complicated
coding task in [14] by using Verilog to build a Fast Fourier Transform (FFT)
module. A failure is first observed by using the following prompt:

  * A failed prompt in [14] to generate FFT module. Help me write an FFT module for my FPGA system in Verilog language. Here are details of my specifications: … I also provide you with the instantiation template: …

The generated code failed because: a) FFT computation is a complicated task
with several sequential or parallel subtasks; b) The LLM lacks the
capabilities of multi-step scheduling. To this end, the authors decouple the
problem into four steps:

  * •

Step 1: Asking ChatGPT to generate two simple IP cores that are frequently
used in the following FFT design:

I am working on an FPGA project in Verilog. Please write two IP cores for me.
The first IP core is for butterfly computation for FFT. Here is its
template:…. The second IP core is for complex multiplication in FFT. I will
use it to multiply the output of a butterfly computation with the twiddle
factor provided… Here is a template of the IP core…

  * •

Step 2: Showing ChatGPT a simple 2-point FFT example with templates and
suggestions and then asking ChatGPT to produce a 4-point FFT IP core:

”I am writing a four-point DIF-FFT on FPGA. You can use the following IP cores
to build the target four-point FT IP core. Here is the template of butterfly
computation IP Core…”
”And here is the template of the two-point FFT IP Core..”
”Further, I also have some suggestions for you…”

  * •

Step 3: Asking ChatGPT to develop an eight-point FFT module based on the
generated 4-point FFT in Step 3:

”I am writing an eight-point DIF-FFT on FPGA. Apart from IP cores Given in
Question One,…, you can also use the fft_4_point IP core generated in Answer
one. You need to look back to Question-1 and Answer-1 for detailed
input/output information on the four IP cores. Once again, I want to emphasize
that:…”

  * •

Step 4: Finally, asking ChatGPT to generate a 16-point FFT using the 8-point
FFT that has been generated in Step 3. This step is repeated in [14] by asking
for a $2N$-point FFT module based on previously generated $N$-point FFT
modules.

Steps 1-4 is an obvious step-by-step CoT approach. Instead of asking for an
8-point FFT module directly, it starts from two simple IP cores and then
provides examples of 2-point FFT modules with detailed suggestions. This is a
very useful technique for LLM-aided project design in telecom networks,
decoupling the objective into several steps with detailed examples and
suggestions.

Finally, we summarize some key lessons from existing studies on the use of LLM
for code generation. Firstly, step-by-step prompt design is an important
lesson that has been demonstrated in several existing studies[14, 18, 104].
Decoupling the complicated multi-step scheduling problem into several stages
will lower the difficulty for LLM’s understanding. For instance, in 5G cloud
RAN simulation, we can divide the network into cloud, edge, and users, and
then use LLM to generate the code for each part sequentially. Secondly,
examples and pseudo-code are important for code generation. The LLM has
excellent ICL capabilities, quickly learning from examples and generalizing to
other scenarios. Xiang et al. [18] also reveal that implementation with
pseudocode first can produce stabilized data types and structures, avoiding
other changes when implementing the following components. There have been many
codes for the telecom field in GitHub and textbooks, taking advantage of these
existing examples is crucial to use LLM techniques. Then, a significant amount
of human effort can be saved in code generation by using LLM models for
debugging and testing. Xiang et al. [18] also shows that most errors can be
solved by sending the error message to the LLM. Many of these errors are
related to data types, which can be avoided by specifying key variables’ data
types. This lesson is also proved in [14], in which the LLM specified the data
types of inputs and outputs to improve the reliability of existing code.
Finally, LLM-aided coding can lower the requirement for professional knowledge
[14, 18]. In particular, Du et al. [14] show that both undergraduate and
graduate students can benefit from the assistance of LLM models, achieving
comparable coding qualities. Xiang et al. [18] prove that undergraduate
students can reproduce the results of some existing network studies by using
the LLM.

Figure 6: Frameworks for LLM-based network configuration generation.

###  IV-D Network Configuration Generation

Network administrators orchestrate the flow of information within a network.
They can guide data from source to destination by configuring a complex set of
parameters for network elements. These configurations impact a wide range of
devices and services, such as switches, routers, servers, and network
interfaces. To ensure a reliable data stream, these settings require precise
calibration across all network functionalities. Over the past ten years, both
academic institutions and the commercial sector have embraced the concept of
Software-Defined Networking (SDN) [117] as a means to streamline network
management, marking a shift away from the older, more rigid networking models.
SDN offers numerous advantages; nonetheless, adjusting network settings
remains a task that often requires manual input. Such manual adjustments can
be expensive, as they demand the skills of specialized developers familiar
with various network protocols, and meanwhile such manual configurations are
also intricate and prone to errors. Numerous initiatives have been launched
with the aim of streamlining the translation of overarching network guidelines
into individual settings for each network component. Such efforts focus on
reducing human errors by creating verifiable and reliable configuration
outputs through rigorous checks [118, 119]. Nonetheless, setting up network
configurations is still considered as a labour-intensive, intricate, and
costly endeavour for network operators.

Recent advancements have demonstrated that the LLM possesses the ability to
generate cohesive and contextually relevant content. They can answer questions
and sustain in-depth conversations with users. Applications like GitHub
Copilot and Amazon CodeWhisperer exemplify these advancements, assisting with
a variety of programming-related tasks. These developments inspire confidence
that the LLM can also be utilized to generate network configurations [24,
107].

One notable development of LLM-aided network configuration is CloudEval-YAML
[24], a benchmark that provides a realistic and scalable assessment framework
specifically for YAML configurations in cloud-native applications. This
benchmark utilizes a hand-crafted dataset and an efficient evaluation platform
to thoroughly examine the performance of LLM models within this context.
Dzeparoska et al. [107] have introduced a pioneering method that employs the
few-shot learning capabilities of the LLM to automate the translation of high-
level user intents into executable policies. This approach facilitates
dynamic, automated management of applications without the necessity for
predefined procedural steps. In a related vein, Wang et al. [108] have
developed NETBUDDY, a multi-stage pipeline that leverages LLM models to
translate high-level network policies specified in natural language into low-
level device configurations. NETBUDDY first uses an LLM to convert the input
into a formal specification, such as a data structure to express reachability.
It then generates forwarding information and configuration scripts from the
formal specification. Finally, NETBUDDY interacts with an LLM multiple times
to sequentially provide topology, addressing details and prototype programs to
automatically generate vendor-agnostic configurations for the switches and
routers. The evaluation of the network emulator demonstrates NETBUDDY’s
ability to enforce path policies and dynamically modify existing deployments.
In addition, Mondal et al. [109] presented Verified Prompt Programming (VPP)
to improve GPT-4’s ability to synthesize router configurations. VPP combines
GPT-4 with verifiers like Batfish [120], which check configurations for syntax
errors and semantic differences. Experiments showed that VPP presented
$10\times$ leverage performance for translating a Cisco configuration to
Juniper format by identifying and fixing syntax errors, structural mismatches,
attribute differences, and policy behaviours through 20 automated prompts.
Implementing no-transit policies across 6 routers achieved $6\times$ leverage
performance with 12 automated prompts guiding GPT-4 to resolve syntax,
topology, and semantic policy errors.

Fig. 6 summarizes three frameworks for LLM-based network configuration
generation. In particular, the first framework employs a simplistic design,
directly utilizing LLM to generate network configurations from natural
language. However, the generated configurations may be inaccurate and require
human inspection and improvement. In the second framework [108], a
hierarchical design is employed, where multiple LLMs collaborate to generate
low-level network configurations step-by-step, aiming to enhance the final
output. The verification scheme is crucial to evaluate the quality of the
produced configuration, which may be placed in the second design as in [109]
and [108] to check the syntax, compilability, and correctness of the generated
output. The third framework [109] is an automated design, incorporating an
automatic verifier once the configuration is generated. This verifier
validates the configuration and allows the LLM to automatically refine the
output. While human inspection is still necessary, this approach significantly
reduces the extent of manual intervention required. It is worth noting that
these frameworks are not mutually exclusive and can be combined. For instance,
in the hierarchical design, an automatic verifier can be added after each LLM
iteration.

These existing studies have demonstrated the potential of using the LLM to
configure networks automatically, which can be very useful in configuring
telecom network settings. The LLM offers promising opportunities for the
automation of tedious tasks, reduction of human error and cost, and rapid
prototyping and deployment of network infrastructure. However, telecom
networks are complex systems with numerous interdependent components, and
there are still many challenges to applying LLM technologies to telecom
network configuration, e.g., contextual understanding, error handling and
verification, security concerns, and interoperability between vendors and
devices. For example, networks often comprise devices from various vendors,
each with its own configuration language and parameters. The LLM must be
capable of understanding and generating configurations that are compatible
across these diverse environments. In addition, network configurations must
adhere to security best practices. The LLM must be equipped to understand and
apply these practices consistently to avoid creating security vulnerabilities.

TABLE V: Summary of LLM-based Generation for Telecom.

LLM-based generation applications |  Specific scenarios |  Main features |  Prompt and fine-tunning requirement |  Advantages compared with conventional approaches |  Applications for telecom fileds
---|---|---|---|---|---
Domain knowledge generation |  Telecom-domain question answering |  Question answering is the most well-known application of LLMs. It represents a significant step forward in the ongoing effort to bridge the knowledge gap in telecom and empower individuals and organizations within the field, including telecom question answering, literature summary and review, etc. |  General domain LLM models can also answer telecom-related questions, but fine-tuning a telecom-specific LLM can provide more reliable and professional answers. CoT prompting may improve the answer quality. |  The use of LLM techniques signifies a shift towards more efficient and accessible knowledge dissemination methods than any existing textbooks, websites, and tutorials for their comprehension and reasoning capabilities. |  1) Building a telecom-domain LLM is a promising direction to make telecom-knowledge more accessible for both professional researchers and the public. 2) Automated troubleshooting is another promising application to automate the problem-solving process in telecom fields. 3) The LLM also have the potential of generating other language-related tasks, e.g., specifications and protocols.
|  Generating solutions based on trouble reports |  Using language models to generate troubleshooting solutions automatically. It considers trouble observations and fault information as input, and produces recommended solutions. |  The language model must be fine-tuned on telecom-domain language and trouble reports datasets. An extra document ranking fine-tuning is required to realize recommendation functions. |  Automated troubleshooting is a very promising technique to greatly save human time and effort, since the conventional approach relies on expert knowledge and trial-and-error tests. |
Code generation |  Code refactoring |  Using the LLM for fundamental code refactoring and design validations, improving the code quality automatically without human intervention. |  The prompt input is easier since no multi-step scheduling is involved. Fine-tuning the LLM based on existing codes can improve the quality of the generated code. |  1) Improving the readability, efficiency, and reliability of the project. 2) Considerably saving human effort on coding, debugging, and testing the project. 3) Lowering the requirement for professional knowledge when developing a system. |  Coding is one of the most time-consuming part in wireless system development. Incorporating LLM-aided coding can greatly save human effort and improve the code quality. However, datasets may be required for fine-tuning, which can be collected from GitHub or wireless textbooks.
|  Coding tasks with multi-step scheduling |  The LLM can also be used to generate complicated projects with multi-step scheduling and sub-tasks. |  The input prompts have to be carefully designed in a CoT approach with appropriate examples and templates. |  |
Network configuration generation |  Automatic network configuration generation by using LLM models. |  Using the LLM to generate network configurations automatically, and then verify by LLM models or humans. |  The prompt must be carefully designed due to the complexity of network configurations, e.g., dividing the prompts into the task-specific part and code generation part. |  The LLM enables efficient generation of network configurations, reducing manual effort and cost in the telecom industry. |  Applications include automatic network provisioning, performance tuning, security and compliance configuration, etc.

###  IV-E Discussions and Analyses

LLM techniques have promising generation capabilities for telecom
applications, and Sections IV-B to IV-D have introduced various scenarios for
generating telecom knowledge, troubleshooting reports, code, and network
configuration. Table V summarized the main features, input and fine-tuning
requirements, advantages, and telecom applications. In the following, we
summarize the key findings and analyses.

Firstly, prompt design is critical for multi-step network generation problems.
Telecom networks are complicated systems, and many tasks require dedicated
design with multi-step scheduling. For example, the wireless code generation
problem in Section IV-C2 proves that using a one-step prompt to generate the
whole project is unpractical, e.g., generating a 64-point FFT module [14], and
reproducing the results of existing publications [18]. Therefore, proper
prompt design is critical to obtain the desired output. For instance, the
prompt design [104] is decoupled into the application prompt and code
generation prompt, in which the application prompt focuses on task-specific
requirements, and the code generation prompt uses plugins and libraries to
instruct LLM models.

Meanwhile, LLM-enabled generation can significantly save humane efforts.
Existing studies have shown that LLM has excellent capabilities for code
refactoring and validation, which are usually solved manually with
considerable human effort. Applying such a technique to the telecom field will
significantly save human labour on projecting coding, validating, and
debugging. For instance, Zhang et al. [105] introduce that the LLM can
successfully repair 86.71% programs for introductory level Python projects,
and adding few-shot examples will raise the ratio to 96.50%. In addition, LLM
models can also abstract fundamental knowledge in the network field from
textbooks, journals, and specification standards, which avoids the time-
consuming literature review process.

LLM models have been trained on many real-world datasets from web pages like
Wikipedia, and they can be further fine-tuned on domain-specific datasets,
e.g., telecom[110] and cybersecurity[121]. Despite the satisfactory
performance, there is no guarantee for the correctness of the generated
output. Such risks are avoided when the generated code or network
configuration can be verified by pre-designed test cases. However, when using
the LLM to summarize or extract knowledge from existing literature, the
quality of generated knowledge is hard to validate. For example, LLM models
may produce wrong numbers or units, and these mistakes can easily mislead
users without professional knowledge. To this end, efficient validation
schemes are crucial to evaluate the performance of generated solutions,
especially for coding and network configuration problems. Human verification
is a simplistic and straightforward approach, but it requires considerable
human labour and can be inefficient. Therefore, automatic validation is the
key to improve the overall efficiency of the whole pipeline, e.g., sending the
code implementation error message to a LLM for automatic debugging[18], and
using LLMs to validate the network configuration files.

##  V LLM-enabled Classification Problems

Classification problems are extensively studied within telecom networks.
Accurate and robust classification is crucial for improving network service
quality and performance. This section will introduce the motivations and
capabilities of LLM technologies in addressing a range of classification
problems, including attack classification and detection, text classification,
image classification, and encrypted traffic classification.

TABLE VI: Summary of LLM-aided classification-related studies and telecom
applications.

Topics |  Refer-ences |  Proposed LLM-aided generation schemes |  Key findings & Conclusions |  Telecom application opportunities
---|---|---|---|---
Security related classification |  [121] |  Building a specialized cybersecurity language model (named SecureBERT) through fine-tuning RoBERTa [45] on a cybersecurity corpus. |  SecureBERT excels at understanding text within a cybersecurity context, which enables a strong generalization capability across various telecom security tasks. |  By fine-tuning pre-trained general LLM models [121, 122, 123] or building security-specific models from scratch [124, 125], LLM models exhibit the advantage in understanding security context, enabling the application of LLM techniques across a range of telecom security tasks. Existing studies show that LLM-based method can outperform existing ML and DL models in terms of classification and detection accuracy. LLM techniques can also provide incident recovery suggestions. However, it is essential to initially create relevant training and testing datasets extracted from security-related telecom language corpora.
|  [124] |  Building a security-specific LLM from scratch designed for detecting network cyber threats, involving several steps: data preparation, data tokenization, model training, and model deployment. |  SecurirtBERT showcases the powerful predictive capabilities of security-specific LLM models in identifying various types of attacks, significantly outperforming the traditional ML and DL models. |
|  [122] |  Building a novel classifier of cybersecurity feature claims (named CyBERT) by fine-tuning a pre-trained BERT language model [44] on industrial control device documents. A large repository is created to gather industrial device information encompassing 41073376 words. |  CyBERT enables the effective identification of cybersecurity claim-related sequences, with an accuracy improvement of 19% in comparison to the general BERT text classifier [44]. |
|  [123] |  Applying transfer learning to a BERT model [44] to extract changeable token embeddings from vulnerability descriptions. A pooling layer is positioned at the top to extract sentence-level semantic features. |  The exploitability prediction framework (named ExBERT) not only accurately predicts software vulnerabilities but also learns sentence-level semantic features and captures long dependencies within descriptions. |
|  [125] |  Applying a BERT model [44] to tokenize URLs within HTTP requests and then passing these tokens to a multilayer perceptron model to distinguish normal and anomalous HTTP requests. |  By integrating NLP with web attack detection, BERT [44] demonstrates strong capabilities in understanding web requests and SQL language, achieving remarkable detection performance that significantly surpasses that of traditional ML detection methods. |
Text classification |  [126] |  It applied an AraBERT model to classify telecom customer satisfaction in Saudi Arabia by using the Twitter dataset. |  BERT-based model obtained more accurate and stable results than conventional CNN and RNN algorithms. |  LLM techniques have inherent advantages in processing text-related tasks. Existing studies have shown that the LLM can achieve a comparable performance as existing CNN or RNNs. It is promising for text-related telecom tasks such as standard developing and user feedback processing[127].
|  [13] |  Fine-tuning several LLM models, e.g., BERT, distilled BERT, RoBERTa and GPT-2, to the telecom domain languages, and using them for 3GPP standard classification problems. |  With proper pre-processing and fine-tuning, the experiment in [13] can achieve an 80% accuracy even if only 20% of the text segments are used. |
Image classification |  [26] |  It investigates the zero-shot image classification capabilities of LLaVA model, which means using the model directly without any extra training. |  The performance can be significantly improved with a combination of carefully crafted prompts, hierarchical classification strategies, and adjusted model temperatures. |  Images are important information for telecom sensing. Enabling efficient image classification can be very useful for many telecom applications, including vision-aided sensing, mmWave beamforming [25], user localization[128], and so on.
|  [129] |  Using LLM’s inherent knowledge to generate descriptive sentences with crucial discriminating characteristics of the image categories. |  This simple approach can effectively improve the zero-shot image classification accuracy on a range of benchmarks. |
Network traffic classification |  [130] |  Capturing long-distance contextual relations within traffic sequence through BERT, and then integrating packet-level token semantic features at the forward and backward positions of BiLSTM, which enhances the BiLSTM attention to packet-level features. |  BiLSTM can capture relevant features of front and rear token sequences after BERT extracts general features of encrypted traffic, learning the long-distance relations within token sequences. |  The LLM facilitate effective encrypted traffic classification, a critical technique in telecom network management while protecting data and user privacy. Note that the assumption of clean pre-training data presents challenges in secure traffic classification. This vulnerability is exposed particularly when attackers craft a poisoned model with backdoors by inserting low-frequency words as toxic embeddings. Such manipulation allows attackers to deceive the normally fine-tuned model during specific classification tasks.
|  [131] |  Building an Encrypted Traffic BERT (named ET-BERT), which aims to learn generic traffic representations from large-scale unlabeled encrypted traffic. |  ET-BERT showcases strong effectiveness and generalization across five encrypted traffic classification tasks, e.g., General Encrypted Application Classification [132], Encrypted Malware Classification, Encrypted Traffic Classification on VPN [133], etc. |

###  V-A Motivations and Classification Capabilities of LLM Models

Conventional classification techniques heavily rely on statistical methods.
However, with the recent advancements in telecom networks, there has been a
surge in multi-modal and heterogeneous network data, e.g., numerical traffic
data, textual security logs, and environmental images, which presents
significant challenges for traditional classification techniques, indicating a
need for more advanced and adaptable approaches. Recently, LLM techniques have
shown their capability to effectively process multi-modal and heterogeneous
data across both natural language and computer vision fields. These
capabilities position them as a promising research direction for addressing
classification problems within telecom networks.

Firstly, LLM technologies can contribute to telecom security by automated
security language understanding and classification. Inspired by numerous
advancements in NLP, LLM excels at explaining textual contents and
transforming them into informative representations, such as GPT [134] and BERT
[44]. With their strong capabilities, LLMs have recently shown exceptional
superiority in attack detection, aiming to enhance the security of telecom
networks[124]. Through fine-tuning pre-trained models or developing LLMs from
scratch, LLMs retain the functional capabilities in general English while
gaining a thorough understanding of the specialized security language,
allowing LLM models to effectively identify and respond to security threats in
telecom networks.

Secondly, LLM techniques have inherent advantages in text-related
classification tasks, which are very useful for text processing and
classification in the telecom field, e.g., customer textual feedback, telecom
standard specifications, technique reports and publications. For example,
enhancing the quality-of-experience (QoE) of telecom services hinges on a
comprehensive understanding of customer feedback, which may include various
real-world topics, ranging from signal strength to sending messages and calls
[135]. Given the superiority across various text-related tasks, LLMs have
strong capabilities to classify customer comments and extract useful feedback,
allowing telecom operators to enhance service quality by properly
understanding user satisfaction levels.

In addition, LLM models can extract visual features from the dynamic and
complex telecom environment. The integration of computer vision and image
processing into the telecom field, such as equipping BSs with cameras to
pinpoint user locations, can boost network efficiency in the dynamically
changing wireless environment. Although primarily focusing on processing and
understanding textual information, some LLM models also have remarkable image
processing capabilities in vision-related tasks[136], including image-to-text
generation [137] and object detection [138], etc. Consequently, this
integration enables the LLM to analyze both visual and network data, which can
effectively bridge the gap between textual and visual data analysis, leading
to a more comprehensive approach for network management.

Finally, LLM’s zero-shot classification capabilities have been demonstrated in
multiple existing studies, such as text and image classification tasks[26,
129]. In particular, it means that the LLM can be used to classify and detect
objects by using the real-world knowledge learned in the pre-training phase,
and no extra training is required for target tasks. Such zero-shot
classification capabilities can be appealing for telecom networks since many
telecom classification tasks need rapid responses, e.g., network attack
detection[124], image processing and classification[25]. With the above
potential and motivations, in the following, we will introduce LLM-enabled
attack classification and detection, text classification, image
classification, and encrypted traffic classification problems.

###  V-B LLM for Telecom Security and Attack Detection

Figure 7: Framework of LLM-based attack detection[124].

The numerous advancements in telecom have led to more complex and
interconnected infrastructures with a wide range of technologies, protocols,
and services, which can pose significant challenges in controlling and
monitoring telecom security. The growing threats and incidences of hostile
attacks have exposed severe vulnerabilities in telecom. For instance, Denial
of Service (DoS) can decrease network availability by overwhelming systems,
and Man-in-the-Middle (MITM) attacks can violate network integrity by secretly
modifying communications between two parties. This underscores the requirement
for robust attack detection mechanisms to monitor the network system against
malicious activities. However, with the evolution of current telecom networks,
a surge of multi-modal network data can be captured, including numerical
measurements such as traffic loads and CSI, and descriptive textual contents
with device logs and network configurations. The data contains a substantial
amount of redundant and correlated information, potentially obscuring critical
patterns in attack detection, which poses significant challenges to achieving
accurate attack detection.

Recently, NLP has achieved numerous successes in capturing informative
features from multi-modal and heterogeneous data across various application
scenarios, including sentiment analysis, speech recognition, and machine
translation, among others. Specifically, LLM techniques have emerged as a
promising direction across various NLP applications, which are beneficial to
explaining textual inputs and transforming them into quantitative
representations. The common method to apply LLM models across various domains
involves employing general-domain models as baselines, followed by fine-tuning
them for specific domain tasks. To enhance the security of telecom networks
through LLM techniques, it is important to note that the security language,
such as ransomware, API, OAuth, and keylogger, significantly differs in
structure and semantics from the general linguistic language. This suggests
that a conventional LLM may find it challenging to fully understand the
specific vocabulary inherent to security-related texts, potentially leading to
limited generalization ability in security applications. To this end, existing
studies that employ LLM techniques for attack detection can be categorized
into two primary directions as follows:

####  V-B1 Fine-tuning pre-trained LLM models

Existing studies have leveraged pre-trained LLMs and adapted them to achieve
specific security objectives through model fine-tuning [121]. For instance,
Aghaei et al. [121] introduce a specialized cybersecurity language model named
SecureBERT, which is capable of processing texts with cybersecurity
implications and effectively applied across a broad range of cybersecurity
tasks, including phishing detection, intrusion detection, code and malware
analysis, etc. In particular, SecureBERT applies a cybersecurity corpus
comprising 1.1 billion words, divided into 2.2 million documents, with each
document averaging 512 words through the Spacy text analytic tool [139]. To
build the security-customized tokenizer, a byte pair encoding method is
employed to extract 50265 tokens from the cybersecurity corpus to generate the
initial token vocabulary. Among all the extracted tokens, SecureBERT and
RoBERTa [45] share 32592 mutual tokens, while SecureBERT identifies 17673
tokens specific to the cybersecurity corpus, including firewall, breach,
crack, ransomware, malware, phishing, and vulnerability, among others. Each
token is represented by an embedding vector with dimensions identical to those
in pre-trained RoBERT, augmented with random Gaussian noise added to the
embedding factor of each token. SecureBERT emulates the architecture framework
of RoBERT [45], encompassing twelve transformer and attention layers, which
are trained on the specifically collected corpus through the customized
tokenizer tailored to the unique task requirements.

SecureBERT is evaluated to predict masked security-related words within a
sentence, which is the task known as masked language models. The testing
dataset is generated by extracting sentences from cyber-security reports with
17341 records. The experiment shows that SecureBERT outperforms RoBERTa,
powerful model on general language, in predicting masked words within a
sentence with a security context, as illustrated in the following
examples[121]:

Comparisons between SecureBERT and RoBERTa in masked tasks[121] Task 1:
“Information from these scans may reveal opportunities for other forms
$<$mask$>$ establishing operational resources, or initial access.” SecureBERT:
reconnaissance.
RoBERTa: early. Task 2: “Search order $<$mask$>$ occurs when an adversary
abuses the order in which Windows searches for programs that are not given a
path.” SecureBERT: hijacking.
RoBERTa: abuse. Task 3: “Botnets are commonly used to conduct $<$mask$>$
attacks against networks and services.” SecureBERT: DDoS.
RoBERTa: automated.

The three predicted terms reconnaissance, hijacking, and DDoS are prevalent in
cybersecurity corpora. SecureBERT accurately understands the security context
to predict these masked words, whereas RoBERTa exhibits incorrect prediction,
underscoring the advantages of SecureBERT in security-related language tasks.

####  V-B2 Building security-specific LLM models from scratch

In addition to fine-tuning, another strategy is to build an LLM from scratch
specifically designed for network-based attack detection. For example, Ferrag
et al. [124] designed SecurityBERT for detecting the ever-evolving cyber
threat landscape, which involves several steps: data preparation, data
tokenization, model training, and model deployment, as shown in Fig. 7. In
particular, the authors utilize a publically available dataset EdgeIIoTset
[140] related to the Internet of Things (IoT) and Industrial IoT (IIoT)
connectivity protocols, categorized into five types of threats: DoS/D-DoS
attacks, information gathering, MITM attacks, injection attacks, and malware
attacks. Then, to leverage the power of LLM models, null features are
eliminated during the feature extraction in [124], and both numerical and
categorical features are converted into textual representations. Specifically,
each feature is combined with its column name and value and then subjected to
hashing. The hashed values from the same instance are merged into a sequence,
which generates a fixed-length textual representation of the network traffic
data while maintaining privacy. After that, ByteLevelBPETokenizer [141] is
subsequently applied to segment the textual representations of the network
traffic data. This segmentation process breaks down the text into smaller
subwords, expected to be found in the tokenizer’s vocabulary. After the pre-
training phase, the model is fine-tuned on a labelled dataset [140] by adding
a Softmax activation function at the output layer, which allows SecurityBERT
to enable the learned contextual representations in the specific task of
attack detection. Finally, in the deployment phase of Fig. 7, once attacks are
identified through SecurityBERT, FalconLLM is further employed to determine
the severity and negative impact of identified attacks, leading to the
formulation of potential mitigation strategies and recovery procedures.

SecurityBERT is employed to identify normal events and 14 distinct attacks in
[124], such as DDoS_UDP, DDoS_ICMP, SQL_Injection, Vulnerability_Scanner, etc.
The experiment shows that SecurityBERT achieves the average accuracy, recall,
and F1-score of 0.98, 0.84, and 0.84, respectively, demonstrating the strong
classification capabilities of security-specific LLM models in identifying
various types of attacks. In addition, SecurityBERT significantly surpasses
the performance of traditional ML and deep learning models such as decision
tree, convolutional neural networks (CNN), recurrent neural network (RNN), and
long short-term memory (LSTM).

Finally, to develop security-specific LLM technologies for telecom networks,
it is essential to initially create relevant training and testing datasets
extracted from security-related telecom language corpora. Following model
fine-tuning with security-customized tokenizers, these language models can
significantly boost performance across various telecom security tasks,
including cyber threat intelligence, vulnerability analysis, and threat action
extraction [142, 143].

###  V-C Text Classification

Text classification and processing is a useful technique for the telecom
industry, and the applications include user enquiries and intent
classification and analyses[127], automated trouble report
classification[144], standard specification classification[30], and so on. In
the following, we introduce two applications in telecom customer feedback
analyses and specification classification.

####  V-C1  Using LLMs for telecom user feedback classification and analyses

Understanding user feedback is crucial for telecom operators to improve the
QoE and maintain customer satisfaction and loyalty. For instance, Vieira et
al. applied CNN and LSTM networks in [127] for sentiment analysis and topic
classification, and the analysis proved that 78.3% of the complaints are
related to weak signal coverage, and 92% of these regions have coverage
problems considering a specific cellular operator. These analyses can be
particularly useful for telecom operators to improve service quality such as
signal coverage and strength. However, user feedback can be complicated by
involving service experiences, suggestions, recommendations, and complaints.
In addition, the feedback can be collected from various sources, e.g., social
media, websites, phone calls, and company collection. These challenges require
more advanced ML techniques to better classify and capture user’s intentions.
The LLM has shown superb performance in a range of text-related tasks, e.g.,
question answering, summarization, dialogue, and sentiment analysis,
outperforming many existing techniques even in zero-shot settings. For
instance, Aftan et. al applied AraBERT model to classify telecom customer
satisfaction in Saudi Arabia by using the Twitter dataset [126], and the BERT-
based model obtained more accurate and stable results than conventional CNN
and RNN algorithms. In addition, using LLM models to analyze customers’
experience and intent has attracted considerable interest from both industry
and academia, e.g., Microsoft has proposed to use LLMs to generate, validate,
and apply user intent taxonomies[145]. Therefore, it shows great promise in
integrating LLM technologies into the telecom industry for text-related
classification tasks.

Figure 8: Framework of LLM-aided 3GPP specification classification[13].

####  V-C2 LLM-aided telecom standard classification

Telecom standards refer to agreed-upon specifications that ensure the
interoperability, security, and reliability of telecom services. Standards
play a critical role in global telecoms [111], such as 3G, 4G, and emerging 5G
for mobile communications, IEEE 802.11 for Wi-Fi, and ITU-T recommendations.
For instance, 3GPP is the main organization for telecom standard development,
which includes three technical specification groups, and each specification
group consists of multiple working groups. Given the large number of existing
specifications with diverse topics, Lina et al. proposed to use LLM models for
specification classification, classifying the text into an existing working
group automatically[13]. Fig.8 summarized the key processes of using LLM
models to classify the 3GPP specifications. With proper pre-processing and
fine-tuning, the experiment in [13] can achieve an 80% accuracy even if only
20% text segments are used. The experiment results also prove that increasing
the length of technical text segments can significantly improve classification
accuracy.

Textual descriptions and documents are frequently involved in the telecom
industry, e.g., user comments, standard specifications, technical and
troubleshooting reports, etc. Incorporating LLM models for text processing and
classification will contribute to more intelligent and reliable telecom
networks.

###  V-D Image Classification

Computer vision is an important approach for environment sensing, and there
have been many existing studies toward vision-aided 6G networks. For instance,
vision-aided blockage prediction and beamforming are investigated in [25] and
[146]. Specifically, the authors assume that the cameras attached to the BS
can capture the environment image and then use deep learning to detect objects
and 3D user locations. These studies have shown the importance of
incorporating computer vision and image processing in telecom fields to better
sense the wireless environment. Therefore, efficient image processing and
object classification are the prerequisites for realizing vision-aided
wireless networks. For example, Civelek et al. [147] proposed an automated
moving object classification technique in wireless multimedia sensor networks,
and such schemes can also be exploited in previous studies such as [25] and
[146] for efficient object detection. In addition, Kim et al. propose an edge-
network-assisted real-time object detection framework [148]. Specifically, the
vehicles can compress the image based on the region of interest and transmit
the compressed one to the edge cloud. Considering the limited computation
resources at the BS, this can be a useful technique for BS-edge-cloud image
processing and environment sensing.

The wireless environment can be very complicated with walking pedestrians,
moving vehicles, building blockages, and other obstacles. Therefore, it
requires dedicated model training and fine-tuning to extract useful
information and identify specific objects. LLM models have been pre-trained on
a huge amount of real-world datasets, and some LLMs, such as flamingo [136]
and GPT-4V [149], have proved versatile capabilities on various vision-related
tasks, e.g., using text to generate images, describing given images, and
object detection. For instance, Matsuura et al. investigate the zero-shot
image classification capabilities of the LLaVA model [26], and they found that
the performance can be significantly improved with a combination of carefully