# Speaker-Guided Encoder-Decoder Framework for Emotion Recognition in
Conversation

Yinan Bao1,2 Qianwen Ma1,2 Lingwei Wei1,2 Wei Zhou1,&Songlin Hu1,2 1Institute
of Information Engineering, Chinese Academy of Sciences
2School of Cyber Security, University of Chinese Academy of Sciences
{baoyinan, maqianwen, weilingwei, zhouwei<EMAIL_ADDRESS>Corresponding
author.

###### Abstract

The emotion recognition in conversation (ERC) task aims to predict the emotion
label of an utterance in a conversation. Since the dependencies between
speakers are complex and dynamic, which consist of intra- and inter-speaker
dependencies, the modeling of speaker-specific information is a vital role in
ERC. Although existing researchers have proposed various methods of speaker
interaction modeling, they cannot explore dynamic intra- and inter-speaker
dependencies jointly, leading to the insufficient comprehension of context and
further hindering emotion prediction. To this end, we design a novel speaker
modeling scheme that explores intra- and inter-speaker dependencies jointly in
a dynamic manner. Besides, we propose a Speaker-Guided Encoder-Decoder (SGED)
framework for ERC, which fully exploits speaker information for the decoding
of emotion. We use different existing methods as the conversational context
encoder of our framework, showing the high scalability and flexibility of the
proposed framework. Experimental results demonstrate the superiority and
effectiveness of SGED.

##  1 Introduction

Emotion recognition in conversation (ERC) aims to identify the emotion of each
utterance in a conversation. Due to its potential applications in creating
empathetic dialogue systems Zhou et al. (2018), social media analysis Li et
al. (2019), and so on, it has been concerned by a considerable number of
researchers recently.

Figure 1: (a) Speaker modeling scheme of existing dynamic speaker-specific
modeling methods, which is based on speaker identity to explore intra-speaker
dependency. (b) Our novel speaker modeling scheme, which uses the speaker
state encoder (SSE) to explore intra- and inter-speaker dependencies
dynamically. $u_{i}$ means the $i_{th}$ utterance in a dialogue and $s_{i}$
means the corresponding speaker state vector of $u_{i}$.

Unlike vanilla emotion recognition of sentences, emotions expressed in a
conversation are dynamic and can be affected by many factors such as
surrounding conversational context, and speaker dependencies Poria et al.
(2019b). Consequently, ERC models require a strong ability to model context,
select crucial information, and capture speaker dependencies, which is more
challenging. Among all the factors, speaker information is vital for tracking
the emotional dynamics of conversations, which consists of two important
aspects: intra- and inter-speaker dependencies Poria et al. (2019a). Intra-
speaker dependency, also known as emotional inertia, deals with the aspect of
emotional influence that speakers have on themselves during conversations
Kuppens et al. (2010). In contrast, inter-speaker dependency refers to the
emotional influences that the counterparts induce into a speaker Poria et al.
(2019c). Exploring intra-speaker dependency and inter-speaker dependency
jointly is conducive to comprehending the complex dialogue context that
implies various emotions.

Numerous approaches have been proposed for ERC with a focus on intra- and
inter-speaker dependencies modeling. They can be divided into two categories:
static speaker-specific modeling Zhang et al. (2019); Ghosal et al. (2019); Li
et al. (2020); Shen et al. (2021a) and dynamic speaker-specific modeling
Majumder et al. (2019); Hu et al. (2021). For the static speaker-specific
modeling methods, they utilize speaker information attaching to each utterance
to specify different connections between utterances. In this way, the injected
prior speaker information is static and shows the limited ability to
facilitate conversational context modeling. Intuitively, the speaker state
that is affected by the intra- and inter-speaker dependencies, changes
dynamically during a dialogue. Different states of a certain speaker greatly
affect various emotional expressions. Thus, it is necessary to implement
dynamic speaker-specific modeling.

As shown in Figure 1(a), existing dynamic speaker-specific modeling methods
Majumder et al. (2019); Hu et al. (2021) design speaker-specific recurrent
neural networks (RNNs) to capture self-dependencies between adjacent
utterances of the same speaker. In this scheme, utterances are classified
based on speaker identity. However, sequences composed of separate utterances
attaching to the same speaker are disjunctive and incomplete, which hinder the
comprehension of the context. Moreover, overreliance on the speaker identity
hinders the modeling of dynamic inter-speaker dependency. In this way, the
model can’t explore dynamic interactions between different speakers, which
greatly affects the psychology of speakers, and then results in different
emotional expressions. Nonetheless, how to explore intra- and inter-speaker
dependencies that change dynamically is still unresolved.

To alleviate the problem, we design a novel speaker modeling scheme that
explores intra- and inter-speaker dependencies dynamically. Besides, we
propose a speaker-guided encoder-decoder framework (SGED), which can combine
with various existing methods. The encoder module consists of two parts:
conversational context encoder (CCE) and speaker state encoder (SSE). We use
different existing methods which ignore dynamic speaker dependencies modeling
as the CCE. For the SSE, we design a novel speaker modeling scheme to generate
a dynamic speaker state sequence that depends on the speaker’s previous state
(intra-speaker dependency) and other neighboring speakers’ states (inter-
speaker dependency). For the decoder module based on RNN, we use speaker
states to guide the decoding of utterances’ emotions step by step. Extensive
experiments indicate the effectiveness of SGED. In summary, our contributions
are three-fold:

  * •

To fully exploit the complex interactions between speakers that are vital for
context comprehension, we propose a speaker-guided encoder-decoder framework
(SGED) for ERC, formulating the modeling of speaker interactions as a flexible
component.

  * •

We are the first to explore dynamic intra- and inter-speaker dependencies
jointly through a subtly designed speaker state encoder, which explores
complex interactions between speakers effectively.

  * •

Experimental results on three benchmark datasets demonstrate the effectiveness
and high scalability of SGED.

##  2 Related Work

Recently, ERC has received increasing attention from the NLP community. Unlike
traditional emotion recognition tasks, ERC models should be endowed with the
ability of speaker interaction modeling and conversational context modeling.

Speaker Interaction Modeling According to whether the speaker-specific
information is modeled dynamically during the dialogue, speaker modeling
methods can be divided into two categories: static and dynamic speaker-
specific modeling. For the static methods, some of them Ghosal et al. (2019);
Zhang et al. (2019) treat dialogue as a graph and inject prior speaker
information as different relations between utterances or treat speakers as
nodes in the graph. DialogXL Shen et al. (2021a) changes the masking
strategies of self-attention to capture intra- and inter-speaker dependencies.
HiTrans Li et al. (2020) exploits an auxiliary task to classify whether two
utterances belong to the same speaker to make the model speaker-sensitive.
However, these methods inject prior static speaker information into the model,
neglecting that speaker states change dynamically according to the varying
intra- and inter-speaker dependencies along with the dialogue. By contrast,
DialogueRNN Majumder et al. (2019) and DialogueCRN Hu et al. (2021) utilize
RNNs to update speakers’ states in a dynamic manner. Nonetheless, they only
consider intra-speaker dependency, ignoring inter-speaker dependency because
of the limited speaker modeling scheme. Thus, we propose a speaker-guided
encoder-decoder framework, which explores dynamic intra- and inter-speaker
dependencies jointly.

Conversational Context Modeling According to the methods of conversational
context modeling, existing models can be mainly divided into graph-based
methods Ghosal et al. (2019); Zhang et al. (2019); Shen et al. (2021a); Zhong
et al. (2019) and recurrence-based methods Majumder et al. (2019); Hu et al.
(2021); Jiao et al. (2019); Hazarika et al. (2018). Recently, DAG-ERC Shen et
al. (2021b) is proposed to model the dialogue in the form of a directed
acyclic graph, which allows the advantages of graph-based methods and
recurrence-based methods to complement each other. Besides, some methods have
exploited external commonsense knowledge to enrich the context representation
learning Zhong et al. (2019); Ghosal et al. (2020). We use different methods
as the context encoder, proving the effectiveness and high scalability of our
proposed speaker-guided encoder-decoder framework.

Figure 2: An overview of SGED. SSE represents the speaker state encoder. As
shown in the right part of the figure, we take $u_{6}$ as an example to show
how to obtain the speaker state vector of it.

##  3 Preliminary

###  3.1 Problem Definition

A conversation containing $N$ utterances is denoted as
$C={\\{u_{1},u_{2},...,u_{N}}\\}$. Each utterance $u_{i}$ consists of $n_{i}$
tokens, namely $u_{i}={\\{x_{i1},x_{i2},...,x_{in_{i}}}\\}$. There are $S$
speakers $s_{1},s_{2},...,s_{S}(S\geq 2)$ in a dialogue. Utterance $u_{i}$ is
uttered by speaker $s_{\phi(u_{i})}$ where the function $\phi$ maps $u_{i}$ to
its corresponding speaker. If $u_{i}$ isn’t the first utterance spoken by
$s_{\phi(u_{i})}$ in the conversation, we denote $u_{\psi(u_{i})}$ as the last
previous utterance expressed by the same speaker where $0<\psi(u_{i})<i$. We
denote ${\\{u_{j}|\forall{j},\psi(u_{i})\leq j<i}\\}$ as the local information
for $u_{i}$. ERC task aims to predict the emotion label $y_{i}$ for each
utterance $u_{i}$ in the conversation $C$.

###  3.2 Textual Feature Extraction

By convention, we use the pre-trained language model to extract utterance-
level representations. Following Shen et al Shen et al. (2021b), we fine-tune
the pre-trained language model RoBERTa-Large Liu et al. (2019) on each ERC
dataset first and then freeze its parameters while training our proposed
framework. Concretely, we first append a special token [CLS] at the beginning
of a given utterance $u_{i}$ to obtain the input sequence
${\\{[CLS],x_{i1},x_{i2},...,x_{in_{i}}}\\}$. Afterward, we extract a
representation of $u_{i}$ from the [CLS]’s embedding of the last layer with a
dimension of 1,024.

##  4 Methodology

As illustrated in Figure 2, there are three components in our proposed
framework SGED: (1) conversational context encoder; (2) speaker state encoder;
(3) speaker-guided decoder.

###  4.1 Conversational Context Encoder

We take the existing baseline as the conversation context encoder of SGED,
which is easily replaceable.111In the experiments, we take different existing
methods, namely bc-LSTM+att Poria et al. (2017), DialogueGCN Ghosal et al.
(2019), and DAG-ERC Shen et al. (2021b), for the conversational context
encoder. For a given dialogue $C$, we generate the utterance representation
vector sequence
$\mathbf{H}=[\mathbf{h}_{1},\mathbf{h}_{2},...,\mathbf{h}_{N}]$ as follows:

$\begin{split}&\mathbf{H}=Encoder(C)\,,\\\ \end{split}$ (1)

where $Encoder$ means the replaceable conversational context encoder.

###  4.2 Speaker State Encoder

Afterward, we generate a speaker state vector for each utterance based on the
utterance representations from the conversational context encoder. The dynamic
speaker state has two factors: intra-speaker dependency and inter-speaker
dependency. We take the sequence of utterance representations as input and
model the two kinds of dependencies simultaneously to obtain the final speaker
state vectors sequentially.

####  4.2.1 Intra-Speaker Dependency Modeling

For a given utterance $u_{i}$ which is uttered by speaker $s_{\phi(u_{i})}$,
we take the last previous speaker state of $s_{\phi(u_{i})}$ and previous
context
$\mathbf{c}_{i}=[\mathbf{h}_{1},\mathbf{h}_{2},...,\mathbf{h}_{i}]\in\mathbb{R}^{h\times
i}$ as input, modeling the intra-speaker dependency implied in the dialogue.

First, we utilize the last previous speaker state vector of $s_{\phi(u_{i})}$
and the representation vector of $u_{i}$ to obtain the query vector
$\mathbf{q}^{intra}_{i}$ as follows:

$\begin{split}&\mathbf{q}^{intra}_{i}=\mathbf{W}^{intra}_{q}[\mathbf{v}_{\psi(u_{i})}||\mathbf{h}_{i}]+\mathbf{b}^{intra}_{q}\,,\\\
\end{split}$ (2)

where $\mathbf{v}_{\psi(u_{i})}\in\mathbb{R}^{h\times 1}$ means the last
previous speaker state vector of $s_{\phi(u_{i})}$; $||$ means the
concatenation operation; $\mathbf{W}^{intra}_{q}\in\mathbb{R}^{h\times 2h}$
and $\mathbf{b}^{intra}_{q}\in\mathbb{R}^{1}$ are model parameters.

Then, we use $\mathbf{q}^{intra}_{i}$ as the query and previous context
$\mathbf{c}_{i}$ as the key and value to implement the attention mechanism,
generating the intra-speaker state vector of $u_{i}$ as follows:

$\begin{split}&\alpha^{intra}_{i}=\mathbf{softmax}(\mathbf{W}_{1}({\mathbf{q}^{intra}_{i}}\odot\mathbf{c}_{i})+\mathbf{b}_{1})\,,\\\
&\mathbf{v}^{intra}_{i}=\alpha^{intra}_{i}\mathbf{c}_{i}^{T}\,,\\\
\end{split}$ (3)

where $\odot$ is an element-wise product operation;
$\mathbf{W}_{1}\in\mathbb{R}^{1\times h}$ and
$\mathbf{b}_{1}\in\mathbb{R}^{1}$ are model parameters;
$\alpha^{intra}_{i}\in\mathbb{R}^{1\times i}$ is the attention score to
aggregate the previous context; $\mathbf{v}^{intra}_{i}\in\mathbb{R}^{1\times
h}$ is the intra-speaker state vector of $u_{i}$.

####  4.2.2 Inter-Speaker Dependency Modeling

Simultaneously, we treat the representation vector of $u_{i}$ and the speaker
state vectors of local information ${\\{u_{j}|\forall{j},\psi(u_{i})\leq
j<i}\\}$ as input, modeling the latent inter-speaker dependency.

We adopt the following methods to generate the inter-speaker state vector of
$u_{i}$:

$\begin{split}&\mathbf{q}^{inter}_{i}=\mathbf{W}^{inter}_{q}\mathbf{h}_{i}+\mathbf{b}^{inter}_{q}\,,\\\
&\alpha^{inter}_{i}=\mathbf{softmax}(\mathbf{W}_{2}({\mathbf{q}^{inter}_{i}}\odot\mathbf{k}_{i})+\mathbf{b}_{2})\,,\\\
&\mathbf{v}^{inter}_{i}=\alpha^{inter}_{i}\mathbf{k}_{i}^{T}\,,\\\
\end{split}$ (4)

where $\mathbf{k}_{i}={\\{\mathbf{v}_{j}|\forall{j},\psi(u_{i})\leq
j<i}\\}\in\mathbb{R}^{h\times l}$ denotes the speaker state vectors of
previous local information; $\odot$ means the element-wise product operation;
$\mathbf{W}^{inter}_{q}\in\mathbb{R}^{h\times h}$,
$\mathbf{b}^{inter}_{q}\in\mathbb{R}^{1}$,
$\mathbf{W}_{2}\in\mathbb{R}^{1\times h}$, and
$\mathbf{b}_{2}\in\mathbb{R}^{1}$ are model parameters;
$\mathbf{v}^{inter}_{i}\in\mathbb{R}^{1\times h}$ is the inter-speaker state
vector of $u_{i}$.

Finally, we fuse the intra-speaker state vector $\mathbf{v}^{intra}_{i}$ and
the inter-speaker state vector $\mathbf{v}^{inter}_{i}$ to generate the final
speaker state vector of $u_{i}$:

$\begin{split}&\mathbf{v}_{i}=\mathbf{tanh}(\mathbf{v}^{intra}_{i}+\mathbf{v}^{inter}_{i})\,,\\\
\end{split}$ (5)

where $\mathbf{v}_{i}\in\mathbb{R}^{1\times h}$ is the speaker state vector of
$u_{i}$. Note that if $u_{i}$ is the first utterance expressed by speaker
$s_{\phi(u_{i})}$ in the dialogue, we let $\mathbf{v}_{i}=\mathbf{h}_{i}$. For
the given dialogue $C$, we obtain the speaker state vector sequence
$\mathbf{V}={\\{\mathbf{v}_{1},\mathbf{v}_{2},...,\mathbf{v}_{N}}\\}$, which
encodes speaker-specific intra-speaker dependency and inter-speaker dependency
dynamically.

###  4.3 Speaker-Guided Decoder

To fully utilize the vital speaker-specific information implied in the
conversation, we use the speaker state vector $\mathbf{v}_{i}$ to guide the
decoding of $u_{i}$’s emotion. In addition, inherent relations between emotion
tags also facilitate ERC Wang et al. (2020). Therefore, we also use the
embedding of the predicted emotion of $u_{i-1}$ to instruct the decoding of
$u_{i}$’s emotion. We subtly design a speaker-guided emotion decoder based on
(gated recurrent unit) GRU to decode each utterance’s emotion in a given
dialogue sequentially.

Given an utterance $u_{i}$, we first match the speaker state vector
$\mathbf{v}_{i}$ with the utterance representation vector $\mathbf{h}_{i}$ as
follows:

$\begin{split}&\mathbf{m}_{i}=\mathbf{ReLU}(\mathbf{v}_{i}\odot(\mathbf{W}_{m}\mathbf{h}_{i}+\mathbf{b}_{m})^{T})\,,\\\
\end{split}$ (6)

where $\mathbf{W}_{m}\in\mathbb{R}^{h\times h}$ and
$\mathbf{b}_{m}\in\mathbb{R}^{1}$ are trainable parameters for feature
transformation; $\mathbf{m}_{i}\in\mathbb{R}^{1\times h}$ is the generated
match vector.

Afterward, we concatenate the match vector $\mathbf{m}_{i}$ and the embedding
$\mathbf{e}_{i-1}$ of the predicted emotion of $u_{i-1}$ and then feed it to
GRU:

$\begin{split}&\mathbf{o}_{i}=\overrightarrow{\mathbf{GRU}}([\mathbf{m}_{i}||\mathbf{e}_{i-1}],\mathbf{o}_{i-1})\,.\\\
\end{split}$ (7)

Finally, we take the concatenation of $\mathbf{h}_{i}\in\mathbb{R}^{h\times
1}$ and $\mathbf{o}_{i}\in\mathbb{R}^{1\times h}$ as the final representation
of $u_{i}$, and feed it to a feed-forward neural network for emotion
prediction:

$\begin{split}&\mathbf{z}_{i}=\mathbf{ReLU}(\mathbf{W}_{o}[\mathbf{h}_{i}||\mathbf{o}_{i}^{T}]+\mathbf{b}_{o})\,,\\\
&\mathcal{P}_{i}=\mathbf{softmax}(\mathbf{W}_{z}\mathbf{z}_{i}+\mathbf{b}_{z})\,,\\\
&\hat{y}_{i}=\operatorname*{argmax}_{g}\mathcal{P}_{i}[g]\,,\\\
&\mathbf{e}_{i}=\mathbf{E}[\hat{y}_{i}]\,,\\\ \end{split}$ (8)

where $\mathbf{W}_{o}\in\mathbb{R}^{h\times 2h}$,
$\mathbf{W}_{z}\in\mathbb{R}^{L\times h}$,
$\mathbf{b}_{o},\mathbf{b}_{z}\in\mathbb{R}^{1}$ are trainable parameters; $L$
is the class number of the dataset; $\hat{y}_{i}$ is the predicted label of
$u_{i}$; $\mathbf{E}$ is the look-up table of emotion embeddings which is
initialized randomly at the beginning; $\mathbf{e}_{i}$ is the embedding of
the predicted emotion of $u_{i}$.

To train the model, we use cross-entropy loss function:

$\begin{split}&\mathcal{L}(\theta)=-\sum_{j=1}^{M}\sum_{t=1}^{N_{j}}log\mathcal{P}_{j,t}[y_{j,t}]\,,\\\
\end{split}$ (9)

where $M$ is the number of dialogues in the training set; $N_{j}$ is the
number of utterances in the $j$-th dialogue; $y_{j,t}$ is the ground-truth
label; $\theta$ is the collection of trainable parameters.

##  5 Experimental Settings

###  5.1 Datasets

We evaluate our proposed framework on three ERC benchmark datasets. The
statistics of them are shown in Table 1.

IEMOCAP Busso et al. (2008): A two-party conversation dataset for ERC, which
includes 6 types of emotion, that is neutral, happiness, sadness, anger,
frustrated, and excited. Since this dataset has no validation set, we follow
Shen et al. (2021b) to use the last 20 conversations in the training set for
validation.

MELD Poria et al. (2019a): A multi-party conversation dataset collected from
the TV series Friends. The emotion labels are: neutral, happiness, surprise,
sadness, anger, disgust, and fear.

EmoryNLP Zahiri and Choi (2018): A multi-party conversation dateset collected
from Friends, but varies from MELD in the choice of scenes and emotion labels.
There are 7 types of emotion, namely neutral, sad, mad, scared, powerful,
peaceful, and joyful.

Although all of the datasets contain multimodal information, we only focus on
the textual information for experiments. Following existing work Shen et al.
(2021b), we choose weighted-average F1 for evaluation metrics.

Dataset | Conversations | Utterances
---|---|---
Train | Val | Test | Train | Val | Test
IEMOCAP | 120 | 31 | 5810 | 1623
MELD | 1038 | 114 | 280 | 9989 | 1109 | 2610
EmoryNLP | 713 | 99 | 85 | 9934 | 1344 | 1328
Table 1: The statistics of three datasets.

###  5.2 Compared Methods

We compare our method with the following baselines.

KET Zhong et al. (2019) is a knowledge-enriched transformer that extracts
commonsense knowledge based on a graph attention mechanism. COSMIC Ghosal et
al. (2020) incorporates different elements of commonsense and leverages them
to learn self-speaker dependency. DialogueRNN Majumder et al. (2019) uses
speaker-specific GRU to model intra-speaker dependency and generate speaker
states sequentially in a dynamic manner. Another dynamic speaker-specific
modeling method is DialogueCRN Hu et al. (2021), which utilizes speaker-level
context to model dynamic intra-speaker dependency and designs a multi-turn
reasoning module to imitate human cognitive thinking. bc-LSTM+att Poria et al.
(2017) is a contextual LSTM network without speaker information modeling,
which is based on an attention mechanism to re-weight the features.
DialogueGCN Ghosal et al. (2019) is a graph-based method that treats speaker
dependencies as static information and designs different relations between
utterances. DAG-ERC Shen et al. (2021b) models a dialogue in the form of a
directed acyclic graph and utilizes speaker identity to establish connections
between utterances.

We use the same feature extractor RoBERTa Liu et al. (2019) as provided by
Shen et al., Shen et al. (2021b). We replace the textual feature in the
previous methods with our extracted features first. Then, we use previous
methods without dynamic speaker-specific information modeling (RoBERTa, bc-
LSTM+att, DialogueGCN, DAG-ERC222Note that we use the one-layer DAG-ERC as the
context encoder, making the parameters fewer than the original DAG-ERC with
multiple layers. In this way, the potential impact of increasing parameters
provided by the decoder is eliminated. Our method with fewer parameters even
surpasses the original DAG-ERC with more parameters, proving the effectiveness
of our method more convincingly.) as the conversational context encoder of the
proposed speaker-guided encoder-decoder framework, investigating the
effectiveness of our speaker modeling scheme.

###  5.3 Implementation Details

We utilize the validation set to tune parameters on each dataset and adopt
Adam Kingma and Ba (2015) as the optimizer. The learning rate and batch size
are selected from {1e-5, 5e-5, 1e-4, 5e-4} and {8, 16, 32}, respectively. The
dimension of hidden vector $h$ is set to 300 and the dimension of label
embedding is set to 100. The feature size for the RoBERTa extractor is 1,024.
We use the fine-tuned RoBERTa features provided by Shen et al., Shen et al.
(2021b). Each training process contains 60 epochs. The results of our proposed
SGED framework reported in the experiments are all based on the average score
of 5 random runs on the test set.

Model | MELD | EmoryNLP | IEMOCAP
---|---|---|---
KET | 58.18 | 33.95 | 59.56
COSMIC | 65.21 | 38.11 | 65.28
DialogueRNN | 57.03 | - | 62.75
\+ RoBERTa | 63.61 | 37.44 | 64.76
DialogueCRN | 58.39 | - | 66.20
\+ RoBERTa | 63.42 | 38.91 | 66.46
RoBERTa | 62.88 | 37.78 | 63.38
SGED + RoBERTa | 63.34 | 38.47 | 64.11
bc-LSTM+att | - | - | -
\+ RoBERTa | 62.95 | 38.28 | 64.51
SGED + bc-LSTM+att | 63.37 | 38.89 | 65.03
DialogueGCN | 58.10 | - | 64.18
\+ RoBERTa | 63.02 | 38.10 | 64.91
SGED + DialogueGCN | 64.55 | 39.73 | 65.90
DAG-ERC | 63.65 | 39.02 | 68.03
DAG-ERC* | 63.39 | 38.84 | 67.45
SGED + DAG-ERC* | 65.46 | 40.24 | 68.53
Table 2: Overall performance on the three datasets. We choose weighted-average F1 to evaluate each method. DAG-ERC* means that we use the one-layer DAG-ERC. Method | EmoryNLP | IEMOCAP
---|---|---
SGED + DAG-ERC* | 40.24 | 68.53
w/o SSE \- intra-speaker | 39.87 ($\downarrow$0.37) | 68.05 ($\downarrow$0.48)
w/o SSE \- inter-speaker | 39.86 ($\downarrow$0.38) | 67.72 ($\downarrow$0.81)
w/o SSE | 39.17 ($\downarrow$1.07) | 67.67 ($\downarrow$0.86)
w/o SGD | 38.96 ($\downarrow$1.28) | 67.30 ($\downarrow$1.23)
w/o SSE + SGD | 38.84 ($\downarrow$1.40) | 67.45 ($\downarrow$1.08)
Table 3: Results of ablation study on EmoryNLP (multi-party) and IEMOCAP (two-
party) dataset. We use the one-layer DAG-ERC as the conversational context
encoder of our framework.

##  6 Results and Analysis

###  6.1 Overall Results

Overall results of our proposed method and the baselines are reported on Table
2. The results demonstrate that methods combined with our speaker-guided
encoder-decoder framework achieve significant improvement.

As shown in Table 2, based on the same feature extractor RoBERTa, the
framework proposed by us outperforms the other dynamic speaker-specific
information modeling methods (DialogueRNN, DialogueCRN) in general. Moreover,
SGED + RoBERTa which encodes each utterance independently in the context
encoder, achieves comparable results with DialogueRNN+RoBERTa, proving the
effectiveness of the SGED proposed by us. Taking different types of existing
methods as the conversational context encoder, SGED which combines the novel
speaker modeling scheme achieves significant improvement all the time. It
indicates the high scalability and flexibility of SGED. Moreover, SGED + DAG-
ERC* reaches the new state-of-the-art on the three benchmark datasets, even
surpasses COSMIC which utilizes external commonsense knowledge.

MELD and EmoryNLP: On these multi-party conversation datasets, we observe that
our SGED framework makes varying degrees of promotion based on different
conversational context encoders. For the recurrence-based method bc-LSTM+att
and context-independent method RoBERTa, the promotion brought by the SGED
framework is about 0.55% on average. However, for the graph-based methods
DialogueGCN and DAG-ERC, our SGED framework achieves an increase of 1.66% on
average. It is because the speaker state encoder of SGED is established based
on the conversational context encoder. Thus, the context comprehension ability
of the conversational context encoder affects the subsequent dynamic speaker-
specific information modeling, and finally influences the decoding of emotion.
The stable improvement brought by SGED indicates that the proposed dynamic
speaker modeling scheme helps to explore crucial dependencies between speakers
and benefits ERC.

IEMOCAP: On the two-party conversation dataset, we observe that the
improvement brought by SGED has little correlation with the type of context
encoder, which is different from the results on the multi-party conversation
datasets. Besides, the improvement of SGED on IEMOCAP is 0.83% on average,
which is slightly lower than the improvements on MELD and EmoryNLP. We argue
that the dependencies between speakers in multi-party conversations are more
complex and intractable. Therefore, it is more necessary to exploit SGED to
model speaker-specific states dynamically for multi-party conversations.

Figure 3: (a) The weighted-average F1 score of conversations with the
different number of speakers on MELD, achieving by DialogueGCN and our SGED
framework which takes DialogueGCN as the conversational context encoder. (b)
An abridged dialogue from MELD in which our SGED framework recognizes most of
the emotions correctly. (c) The confusion matrix of conversations with 6
speakers excluding the diagonal on MELD, is achieved by our SGED framework
which takes DialogueGCN as the conversational context encoder.

###  6.2 Ablation Study

To investigate the effects of different modules in SGED, we conduct several
ablation studies on EmoryNLP and IEMOCAP. We remove 1) intra-speaker
dependency in speaker state encoder (SSE), 2) inter-speaker dependency in SSE,
3) SSE, 4) speaker-guided decoder (SGD), and 5) SSE and SGD, respectively. The
results are reported on Table 3.

##### Effect of Different Dynamic Dependencies between Speakers.

As shown in Table 3, the result on EmoryNLP declines when removing either
intra-speaker dependency or inter-speaker dependency in the SSE. However, SGED
without inter-speaker dependency performs worse than without intra-speaker
dependency on IEMOCAP. It proves that both dynamic intra- and inter-speaker
dependencies are important for multi-party conversations while dynamic inter-
speaker dependency is more important for two-party conversations. Moreover,
the result drops sharply when eliminating the entire speaker state encoder
(SSE) that explores both intra- and inter-speaker dependencies in a dynamic
manner, which indicates that the two kinds of dynamic dependency are
complementary to each other.

##### Effect of Speaker-Guided Decoder.

To evaluate the effectiveness of the speaker-guided decoder (SGD), we remove
it and only use the speaker state vector for prediction. As shown in Table 3,
it is clear that the speaker-guided decoder proposed by us boosts model
performance a lot. Removing the SGD not only has a great impact on multi-party
conversations (1.28%) but also two-party conversations (1.23%), which proves
that the aggregation of speaker state feature and semantic feature benefits
emotion recognition in conversation.

##### Effect of the Novel Speaker Modeling Scheme.

Our proposed speaker modeling scheme consists of speaker state encoder (SSE)
and speaker-guided decoder (SGD). As reported on Tabel 3, removing SSE and SGD
simultaneously results in a significant drop. It indicates the effectiveness
of the proposed speaker modeling scheme, which can capture dynamic speaker
interactions and promote emotion prediction.

###  6.3 Discussion

As shown in Figure 3 (a), we illustrate the results of dialogues with the
different number of speakers on MELD, achieved by DialogueGCN and our SGED
framework which takes DialogueGCN as the conversational context encoder. We
observe that our proposed method exceeds DialogueGCN on dialogues with
different speaker numbers except 6. Compared with DialogueGCN, our method
achieves the greatest promotion on dialogues with 5 speakers. We will take an
example with 5 speakers for the case study and conduct an error analysis on
dialogues with 6 speakers.

####  6.3.1 Case Study

In Figure 3 (b), we present an abridged dialogue that has 5 speakers
originally and our method recognizes most emotions correctly. For person A, we
recognize $u_{3}$ as anger correctly though it is a short utterance that is
intractable. We argue that the intra-speaker dependency of A captured by our
method facilitates the recognition. Moreover, although person A’s previous
utterances ($u_{1},u_{2},u_{3}$) all express anger, our method recognizes
$u_{7}$ as happiness which is obviously different from anger, based on the
inter-speaker dependencies among person C ($u_{5}$), B ($u_{6}$), and A
($u_{7}$). For person B, we recognize $u_{8}$ as happiness correctly, due to
the trade-off of inter-speaker dependency between person A ($u_{7}$) and B
($u_{8}$) and intra-speaker dependency of B captured by our method. It
indicates that SGED models dynamic speaker states effectively with integration
of intra-speaker dependency and inter-speaker dependency.

####  6.3.2 Error Analysis

In Figure 3 (c), we present the confusion matrix of conversations with 6
speakers excluding the diagonal, achieved by our method. From the heat map,
neutral samples are more confused with non-neutral (e.g., happiness) samples,
which can be alleviated by adding cues from other modalities. In addition to
this, distinguishing similar emotions (e.g., happiness and surprise) still
disturbs our method.

##  7 Conclusion

In this paper, we propose a speaker-guided encoder-decoder framework (SGED)
for ERC, which captures dynamic intra- and inter-speaker dependencies to
benefit the understanding of the complex context. In this framework, we
propose a novel speaker state encoder (SSE) and design a speaker-guided
decoder (SGD). SSE is competent at exploring intra- and inter-speaker
dependencies dynamically. Moreover, we treat different existing methods as the
conversational context encoder of SGED, showing its high scalability and
flexibility. Experiments on three benchmark datasets prove the superiority and
effectiveness of our method. Especially, our method achieves a more
significant improvement on the multi-party conversation datasets, indicating
its strong ability to model dynamic multi-speaker states during a dialogue.

## References

  * Busso et al. [2008] Carlos Busso, Murtaza Bulut, Chi-Chun Lee, Abe Kazemzadeh, Emily Mower, Samuel Kim, Jeannette N. Chang, Sungbok Lee, and Shrikanth S. Narayanan.  IEMOCAP: interactive emotional dyadic motion capture database.  Lang. Resour. Evaluation, 42(4):335–359, 2008.
  * Ghosal et al. [2019] Deepanway Ghosal, Navonil Majumder, Soujanya Poria, Niyati Chhaya, and Alexander F. Gelbukh.  Dialoguegcn: A graph convolutional neural network for emotion recognition in conversation.  In EMNLP-IJCNLP, pages 154–164, 2019.
  * Ghosal et al. [2020] Deepanway Ghosal, Navonil Majumder, Alexander F. Gelbukh, Rada Mihalcea, and Soujanya Poria.  COSMIC: commonsense knowledge for emotion identification in conversations.  In Trevor Cohn, Yulan He, and Yang Liu, editors, EMNLP, volume EMNLP 2020 of Findings of ACL, pages 2470–2481, 2020.
  * Hazarika et al. [2018] Devamanyu Hazarika, Soujanya Poria, Rada Mihalcea, Erik Cambria, and Roger Zimmermann.  ICON: interactive conversational memory network for multimodal emotion detection.  In EMNLP, pages 2594–2604, 2018.
  * Hu et al. [2021] Dou Hu, Lingwei Wei, and Xiaoyong Huai.  Dialoguecrn: Contextual reasoning networks for emotion recognition in conversations.  In ACL/IJCNLP, pages 7042–7052, 2021.
  * Jiao et al. [2019] Wenxiang Jiao, Haiqin Yang, Irwin King, and Michael R. Lyu.  Higru: Hierarchical gated recurrent units for utterance-level emotion recognition.  In NAACL-HLT, pages 397–406, 2019.
  * Kingma and Ba [2015] Diederik P. Kingma and Jimmy Ba.  Adam: A method for stochastic optimization.  In ICLR, 2015.
  * Kuppens et al. [2010] Peter Kuppens, Nicholas B Allen, and Lisa B Sheeber.  Emotional inertia and psychological maladjustment.  Psychological science, 21(7):984–991, 2010.
  * Li et al. [2019] Runnan Li, Zhiyong Wu, Jia Jia, Yaohua Bu, Sheng Zhao, and Helen Meng.  Towards discriminative representation learning for speech emotion recognition.  In IJCAI, pages 5060–5066, 2019.
  * Li et al. [2020] Jingye Li, Donghong Ji, Fei Li, Meishan Zhang, and Yijiang Liu.  Hitrans: A transformer-based context- and speaker-sensitive model for emotion detection in conversations.  In COLING, pages 4190–4200, 2020.
  * Liu et al. [2019] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov.  Roberta: A robustly optimized BERT pretraining approach.  CoRR, abs/1907.11692, 2019.
  * Majumder et al. [2019] Navonil Majumder, Soujanya Poria, Devamanyu Hazarika, Rada Mihalcea, Alexander F. Gelbukh, and Erik Cambria.  Dialoguernn: An attentive RNN for emotion detection in conversations.  In AAAI, pages 6818–6825, 2019.
  * Poria et al. [2017] Soujanya Poria, Erik Cambria, Devamanyu Hazarika, Navonil Majumder, Amir Zadeh, and Louis-Philippe Morency.  Context-dependent sentiment analysis in user-generated videos.  In ACL, pages 873–883, 2017.
  * Poria et al. [2019a] Soujanya Poria, Devamanyu Hazarika, Navonil Majumder, Gautam Naik, Erik Cambria, and Rada Mihalcea.  MELD: A multimodal multi-party dataset for emotion recognition in conversations.  In ACL, pages 527–536, 2019.
  * Poria et al. [2019b] Soujanya Poria, Navonil Majumder, Rada Mihalcea, and Eduard H. Hovy.  Emotion recognition in conversation: Research challenges, datasets, and recent advances.  IEEE Access, 7:100943–100953, 2019.
  * Poria et al. [2019c] Soujanya Poria, Navonil Majumder, Rada Mihalcea, and Eduard H. Hovy.  Emotion recognition in conversation: Research challenges, datasets, and recent advances.  IEEE Access, 7:100943–100953, 2019.
  * Shen et al. [2021a] Weizhou Shen, Junqing Chen, Xiaojun Quan, and Zhixian Xie.  Dialogxl: All-in-one xlnet for multi-party conversation emotion recognition.  In AAAI, pages 13789–13797, 2021.
  * Shen et al. [2021b] Weizhou Shen, Siyue Wu, Yunyi Yang, and Xiaojun Quan.  Directed acyclic graph network for conversational emotion recognition.  In ACL/IJCNLP, pages 1551–1560, 2021.
  * Wang et al. [2020] Yan Wang, Jiayu Zhang, Jun Ma, Shaojun Wang, and Jing Xiao.  Contextualized emotion recognition in conversation as sequence tagging.  In SIGdial, pages 186–195, 2020.
  * Zahiri and Choi [2018] Sayyed M. Zahiri and Jinho D. Choi.  Emotion detection on TV show transcripts with sequence-based convolutional neural networks.  In AAAI, volume WS-18 of AAAI Workshops, pages 44–52, 2018.
  * Zhang et al. [2019] Dong Zhang, Liangqing Wu, Changlong Sun, Shoushan Li, Qiaoming Zhu, and Guodong Zhou.  Modeling both context- and speaker-sensitive dependence for emotion detection in multi-speaker conversations.  In IJCAI, pages 5415–5421, 2019.
  * Zhong et al. [2019] Peixiang Zhong, Di Wang, and Chunyan Miao.  Knowledge-enriched transformer for emotion detection in textual conversations.  In EMNLP-IJCNLP, pages 165–176, 2019.
  * Zhou et al. [2018] Hao Zhou, Minlie Huang, Tianyang Zhang, Xiaoyan Zhu, and Bing Liu.  Emotional chatting machine: Emotional conversation generation with internal and external memory.  In AAAI, pages 730–739, 2018.


$\mu$-Hankel Operators on Compact Abelian Groups

Adolf Mirotin

Abstract. $(\mu;\nu)$-Hankel operators between separable Hilbert spaces were
introduced and studied recently ($\mu$-Hankel operators on Hilbert spaces,
Opuscula Math., 41 (2021), 881–899). This paper, is devoted to generalization
of $(\mu;\nu)$-Hankel operators to the (non-separable in general) case of
Hardy spaces over compact and connected Abelian groups. In this setting
bounded $(\mu;\nu)$-Hankel operators are fully described under some natural
conditions. Examples of integral operators are considered.

Key wards: Hankel operator, $\mu$-Hankel operator, Hardy space, compact
Abelian group, ordered group, nuclear operator, integral operator.

Mathematics Subject Classification: 47B35, 47A62, 47B90

##  1\. Introduction

This paper is motivated by the following questions. Let $q\in\mathbb{C}$. The
generalized Cauchy transform

$\mathbf{A}f(z)=\int_{\mathbb{D}}\frac{f(q\zeta)}{z-\zeta}d\sigma(\zeta)\quad(|z|=1)$

where $\sigma$ is a finite (in general complex) measure on the unite disc
$\mathbb{D}$ is an integral Hankel operator for $q=1$. In this case the
boundedness of this operator in Hardy spaces on $\mathbb{D}$ were studied by
Widom and others. But the case $q\neq 1$, $0<|q|\leq 1$ remained unexplored.

The analogous question can be posed for the another natural generalization of
the Cauchy transform

$\mathbf{B}f(z):=\int_{\overline{\mathbb{D}}}\frac{f(\zeta)}{qz-\zeta}d\tau(\zeta)\quad(|z|=1)$

where $|q|>1$ and similar operators.

Operators mentioned above are special cases of the so-called
$(\mu;\nu)$-Hankel operators (see Examples 1–3 below). In [13]
$(\mu;\nu)$-Hankel operators $A=A_{(\mu;\nu),\alpha}$ between separable
Hilbert spaces $\mathcal{H}$ and $\mathcal{H}^{\prime}$ where $\mu$ and $\nu$
are a nonzero complex parameters, $(\alpha_{n})$ is a sequence of complex
numbers were introduced as operators with matrices of the form $\langle
Ae_{k},e^{\prime}_{j}\rangle=\mu^{k}\nu^{j}\alpha_{k+j}$ where $(e_{k})$ and
$(e^{\prime}_{j})$ are some orthonormal bases of $\mathcal{H}$ and
$\mathcal{H}^{\prime}$ respectively. Conditions of boundedness of such
operators were given. It was shown also that in the case $|\mu|\neq 1$ bounded
$\mu$-Hankel operators are nuclear and in the case $|\mu|=1$ connections with
Hankel operators were investigated.

This paper, is devoted to the non-separable case. In our opinion a convenient
setting of the problem is as follows. Consider compact and connected Abelian
group $G$ with a fixed total group ordering in its dual $X$. Let $H^{2}(G)$ be
the corresponding Hardy space on $G$ [16] and $H^{2}_{-}(G)=L^{2}(G)\ominus
H^{2}(G)$. Then the positive cone $X_{+}$ and the negative cone
$X_{-}=X\setminus X_{+}$ are orthogonal bases of $H^{2}(G)$ and $H^{2}_{-}(G)$
respectively. Since the Hilbert dimension of $H^{2}(G)$ and $H^{2}_{-}(G)$
equals to the cardinality of $X$, this spaces are non-separable for non-
metrizable $G$. Consider a pare of non-null homomorphisms
$\mu,\nu\in\mathrm{Hom}(X,\mathbb{C})$ (here $\mathbb{C}$ is considered as a
multiplicative semigroup). We define a $(\mu;\nu)$-Hankel operator between
$H^{2}(G)$ and $H^{2}_{-}(G)$ as an operator whose matrix with respect to the
bases $X_{+}$ and $X_{-}$ is of the form $(\mu(\chi)\nu(\xi)a(\chi\xi))$ (here
$\chi$ runs over $X_{+}$ and $\overline{\xi}$ runs over $X_{-}$). In this
setting bounded $(\mu;1)$-Hankel operators (from now on called $\mu$-Hankel
operators) will be fully described under some natural conditions. The general
case can be easily reduced to the case of $\mu$-Hankel operators. Examples of
integral $\mu$-Hankel operators are investigated.

In the last section we briefly discuss the class of $(1;\nu)$-Hankel operators
(called $\nu$-Hankel operators) that are in some sense ”dual” to $\mu$-Hankel
operators.

In the case $G=\mathbb{T}$ (the circle group), $X=\mathbb{Z}$ (the group of
integers) we get the situation considered in [13] but our main results are new
even in this case.

##  2\. Preliminaries

This section collects all preliminary information we need in the next parts of
the paper and the general definition of a $(\mu;\nu$-Hankel operator.

In the following unless otherwise stated $G$ stands for a compact and
connected Abelian group and a total order (which agrees with the group
structure) is fixed on its dual group $X$. In turn, $X$ is the dual group for
$G$ by the Pontryagin–van Kampen theorem.

Let $\chi_{0}\equiv 1$ be the unit character and $X_{+}:=\\{\chi\in
X:\chi\geq\chi_{0}\\}$ the positive cone in $X$. In other words, $X_{+}$ is a
subsemigroup of $X$ such that $X_{+}^{-1}\cup X_{+}=X$, and $X_{+}^{-1}\cap
X_{+}=\\{\chi_{0}\\}$ (see, e.g., [16, Chapter 8]). The inequality
$\chi\geq\xi$ for $\chi,\xi\in X$ is equivalent to the inclusion
$\chi\xi^{-1}\in X_{+}$. We put also $X_{-}:=X\setminus X_{+}$. Then
$X_{-}=X_{+}^{-1}\setminus\\{\chi_{0}\\}=\overline{X_{+}}\setminus\\{\chi_{0}\\}$
(the bar stands for the complex conjugation).

As is well known, a (discrete) Abelian group $X$ can be totally ordered if and
only if it is torsion-free, which in turn is equivalent to the condition that
its character group $G$ is connected. In general the group $X$ may possess
many different total orderings.

Note that groups $X$ for which $X_{+}$ has the smallest (first) element
$\chi_{1}$ were described in [5, Lemma 3.2]. In this case we denote by $X^{i}$
the (infinite cyclic) subgroup of $X$ generated by $\chi_{1}$.

In applications, often $X$ is a dense subgroup of $\mathbb{R}^{d}$ endowed
with the discrete topology and $G=\mathbf{b}X$ is its Bohr compactification,
or $X=\mathbb{Z}^{d}$ so that $G=\mathbb{T}^{d}$ is the $d$-torus. Other
interesting examples are the infinite dimensional torus $\mathbb{T}^{\infty}$,
an $\mathbf{a}$-adic solenoids $\Sigma_{\mathbf{a}}$, and their finite and
countable products (see, e.g., [2], [11]).

######  Definition 2.1.

Let $G$ be a compact Abelian group, $X$ its dual group. The Hardy space
$H^{2}(G)$ on $G$ is the following subspace of $L^{2}(G)$ (see, e.g., [16]):

$H^{2}(G)=\\{f\in L^{2}(G):\widehat{f}(\chi)=0\forall\chi\in X_{-}\\};$

where $\widehat{f}$ denotes the Fourier transform of a function $f\in
L^{1}(G)$ the norm and the inner product in $L^{2}(G)$ (as well as the norm
and the inner product in $H^{2}(G)$) is denoted by $\langle\cdot,\cdot\rangle$
and $\|\cdot\|_{2}$ respectively.

The orthogonal complement of $H^{2}(G)$ in $L^{2}(G)$ is denoted by
$H^{2}_{-}(G)$. Then

$H^{2}_{-}(G)=\\{f\in L^{2}(G):\widehat{f}(\xi)=0\forall\xi\in X_{+}\\}.$

The set $X_{+}$ forms an orthonormal basis in $H^{2}(G)$; while $X_{-}$ is an
orthonormal basis in $H^{2}_{-}(G)$ (the Hilbert dimension of these spaces is
equal to the cardinality of $X$). We denote by $P_{+}$ and $P_{-}=I-P_{+}$ the
orthogonal projections from $L^{2}(G)$ onto $H^{2}(G)$ and $H^{2}_{-}(G)$,
respectively.

######  Definition 2.2.

Let $\varphi\in L^{2}(G)$. An operator $H_{\varphi}:H^{2}(G)\to H^{2}_{-}(G)$
defined by

$H_{\varphi}f=P_{+}(\varphi f)$

is called a Hankel operator on $G$ with symbol $\varphi$.

For $\varphi\in L^{\infty}(G)$ this operator is bounded. The theory of such
operators was developed in [19], [7], [3], [12] (see also [8]).

In this paper, we consider the next generalization of Hankel operators over
$G$.

######  Definition 2.3.

Consider a non-null homomorphisms $\mu$ and $\nu$ of the semigroup $X_{+}$ to
a multiplicative semigroup $\mathbb{C}$ (a generalized semi-characters of
$X_{+}$) and a function $a:X_{+}\setminus\\{\chi_{0}\\}\to\mathbb{C}$. A
linear operator $A=A_{(\mu;\nu),a}$ between the spaces $H^{2}(G)$ and
$H^{2}_{-}(G)$ that is defined at least on the linear subspace
$\mathrm{span}(X_{+})$ of $H^{2}(G)$ and satisfies

$\langle
A_{(\mu;\nu),a}\chi,\overline{\xi}\rangle=\mu(\chi)\nu(\xi)a(\chi\xi)\ \
\forall\chi\in X_{+},\xi\in X_{+}\setminus\\{\chi_{0}\\}$

will be called a $(\mu;\nu)$-Hankel operator.

We put $A_{\mu,a}:=A_{(\mu;1),a}$ and call such operator a $\mu$-Hankel
operator. If $\nu(\xi)\neq 0$ $\forall\xi\in X_{+}$ we have
$A_{(\mu;\nu),a}=A_{\left(\frac{\mu}{\nu};1\right),a\nu}$. That is why the
main part of the paper is devoted to $\mu$-Hankel operators. We put also
$B_{\nu,a}:=A_{(1;\nu),a}$ and call such operator a $\nu$-Hankel operator. In
the last section of the paper $\nu$-Hankel operators are also considered in
brief.

We use also following notation. Let $H_{1}$ and $H_{2}$ be two Hilbert spaces.
Given two vectors $y\in H_{1}$, and $f\in H_{2}$ we define the rank-one
operator $f\otimes y:H_{1}\to H_{2}$ by

$(f\otimes y)x=\langle x,y\rangle f,\mbox{ for all }x\in H_{1}.$

We will use throughout this paper the following easy-to-check properties of
this operator $(i)A(f\otimes y)B=(Af)\otimes(B^{*}y)\mbox{ for
}A\in\mathcal{L}(H_{2}),B\in\mathcal{L}(H_{1})$, $(ii)(f\otimes
y)^{*}=y\otimes f$, and $(iii)\|f\otimes y\|=\|f\|\|y\|$.

As usual we denote by $\mathcal{L}(V,W)$ the space of all bounded linear
operators between Banach spaces $V$ and $W$,
$\mathcal{L}(V):=\mathcal{L}(V,V)$.

##  3\. $\mu$-Hankel Operators on G

###  3.1. Conditions of boundedness of $\mu$-Hankel operators and the
generalized Hankel equation

Remark 1. Note that $\overline{\xi}$ runs over the orthonormal basis $X_{-}$
of $H^{2}_{-}(G)$ in the definition 2.3. Thus, for each $\chi\in X_{+}$ the
numbers $\mu(\chi)a(\chi\xi)$ are Fourier coefficients of the function
$A_{\mu,a}\chi$ with respect to this basis. Taking $\chi=\chi_{0}$ we conclude
that $\|A_{\mu,a}\chi_{0}\|=\|a\|_{\ell^{2}(X_{+}\setminus\\{\chi_{0}\\})}$ by
Parseval’s formula and thus $a(\xi)\neq 0$ for an at most countable set of
$\xi\in X_{+}$.

######  Theorem 3.4.

Let $\mu\in\ell^{2}(X_{+})$. A $\mu$-Hankel operator $A_{\mu,a}$ extends from
$\mathrm{span}(X_{+})$ to a bounded operator between the spaces $H^{2}(G)$ and
$H^{2}_{-}(G)$ if and only if $a\in\ell^{2}(X_{+}\setminus\\{\chi_{0}\\})$. In
this case the extension is unique and

$\|A_{\mu,a}\|\leq\|\mu\|_{\ell^{2}(X_{+})}\|a\|_{\ell^{2}(X_{+}\setminus\\{\chi_{0}\\})}.$

###### Proof.

The necessity of the condition $a\in\ell^{2}(X_{+}\setminus\\{\chi_{0}\\})$
was mentioned in Remark 1.

Now let $\mu,a\in\ell^{2}(X_{+}\setminus\\{\chi_{0}\\})$ and $A=A_{\mu,a}$.
For each polynomial $f\in\mathrm{span}(X_{+})$ we have $f=\sum_{\chi\in
X_{+}}\langle f,\chi\rangle\chi$ (a finite sum) and thus $Af=\sum_{\chi\in
X_{+}}\langle f,\chi\rangle A\chi$. It follows that

$\displaystyle Af$ $\displaystyle=$ $\displaystyle\sum_{\xi\in
X_{+}\setminus\\{\chi_{0}\\}}\langle
Af,\overline{\xi}\rangle\overline{\xi}=\sum_{\xi\in
X_{+}\setminus\\{\chi_{0}\\}}\left(\sum_{\chi\in X_{+}}\langle
f,\chi\rangle\langle A\chi,\overline{\xi}\rangle\right)\overline{\xi}$
$\displaystyle=$ $\displaystyle\sum_{\xi\in
X_{+}\setminus\\{\chi_{0}\\}}\left(\sum_{\chi\in X_{+}}\langle
f,\chi\rangle\mu(\chi)a(\chi\xi)\right)\overline{\xi}.$

Therefore by Parseval’s formula and Cauchy-Bunyakovskii-Schwartz inequality
one has

$\displaystyle\|Af\|^{2}$ $\displaystyle=$ $\displaystyle\sum_{\xi\in
X_{+}\setminus\\{\chi_{0}\\}}\left|\sum_{\chi\in X_{+}}\langle
f,\chi\rangle\mu(\chi)a(\chi\xi)\right|^{2}$ $\displaystyle\leq$
$\displaystyle\sum_{\xi\in X_{+}\setminus\\{\chi_{0}\\}}\left(\sum_{\chi\in
X_{+}}|\mu(\chi)|^{2}\sum_{\chi\in X_{+}}|a(\chi\xi)\langle
f,\chi\rangle|^{2}\right)$ $\displaystyle=$
$\displaystyle\|\mu\|_{\ell^{2}}^{2}\sum_{\chi\in X_{+}}|\langle
f,\chi\rangle|^{2}\sum_{\xi\in X_{+}\setminus\\{\chi_{0}\\}}|a(\chi\xi)|^{2}$
$\displaystyle\leq$
$\displaystyle\|\mu\|_{\ell^{2}(X_{+})}^{2}\|a\|_{\ell^{2}(X_{+}\setminus\\{\chi_{0}\\})}^{2}\|f\|^{2}.$

To finish the proof it remains to note that $\mathrm{span}(X_{+})$ is dense in
$H^{2}(G)$ by [10, Lemma 1]111There is a typo in the proof of this lemma. It
should to be $H^{p}_{T}(G)$ instead of $H^{p}(G)$ in the beginning of the
proof..

∎

######  Theorem 3.5.

A bounded operator $A:H^{2}(G)\to H^{2}_{-}(G)$ is $\mu$-Hankel if and only if
it satisfies the generalized Hankel equation

$AS_{\chi}=\mu(\chi)P_{-}\mathcal{S}_{\chi}A\ \ \forall\chi\in X_{+},$ (3.1)

where $\mathcal{S}_{\chi}f:=\chi f$ for all $f\in L^{2}(G)$ is a bilateral
shift operator on $L^{2}(G)$ with respect to $\chi$,
$S_{\chi}:=\mathcal{S}_{\chi}|H^{2}(G)$ a unilateral shift in $H^{2}(G)$.

###### Proof.

Let an operator $A_{\mu,a}:H^{2}(G)\to H^{2}_{-}(G)$ be bounded. Then for
$\chi,\eta\in X_{+},\xi\in X_{+}\setminus\\{\chi_{0}\\}$ we have

$\displaystyle\langle A_{\mu,a}S_{\chi}\eta,\overline{\xi}\rangle$
$\displaystyle=$ $\displaystyle\langle
A_{\mu,a}(\chi\eta),\overline{\xi}\rangle=\mu(\chi)(\mu(\eta)a(\eta(\chi\xi)))$
$\displaystyle=$ $\displaystyle\mu(\chi)\langle
A_{\mu,a}\eta,\overline{\chi\xi}\rangle=\mu(\chi)\langle\chi
A_{\mu,a}\eta,\overline{\xi}\rangle$ $\displaystyle=$
$\displaystyle\mu(\chi)\langle\chi
A_{\mu,a}\eta,P_{-}\overline{\xi}\rangle=\langle\mu(\chi)P_{-}\mathcal{S}_{\chi}A_{\mu,a}\eta,\overline{\xi}\rangle.$

Since $A_{\mu,a}$ is bounded, (3.1) follows.

Conversely, let (3.1) holds for a bounded operator $A:H^{2}(G)\to
H^{2}_{-}(G)$. We put $a(\xi):=\langle A\chi_{0},\overline{\xi}\rangle$ for
$\xi\in X_{+}$. Since $\chi=S_{\chi}\chi_{0}$, one has for $\chi\in
X_{+},\xi\in X_{+}\setminus\\{\chi_{0}\\}$ that

$\displaystyle\langle A\chi,\overline{\xi}\rangle$ $\displaystyle=$
$\displaystyle\langle
AS_{\chi}\chi_{0},\overline{\xi}\rangle=\langle\mu(\chi)P_{-}\mathcal{S}_{\chi}A\chi_{0},\overline{\xi}\rangle$
$\displaystyle=$
$\displaystyle\mu(\chi)\langle\mathcal{S}_{\chi}A\chi_{0},P_{-}\overline{\xi}\rangle=\mu(\chi)\langle
A\chi_{0},\overline{\chi\xi}\rangle=\mu(\chi)a(\chi\xi).$

∎

###  3.2. The unimodular case

######  Theorem 3.6.

Let $|\mu(\chi)|=1\ \ \forall\chi\in X_{+}$. An operator $A$ between the
spaces $H^{2}(G)$ and $H^{2}_{-}(G)$ is bounded and $\mu$-Hankel if and only
if $A=H_{\varphi}U$ for some unitary operator $Uf(x):=f(gx)$ in $H^{2}(G)$
($g\in G$) and a Hankel operator $H_{\varphi}$ on $G$ with $\varphi\in
L^{\infty}(G)$. In this case,

$\|A\|=\mathrm{dist}_{L^{\infty}}(\varphi,H^{\infty}(G)).$

###### Proof.

Assume that $A=A_{\mu,a}$ and this operator is bounded. The rule
$\widetilde{\mu}(\chi^{-1}\chi^{\prime}):=\mu(\chi)^{-1}\mu(\chi^{\prime})$
where $\chi,\chi^{\prime}\in X_{+}$ extends $\mu$ from $X_{+}$ to a character
$\widetilde{\mu}$ of the group $X$. By the Pontryagin–van Kampen theorem there
is such an element $g\in G$ that $\widetilde{\mu}(\chi)=\chi(g)$ for all
$\chi\in X$. The translation operator $Uf(x):=f(gx)$ acts in $H^{2}(G)$ and is
unitary. Consider a bounded operator $H:H^{2}(G)\to H^{2}_{-}(G)$,
$H:=A_{\mu,a}U^{-1}$. Then for all $\chi\in X_{+},\xi\in
X_{+}\setminus\\{\chi_{0}\\}$ one has

$\displaystyle\langle H\chi,\overline{\xi}\rangle$ $\displaystyle=$
$\displaystyle\langle
A_{\mu,a}U^{-1}\chi,\overline{\xi}\rangle=\langle\chi(g^{-1})A_{\mu,a}\chi,\overline{\xi}\rangle$
$\displaystyle=$ $\displaystyle\chi(g)^{-1}\mu(\chi)a(\chi\xi)=a(\chi\xi).$

It follows that for all $\chi,\chi^{\prime}\in X_{+},\xi\in
X_{+}\setminus\\{\chi_{0}\\}$

$\langle
HS_{\chi}\chi^{\prime},\overline{\xi}\rangle=a(\chi\chi^{\prime}\xi)=\langle
H\chi^{\prime},\overline{\chi\xi}\rangle=\langle\chi
H\chi^{\prime},P_{-}\overline{\xi}\rangle=\langle
P_{-}\mathcal{S}_{\chi}H\chi^{\prime},\overline{\xi}\rangle.$

Thus, the Hankel equation $HS_{\chi}=P_{-}\mathcal{S}_{\chi}H$ holds for all
$\chi\in X_{+}$ and therefore by a version of the Nehari–Wang theorem [7,
Theorem 1.5] the operator $H$ is Hankel on $G$ (and vice versa),
$H=H_{\varphi}$ for some $\varphi\in L^{\infty}(G)$, and

$\|A_{\mu,a}\|=\|HU\|=\|H_{\varphi}\|=\mathrm{dist}_{L^{\infty}}(\varphi,H^{\infty}(G)).$

This proves the necessity.

To prove the sufficiency, note that by the Hankel equation for $H_{\varphi}$
[7, Theorem 1.5] we have
$H_{\varphi}S_{\chi}U=P_{-}\mathcal{S}_{\chi}H_{\varphi}U$ for all $\chi\in
X_{+}$. Since $S_{\chi}U=\chi(g)^{-1}US_{\chi}=\mu(\chi)^{-1}US_{\chi}$, it
follows that the operator $H_{\varphi}U$ satisfies (3.1) and Theorem 3.5
works. ∎

The previous theorem enables us to use for $\mu$-Hankel operators in the
unimodular case results on Hankel operators over $G$ from [7]. In particular,
we get the following corollary.

######  Corollary 3.7.

Let $|\mu(\chi)|=1\ \ \forall\chi\in X_{+}$. A non-trivial compact
$\mu$-Hankel operator on $G$ exists if and only if there is a first positive
element $\chi_{1}\in X_{+}$.

###### Proof.

This follows from the previous theorem and the fact that a non-trivial compact
Hankel operator on $G$ exists if and only if there is a first positive element
in $X$ [19] (see also [7, Theorem 2.1]). ∎

###  3.3. The non-unimodular case

In this subsection we assume that a group $X$ possesses the first positive
element $\chi_{1}$ and $\chi_{1}^{0}:=\chi_{0}$. By $H^{2}(\chi_{1})$ we
denote the Hilbert subspace of $H^{2}(G)$ with orthonormal basis
$\\{\chi_{1}^{n}:n\in\mathbb{Z}_{+}\\}$. We put $X_{+}^{i}:=X^{i}\cap
X_{+}=\\{\chi_{1}^{n}:n\in\mathbb{Z}_{+}\\}$.

######  Lemma 3.8.

([7]) The set $X_{+}\setminus X_{+}^{i}$ is an ideal of the semigroup $X_{+}$.

###### Proof.

First note that $\chi\in X_{+}\setminus X_{+}^{i}$ if and only if
$\chi>\chi_{1}^{n}\ \forall n\in\mathbb{Z}_{+}$. Indeed, the ”if” part is
obvious. Now assume that $\chi\in X_{+}\setminus X_{+}^{i}=X_{+}\setminus
X^{i}$ but $\chi\leq\chi_{1}^{n}$ for some $n\in\mathbb{Z}_{+}$. Since
$\chi_{0}\leq\chi$ and the group $X^{i}$ is convex [10, Theorem 2], we get
$\chi\in X^{i}$, a contradiction.

Thus, for all $\xi\in X_{+}$ and $\chi\in X_{+}\setminus X_{+}^{i}$ we have
$\xi\chi>\xi\chi_{1}^{n}\forall n\in\mathbb{Z}_{+}$ and therefore $\xi\chi\in
X_{+}\setminus X_{+}^{i}$. The proof is complete. ∎

The next theorem describes bounded $\mu$-Hankel operators in the case where
$|\mu(\chi_{1})|<1$. In the following lemma we use the techniques found in
[17] (see also [6]). Recall that $S_{\chi}=\mathcal{S}_{\chi}|H^{2}(G)$ for
$\chi\in X_{+}$.

######  Lemma 3.9.

Let $Q\in\mathcal{L}(H^{2}_{-}(G))$, $\|Q\|<1$. Then an operator $X$ from
$\mathcal{L}(H^{2}(G),H^{2}_{-}(G))$ is a solution of the operator equation

$QX=XS_{\chi_{1}}$ (3.2)

if and only if it has the form

$X=\sum_{n=0}^{\infty}(Q^{n}\varphi_{0})\otimes\chi_{1}^{n}$ (3.3)

for some function $\varphi_{0}\in H^{2}_{-}(G)$.

###### Proof.

If $X$ satisfies (3.2) then
$QXS_{\chi_{1}}^{*}=XS_{\chi_{1}}S_{\chi_{1}}^{*}$. Since
$S_{\chi}^{*}f=P_{+}(\overline{\chi}f)$ for $\chi\in X_{+}$, $f\in H^{2}(G)$,
we have $S_{\chi_{1}}S_{\chi_{1}}^{*}=I_{H^{2}}-\chi_{0}\otimes\chi_{0}$
because both sides here coincide on $X_{+}$. Therefore
$QXS_{\chi_{1}}^{*}=X(I_{H^{2}}-\chi_{0}\otimes\chi_{0}$). It follows that

$(\mathcal{I}-\mathcal{T})X=\varphi_{0}\otimes\chi_{0},$

where $\mathcal{T}X:=QXS_{\chi_{1}}^{*}$ is a linear transformation from
$\mathcal{L}(\mathcal{L}(H^{2}(G),H^{2}_{-}(G)))$,
$\|\mathcal{T}\|\leq\|Q\|<1$, $\mathcal{I}$ is the unit operator that acts in
$\mathcal{L}(\mathcal{L}(H^{2}(G),H^{2}_{-}(G)))$, and
$\varphi_{0}:=X\chi_{0}$. Thus,

$X=(\mathcal{I}-\mathcal{T})^{-1}(\varphi_{0}\otimes\chi_{0})=\sum_{n=0}^{\infty}\mathcal{T}^{n}(\varphi_{0}\otimes\chi_{0})=\sum_{n=0}^{\infty}(Q^{n}\varphi_{0})\otimes\chi_{1}^{n}.$

Conversely, let (3.3) holds. Since
$S_{\chi_{1}}^{*}=P_{+}\mathcal{S}_{\overline{\chi_{1}}}|H^{2}(G)$, and
$(\varphi_{0}\otimes\chi_{0})S_{\chi_{1}}=\varphi_{0}\otimes
S_{\chi_{1}}^{*}\chi_{0}=\varphi_{0}\otimes(P_{+}\overline{\chi_{1}})=0$, we
get

$\displaystyle QX$ $\displaystyle=$
$\displaystyle\sum_{n=0}^{\infty}(Q^{n+1}\varphi_{0})\otimes\chi_{1}^{n}$
$\displaystyle=$ $\displaystyle\sum_{n=0}^{\infty}(Q^{n+1}\varphi_{0})\otimes
S_{\chi_{1}}^{*}(\chi_{1}^{n+1})=\sum_{k=1}^{\infty}((Q^{k}\varphi_{0})\otimes\chi_{1}^{k})S_{\chi_{1}}$
$\displaystyle=$
$\displaystyle\sum_{k=1}^{\infty}((Q^{k}\varphi_{0})\otimes\chi_{1}^{k})S_{\chi_{1}}+(\varphi_{0}\otimes\chi_{0})S_{\chi_{1}}$
$\displaystyle=$
$\displaystyle\left(\sum_{n=0}^{\infty}(Q^{n}\varphi_{0})\otimes\chi_{1}^{n}\right)S_{\chi_{1}}=XS_{\chi_{1}}.$

∎

######  Theorem 3.10.

Let a $\mu$-Hankel operator $A=A_{\mu,a}$ is bounded and $q:=\mu(\chi_{1})$,
$0<|q|<1$. Then

$A=\sum_{n=0}^{\infty}q^{n}\varphi_{n}\otimes\chi_{1}^{n}$ (3.4)

where $\varphi_{n}$ is a sequence of functions from $H^{2}_{-}(G)$ with
Fourier coefficients

$\langle\varphi_{n},\overline{\xi}\rangle=a(\chi_{1}^{n}\xi)\ \ \forall
n\in\mathbb{Z}_{+},\xi\in X_{+}\setminus\\{\chi_{0}\\}$ (3.5)

that is uniformly bounded in norm. Moreover, $A|H^{2}(\chi_{1})^{\bot}=0$,
operator $A$ is nuclear and its norm satisfies
$\|A\|\leq\|\varphi_{0}\|/(1-|q|)$.

Conversely, if a sequence $\varphi_{n}\in H^{2}_{-}(G)$ is uniformly bounded
in norm, and (3.5) holds each operator of the form (3.4) with some
$q\in\mathbb{D}$ is nuclear and equals to $A_{\mu,a}$, where $\mu(\chi_{1})=q$
and $\mu(\chi)=0$ for all $\chi\in X_{+}\setminus X^{i}$.

###### Proof.

Necessity. The equation (3.1) implies that

$\mu(\chi_{1})P_{-}\mathcal{S}_{\chi_{1}}A=AS_{\chi_{1}}.$ (3.6)

Putting $X=A$ and $Q=\mu(\chi_{1})P_{-}\mathcal{S}_{\chi_{1}}$ in Lemma 3.9 we
get from (3.6) and (3.3) that

$\displaystyle
A=\sum_{n=0}^{\infty}\mu(\chi_{1})^{n}(P_{-}\mathcal{S}_{\chi_{1}})^{n}\varphi_{0}\otimes
S_{\chi_{1}}^{n}\chi_{0}=\sum_{n=0}^{\infty}\mu(\chi_{1})^{n}\varphi_{n}\otimes\chi_{1}^{n},$
(3.7)

where $\varphi_{0}=A\chi_{0}$,
$\varphi_{n}:=(P_{-}\mathcal{S}_{\chi_{1}})^{n}\varphi_{0}$,
$\|\varphi_{n}\|\leq\|(P_{-}\mathcal{S}_{\chi_{1}})\|^{n}\|\varphi_{0}\|\leq\|\varphi_{0}\|$.

One can prove by induction that
$\langle\varphi_{n},\overline{\xi}\rangle=a(\chi_{1}^{n}\xi)$
($n\in\mathbb{Z}_{+}$). Indeed, for $n=0$ this is true due to the equalities
$\varphi_{0}=A\chi_{0}$ (see the proof of Lemma 3.9) and
$\chi_{1}^{0}=\chi_{0}$. And the induction step follows from the equality

$\displaystyle\langle\varphi_{n+1},\overline{\xi}\rangle$ $\displaystyle=$
$\displaystyle\langle(P_{-}\mathcal{S}_{\chi_{1}})^{n+1}\varphi_{0},\overline{\xi}\rangle=\langle\chi_{1}(P_{-}\mathcal{S}_{\chi_{1}})^{n}\varphi_{0},P_{-}\overline{\xi}\rangle$
$\displaystyle=$
$\displaystyle\langle(P_{-}\mathcal{S}_{\chi_{1}})^{n}\varphi_{0},\overline{\chi_{1}\xi}\rangle=\langle\varphi_{n},\overline{\chi_{1}\xi}\rangle.$

Formula (3.4) shows that $Af=0$ for $f\bot H^{2}(\chi_{1})$ and thus
$A|H^{2}(\chi_{1})^{\bot}=0$. Finally, since the series in (3.4) in one-
dimensional operators converges absolutely in the operator norm, $A$ is
nuclear and

$\|A\|\leq\sum_{n=0}^{\infty}|q|^{n}\|\varphi_{n}\otimes\chi_{1}^{n}\|\leq\sum_{n=0}^{\infty}|q|^{n}\|(P_{-}\mathcal{S}_{\chi_{1}})^{n}\varphi_{0}\|\|\chi_{1}^{n}\|\leq\frac{\|\varphi_{0}\|}{1-|q|}.$

Sufficiency. Let (3.4) holds with some $q\in\mathbb{D}$ and $\varphi_{n}\in
H^{2}_{-}(G)$, $\varphi_{n}$ are uniformly bounded in norm, and (3.5) is valid
for some $a:X_{+}\to\mathbb{C}$. Then $A$ is a nuclear operator between
$H^{2}(G)$ and $H^{2}_{-}(G)$, $A|H^{2}(\chi_{1})^{\bot}=0$ and
$A\chi_{1}^{k}=q^{k}\varphi_{k}$ for all $k\in\mathbb{Z}_{+}$. Define
$\mu:X_{+}\to\mathbb{D}$ by the rule $\mu(\chi_{1}^{n})=q^{n}$ for
$n\in\mathbb{Z}_{+}$ and $\mu|(X_{+}\setminus X^{i})=0$. Then $\mu$ is a semi-
character by Lemma 3.8 and we have

$\langle
A\chi_{1}^{k},\overline{\xi}\rangle=q^{k}\langle\varphi_{k},\overline{\xi}\rangle=\mu(\chi_{1}^{k})a(\chi_{1}^{k}\xi)$

for $k\in\mathbb{Z}_{+}$ and $\xi\in X_{+}\setminus\\{\chi_{0}\\}$. Finally,
$\langle A\chi,\overline{\xi}\rangle=0=\mu(\chi)a(\chi\xi)$ if $\chi\in
X_{+}\setminus X^{i}$. Thus, $A=A_{\mu,a}$. This completes the proof. ∎

In the following for $\chi\in X_{+}$ we put
$S_{\overline{\chi}}:=\mathcal{S}_{\overline{\chi}}|H^{2}_{-}(G)$.

######  Lemma 3.11.

Let $R\in\mathcal{L}(H^{2}(G))$, $\|R\|<1$. Then an operator $Y$ from
$\mathcal{L}(H^{2}_{-}(G),H^{2}(G))$ is a solution of the operator equation

$RY=YS_{\overline{\chi_{1}}}$ (3.8)

if and only if it has the form

$Y=\sum_{n=0}^{\infty}(R^{n}\psi_{1})\otimes\overline{\chi_{1}}^{n+1}$ (3.9)

for some function $\psi_{1}\in H^{2}(G)$.

###### Proof.

The proof is similar to the proof of Lemma 3.9. If $Y$ satisfies (3.8) then
$RYS_{\overline{\chi_{1}}}^{*}=YS_{\overline{\chi_{1}}}S_{\overline{\chi_{1}}}^{*}$.
But
$S_{\overline{\chi_{1}}}S_{\overline{\chi_{1}}}^{*}=I_{H^{2}_{-}}-\overline{\chi_{1}}\otimes\overline{\chi_{1}}$
(since $S_{\overline{\chi_{1}}}^{*}=P_{-}\mathcal{S}_{\chi_{1}}|H^{2}_{-}(G)$,
both sides of this equation coincide on $X_{-}$). So, we have
$RYS_{\overline{\chi_{1}}}^{*}=Y(I_{H^{2}_{-}}-\overline{\chi_{1}}\otimes\overline{\chi_{1}})$
or equivalently
$Y-RYS_{\overline{\chi_{1}}}^{*}=\psi_{1}\otimes\overline{\chi_{1}}$ where
$\psi_{1}:=Y\overline{\chi_{1}}$. This means that

$(\mathcal{I}_{1}-\mathcal{T}_{1})Y=\psi_{1}\otimes\overline{\chi_{1}}$ (3.10)

where $\mathcal{T}_{1}Y:=RYS_{\overline{\chi_{1}}}^{*}$ is an operator valued
transformation from the space
$\mathcal{L}(\mathcal{L}(H^{2}_{-}(G),H^{2}(G)))$ and $\mathcal{I}_{1}$ stands
for the identity in $\mathcal{L}(\mathcal{L}(H^{2}_{-}(G),H^{2}(G)))$. Since
$\|\mathcal{T}_{1}\|\leq\|R\|<1$, (3.10) implies that

$Y=(\mathcal{I}_{1}-\mathcal{T}_{1})^{-1}(\psi_{1}\otimes\overline{\chi_{1}})=\sum_{n=0}^{\infty}\mathcal{T}_{1}^{n}(\psi_{1}\otimes\overline{\chi_{1}})=\sum_{n=0}^{\infty}(R^{n}\psi_{1})\otimes\overline{\chi_{1}}^{n+1}.$
(3.11)

Conversely, let (3.9) holds. Since
$S_{\overline{\chi_{1}}}^{*}=P_{-}\mathcal{S}_{\chi_{1}}|H^{2}_{-}(G)$, and
$(\psi_{1}\otimes\overline{\chi_{1}})S_{\chi_{1}}=\psi_{1}\otimes
S_{\chi_{1}}^{*}\overline{\chi_{1}}=\psi_{1}\otimes(P_{-}\overline{\chi_{0}})=0$,
we get

$\displaystyle RY$ $\displaystyle=$
$\displaystyle\sum_{n=0}^{\infty}(R^{n+1}\psi_{1})\otimes\overline{\chi_{1}}^{n+1}$
$\displaystyle=$
$\displaystyle\sum_{k=1}^{\infty}(R^{k}\psi_{1})\otimes\overline{\chi_{1}}^{k}=\sum_{k=1}^{\infty}(R^{k}\psi_{1})\otimes
S_{\overline{\chi_{1}}}^{*}\overline{\chi_{1}}^{k+1}$ $\displaystyle=$
$\displaystyle\sum_{k=1}^{\infty}(R^{k}\psi_{1})\otimes\overline{\chi_{1}}^{k+1}S_{\overline{\chi_{1}}}+(\psi_{1}\otimes\overline{\chi_{1}})S_{\chi_{1}}$
$\displaystyle=$
$\displaystyle\left(\sum_{n=0}^{\infty}(R^{n}\psi_{1})\otimes\overline{\chi_{1}}^{n+1}\right)S_{\chi_{1}}=YS_{\chi_{1}}.$

∎

Now we shall describe bounded $\mu$-Hankel operators in the case where
$|\mu(\chi_{1})|>1$. In the following theorem $H^{2}(\overline{\chi_{1}})$
denotes the Hilbert subspace of $H^{2}_{-}(G)$ with orthonormal basis
$\\{\overline{\chi_{1}}^{n}:n\in\mathbb{N}\\}$, and $\mathrm{Im}A$ stands for
the image of an operator $A$.

######  Theorem 3.12.

Let a $\mu$-Hankel operator $A=A_{\mu,a}$ is bounded and $|\mu(\chi_{1})|>1$.
Then

$A=\sum_{n=1}^{\infty}\left(\frac{1}{\mu(\chi_{1})}\right)^{n-1}\overline{\chi_{1}}^{n}\otimes\psi_{n}$
(3.12)

where $\psi_{n}$ is a sequence of functions from $H^{2}(G)$ with Fourier
coefficients

$\langle\psi_{n},\chi\rangle=\overline{\mu(\chi_{1}^{n-1}\chi)a(\chi_{1}^{n}\chi)}\forall
n\in\mathbb{N},\chi\in X_{+}$ (3.13)

that is uniformly bounded in norm. Moreover, $\mathrm{Im}A\subset
H^{2}(\overline{\chi_{1}})$, the function $a$ is supported in
$X_{+}^{i}\setminus\\{\chi_{0}\\}$, operator $A$ is nuclear and its norm
satisfies $\|A\|\leq|\mu(\chi_{1})|\|\psi_{1}\|/(|\mu(\chi_{1})|-1)$.

Conversely, if a sequence $\psi_{n}\in H^{2}(G)$ is uniformly bounded in norm,
and (3.13) holds with some $\mu\in\mathrm{Hom}(X_{+},\mathbb{C})$,
$|\mu(\chi_{1})|>1$ and some function $a:X_{+}\to\mathbb{C}$ supported in
$X_{+}^{i}\setminus\\{\chi_{0}\\}$, then each operator of the form (3.12) is
nuclear and equals to $A_{\mu,a}$.

###### Proof.

Necessity. The equation (3.1) implies that
$AS_{\overline{\chi_{1}}}=\mu(\chi_{1})P_{-}\mathcal{S}_{\chi_{1}}A$. Or after
passing to conjugates

$S_{\overline{\chi_{1}}}^{*}A^{*}=\overline{\mu(\chi_{1})}(P_{-}\mathcal{S}_{\chi_{1}}A)^{*}.$
(3.14)

We claim that
$(P_{-}\mathcal{S}_{\chi_{1}}A)^{*}=A^{*}S_{\overline{\chi_{1}}}$. Indeed,
since $S_{\overline{\chi_{1}}}=(P_{-}\mathcal{S}_{\chi_{1}}|H^{2}_{-})^{*}$,
for each $\chi\in X_{+}$, $\xi\in X_{+}\setminus\\{\chi_{0}\\}$ one has

$\langle\chi,A^{*}S_{\overline{\chi_{1}}}\overline{\xi}\rangle=\langle\chi,A^{*}(P_{-}\mathcal{S}_{\chi_{1}}|H^{2}_{-})^{*}\overline{\xi}\rangle=\langle
P_{-}\mathcal{S}_{\chi_{1}}A\chi,\overline{\xi}\rangle.$

Now (3.14) takes the form
$(\overline{\mu(\chi_{1})})^{-1}S_{\overline{\chi_{1}}}^{*}A^{*}=A^{*}S_{\overline{\chi_{1}}}$.
Putting $Y=A^{*}$,
$R=(\overline{\mu(\chi_{1})})^{-1}S_{\overline{\chi_{1}}}^{*}$ in Lemma 3.11
we conclude that

$A^{*}=\sum_{n=0}^{\infty}\frac{1}{\overline{\mu(\chi_{1})}^{n}}((S_{\overline{\chi_{1}}}^{n})^{\ast}\psi_{1})\otimes\overline{\chi_{1}}^{n+1}=\sum_{n=0}^{\infty}\left(\frac{1}{\overline{\mu(\chi_{1})}}\right)^{n}(\psi_{n+1}\otimes\overline{\chi_{1}}^{n+1}),$

where $\psi_{1}=A^{*}\overline{\chi_{1}}$ (see the proof of Lemma 3.11),
$\psi_{n+1}:=(S_{\overline{\chi_{1}}}^{n})^{*}\psi_{1}$ and formula (3.12)
follows. The inclusion $\mathrm{Im}A\subset H^{2}(\overline{\chi_{1}})$ is a
direct consequence of (3.12).

Further, for all $\xi\notin X_{+}^{i}\setminus\\{\chi_{0}\\}$ we have

$\langle(\overline{\chi_{1}}^{n}\otimes\psi_{n})\chi_{0},\overline{\xi}\rangle=\langle\chi_{0},\psi_{n}\rangle\langle\overline{\chi_{1}}^{n},\overline{\xi}\rangle=0.$

It follows in view of (3.12) that

$a(\xi)=\mu(\chi_{0})a(\chi_{0}\xi)=\langle
A\chi_{0},\overline{\xi}\rangle=0.$

Thus the function $a$ is supported in $X_{+}^{i}\setminus\\{\chi_{0}\\}$.

To prove (3.13) consider $n\in\mathbb{Z}_{+},\chi\in X_{+}$. Then

$\displaystyle\langle\psi_{n+1},\chi\rangle$ $\displaystyle=$
$\displaystyle\langle(S_{\chi_{1}}^{n})^{*}\psi_{1},\chi\rangle=\langle\psi_{1},S_{\chi_{1}}^{n}\chi\rangle=\langle
A^{*}\overline{\chi_{1}},\chi_{1}^{n}\chi\rangle$ $\displaystyle=$
$\displaystyle\langle\overline{\chi_{1}},A(\chi_{1}^{n}\chi)\rangle=\overline{\langle
A(\chi_{1}^{n}\chi),\overline{\chi_{1}}\rangle}=\overline{\mu(\chi_{1}^{n}\chi)a(\chi_{1}^{n+1}\chi)}.$

For all $n\in\mathbb{N}$ we have also
$\|\psi_{n}\|\leq\|(S_{\chi_{1}}^{n-1})^{*}\|\|\psi_{1}\|\leq\|\psi_{1}\|$.

Moreover, $A$ is nuclear and

$\displaystyle\|A\|$ $\displaystyle\leq$
$\displaystyle\sum_{n=1}^{\infty}\left(\frac{1}{|\mu(\chi_{1})|}\right)^{n-1}\|\overline{\chi_{1}}^{n}\|\|\psi_{n}\|$
$\displaystyle\leq$
$\displaystyle\sum_{n=1}^{\infty}\left(\frac{1}{|\mu(\chi_{1})|}\right)^{n-1}\|\psi_{1}\|=\frac{|\mu(\chi_{1})|}{|\mu(\chi_{1})|-1}\|\psi_{1}\|.$

Sufficiency. Let (3.12) and (3.13) hold and the function $a$ is supported in
$X_{+}^{i}\setminus\\{\chi_{0}\\}$. For all $\chi\in X_{+}$, $\xi\in
X_{+}\setminus\\{\chi_{0}\\}$ we have

$\langle
A\chi,\overline{\xi}\rangle=\sum_{n=1}^{\infty}\left(\frac{1}{\mu(\chi_{1})}\right)^{n-1}\langle(\overline{\chi_{1}}^{n}\otimes\psi_{n})\chi,\overline{\xi}\rangle.$

But

$(\overline{\chi_{1}}^{n}\otimes\psi_{n})\chi=\overline{\langle\psi_{n},\chi\rangle}\overline{\chi_{1}}^{n}=\mu(\chi_{1}^{n-1}\chi)a(\chi_{1}^{n}\chi)\overline{\chi_{1}}^{n},$

and then

$\langle(\overline{\chi_{1}}^{n}\otimes\psi_{n})\chi,\overline{\xi}\rangle=\mu(\chi_{1}^{n-1}\chi)a(\chi_{1}^{n}\chi)\langle\overline{\chi_{1}}^{n},\overline{\xi}\rangle=\begin{cases}0,\xi\notin
X_{+}^{i}\setminus\\{\chi_{0}\\};\\\
\mu(\chi_{1}^{n-1}\chi)a(\chi_{1}^{n}\chi),\xi=\chi_{1}^{n}.\end{cases}$

Therefore

$\langle A\chi,\overline{\xi}\rangle=\begin{cases}0,\xi\notin
X_{+}^{i}\setminus\\{\chi_{0}\\};\\\ \mu(\chi)a(\xi\chi);\xi\in
X_{+}^{i}\setminus\\{\chi_{0}\\}.\end{cases}$

On the other hand, if $\xi\notin X_{+}^{i}\setminus\\{\chi_{0}\\}$ we get
$\xi\in X_{+}\setminus X_{+}^{i}$ and thus $\chi\xi\in X_{+}\setminus
X_{+}^{i}$ by Lemma 3.8 which implies $a(\chi\xi)=0$. So, we have $\langle
A\chi,\overline{\xi}\rangle=\mu(\chi)a(\xi\chi)$ in the case $\xi\notin
X_{+}^{i}\setminus\\{\chi_{0}\\}$, as well. ∎

###  3.4. Examples of $\mu$-Hankel operators

Let $G=\mathbb{T}$ the circle group. Each character of $\mathbb{T}$ has the
form $\chi_{k}(z)=z^{k}$, $k\in\mathbb{Z}$. Thus, in this case
$X_{+}=\\{\chi_{k}:k\in\mathbb{Z}_{+}\\}$,
$X_{-}=\\{\overline{\chi_{j}}:j\in\mathbb{N}\\}$.

Example 1. Let $q\in\mathbb{C}$, $0<|q|<1$, and $\sigma$ a finite (in general
complex) regular Borel measure on the open unit disk
$\mathbb{D}\subset\mathbb{C}$ such that
$\int_{\mathbb{D}}d|\sigma|(\zeta)/(1-|\zeta|)<\infty$. Consider the following
integral operator, at least for trigonometric polynomials
$f\in\mathrm{span}(X_{+})$ of analytic type:

$\mathbf{A}f(z)=\int_{\mathbb{D}}\frac{f(q\zeta)}{z-\zeta}d\sigma(\zeta)\quad(|z|=1)$
(3.15)

and the sequence of moments of the measure $\sigma$

$\gamma_{n}:=\int_{\mathbb{D}}\zeta^{n}d\sigma(\zeta)\quad(n\in\mathbb{Z}_{+}).$

For every $k\in\mathbb{Z}_{+}$ we have

$\displaystyle\mathbf{A}\chi_{k}(z)$ $\displaystyle=$ $\displaystyle
q^{k}\int_{\mathbb{D}}\frac{\zeta^{k}}{z-\zeta}d\sigma(\zeta)=q^{k}z^{-1}\int_{\mathbb{D}}\frac{\zeta^{k}}{1-z^{-1}\zeta}d\sigma(\zeta)$
$\displaystyle=$ $\displaystyle
q^{k}z^{-1}\int_{\mathbb{D}}\sum_{n=0}^{\infty}\zeta^{k+n}z^{-n}d\sigma(\zeta)=q^{k}\sum_{n=0}^{\infty}\int_{\mathbb{D}}\zeta^{k+n}d\sigma(\zeta)z^{-n-1}$
$\displaystyle=$ $\displaystyle
q^{k}\sum_{n=0}^{\infty}\gamma_{k+n}\chi_{-n-1}(z).$

Above the term-by-term integration of the series is legal, since for all
$k\in\mathbb{Z}_{+}$ one has

$\displaystyle\sum_{n=0}^{\infty}\int_{\mathbb{D}}|\zeta^{k+n}|d|\sigma|(\zeta)$
$\displaystyle\leq$
$\displaystyle\sum_{n=0}^{\infty}\int_{\mathbb{D}}|\zeta|^{n}d|\sigma|(\zeta)$
$\displaystyle=$
$\displaystyle\int_{\mathbb{D}}\sum_{n=0}^{\infty}|\zeta|^{n}d|\sigma|(\zeta)=\int_{\mathbb{D}}\frac{d|\sigma|(\zeta)}{1-|\zeta|}<\infty.$

Moreover, this estimate shows also that for all $k\in\mathbb{Z}_{+}$

$\sum_{n=0}^{\infty}|\gamma_{k+n}|^{2}\leq\left(\sum_{n=0}^{\infty}|\gamma_{k+n}|\right)^{2}\leq\left(\sum_{n=0}^{\infty}\int_{\mathbb{D}}|\zeta^{k+n}|d|\sigma|(\zeta)\right)^{2}<\infty.$

Thus, $(\gamma_{k+n})_{n\in Z_{+}}\in\ell^{2}(\mathbb{Z}_{+})$ for all
$k\in\mathbb{Z}_{+}$, and formula (3.4) implies that $\mathbf{A}\chi_{k}\in
H^{2}_{-}(\mathbb{T})$ and

$\displaystyle\langle\mathbf{A}\chi_{k},\overline{\chi_{j}}\rangle=q^{k}\sum_{n=0}^{\infty}\gamma_{k+n}\langle\chi_{-n-1},\chi_{-j}\rangle=q^{k}\gamma_{k+j-1}.$
(3.17)

It follows that $\mathbf{A}=A_{\mu,a}$ where $\mu(\chi_{k})=q^{k}$ for
$k\in\mathbb{Z}_{+}$ and $a(\chi_{k})=\gamma_{k-1}$ for $k\in\mathbb{N}$. Now
Theorem 3.4 (i) shows that operator $\mathbf{A}$ is bounded (the sequence
$\gamma=(\gamma_{n})$ belongs to $\ell^{2}(\mathbb{Z}_{+})$) and

$\displaystyle\|\mathbf{A}\|\leq\frac{\|\gamma\|_{\ell^{2}}}{\sqrt{1-|q|^{2}}}.$
(3.18)

Moreover, in this case operator $\mathbf{A}$ is nuclear by Theorem 3.10.

Formula (3.5) shows that
$\langle\varphi_{n},\overline{\chi_{j}}\rangle=\gamma_{n+j-1}$. Therefore

$\varphi_{n}(z)=\sum_{n=0}^{\infty}\gamma_{n+j-1}\overline{z}^{j}=\int_{\mathbb{D}}\frac{\zeta^{n-1}}{1-\overline{z}\zeta}d\sigma(\zeta).$

By Theorem 3.10

$\mathbf{A}f=\sum_{n=0}^{\infty}q^{n}\langle f,z^{n}\rangle\varphi_{n}.$

The operator $\mathbf{A}$ can be transferred to an operator acting on
$H^{2}(\mathbb{T})$. Indeed, consider the ”flip” operator
$Jf(z)=\overline{z}f(\overline{z})$, $f\in L^{2}(\mathbb{T})$. This is a
unitary operator and an involution on $L^{2}(\mathbb{T})$, and
$JH^{2}_{-}(\mathbb{T})=H^{2}(\mathbb{T})$ (and also
$JH^{2}(\mathbb{T})=H^{2}_{-}(\mathbb{T})$). Then the operator

$(J\mathbf{A})f(z)=\int_{\mathbb{D}}\frac{f(q\zeta)}{1-z\zeta}d\sigma(\zeta)$

is nuclear in $H^{2}(\mathbb{T})$ and its norm satisfies the estimate (3.18).
Since

$\langle
J\mathbf{A}\chi_{k},\chi_{j}\rangle=\langle\mathbf{A}\chi_{k},J\chi_{j}\rangle=q^{k}\gamma_{k+j}$

by (3.17), one has

$\mathrm{tr}(J\mathbf{A})=\sum_{k=0}^{\infty}q^{k}\gamma_{2k}=\sum_{k=0}^{\infty}q^{k}\int_{\mathbb{D}}\zeta^{2k}d\sigma(\zeta)=\int_{\mathbb{D}}\frac{d\sigma(\zeta)}{1-q\zeta^{2}}.$

To finish this example recall that every function $g=\sum_{j\geq
1}c_{j}\overline{\chi_{j}}$ from $H^{2}_{-}(\mathbb{T})$ extends to an anti-
analytic function $g(z)=\sum_{j\geq 1}c_{j}\overline{z}^{j}$ on $\mathbb{D}$
with the property $g(0)=0$. We shall show that the adjoint operator for
$\mathbf{A}$ is as follows

$\displaystyle\mathbf{A}^{*}g(z)$ $\displaystyle=$
$\displaystyle\int_{\mathbb{D}}\left(\overline{q}\frac{g(\zeta)}{\overline{z}-\overline{q\zeta}}+\frac{g(q\zeta)}{\overline{q\zeta}}\right)d\overline{\sigma}(\zeta)$
$\displaystyle=$
$\displaystyle\overline{q}\int_{\mathbb{D}}\frac{g(\zeta)}{\overline{z}-\overline{q\zeta}}d\overline{\sigma}(\zeta)+\left(\int_{\mathbb{D}}\frac{g(q\zeta)}{\overline{q\zeta}}d\overline{\sigma}(\zeta)\right)\chi_{0}(z)$
$\displaystyle=$ $\displaystyle Bg(z)+Cg(z)\qquad(g\in
H^{2}_{-}(\mathbb{T})).$

Indeed,

$\langle\chi_{k},B\overline{\chi_{j}}\rangle=q\int_{\mathbb{T}}z^{k}\int_{\mathbb{D}}\frac{\zeta^{j}}{z-q\zeta}d\sigma(\zeta)dm(z)=q\int_{\mathbb{D}}\zeta^{j}\int_{\mathbb{T}}\frac{z^{k}}{z-q\zeta}dm(z)d\sigma(\zeta)$

where $m$ stands for the normalized Haar measure on $\mathbb{T}$. Since

$\int_{\mathbb{T}}f(z)dm(z)=\frac{1}{2\pi}\int_{0}^{2\pi}f(e^{it})dt=\frac{1}{2\pi
i}\int_{\mathbb{T}}f(z)\frac{dz}{z},$

one has for $k\in\mathbb{Z}_{+}$

$\int_{\mathbb{T}}\frac{z^{k}}{z-q\zeta}dm(z)=\frac{1}{2\pi
i}\int_{\mathbb{T}}\frac{z^{k-1}}{z-q\zeta}dz=\begin{cases}(q\zeta)^{k-1},k\geq
1;\\\ 0,k=0.\end{cases}$

Thus,

$\langle\chi_{k},B\overline{\chi_{j}}\rangle=\begin{cases}q^{k}\gamma_{k+j-1},k\geq
1;\\\ 0,k=0.\end{cases}$

On the other hand,

$\langle\chi_{k},C\overline{\chi_{j}}\rangle=\begin{cases}0,k\geq 1;\\\
q^{j-1}\gamma_{j-1},k=0.\end{cases}$

Now formula (3.17) yields, that

$\langle\mathbf{A}\chi_{k},\overline{\chi_{j}}\rangle=\langle\chi_{k},(B+C)\overline{\chi_{j}}\rangle,\
\ k\in\mathbb{Z}_{+},j\in\mathbb{N}.$

This formula and (3.17) show that the operator $B+C$ has the same matrix
$(a_{jk})$ with respect to the standard bases of spaces
$H^{2}_{-}(\mathbb{T})$ and $H^{2}_{+}(\mathbb{T})$ as $\mathbf{A}^{*}$ and
that $\sum_{j,k}|a_{jk}|^{2}<\infty$. It follows that $B+C$ is bounded (in
fact, it is Hilbert–Schmidt, see, e. g., [18, Theorem 6.22]) and thus
$B+C=\mathbf{A}^{*}$.

Example 2. Consider an operator $\mathbf{A}$ given by the formula (3.15) where
$|q|=1$. The operator $Uf(z)=f(qz)$ is unitary in $H^{2}(\mathbb{T})$ and
$\mathbf{A}=HU$, where the operator

$Hf(z):=\int_{\mathbb{D}}\frac{f(\zeta)}{z-\zeta}d\sigma(\zeta)$

between $H^{2}(\mathbb{T})$ and $H^{2}_{-}(\mathbb{T})$ is Hankel. Indeed, let
$Jf(z)=\overline{z}f(\overline{z})$, $f\in L^{2}(\mathbb{T})$ be as in Example
1. Then $H=J\Gamma$ where

$\Gamma f(z):=\int_{\mathbb{D}}\frac{f(\zeta)}{1-z\zeta}d\sigma(\zeta)$

is a Hankel operator in $H^{2}(\mathbb{T})$ [15, Section 6.3]. Thus, the
operator $\mathbf{A}$ between $H^{2}(\mathbb{T})$ and $H^{2}_{-}(\mathbb{T})$
is bounded if and only if $\Gamma$ is bounded in $H^{2}(\mathbb{T})$. Let the
measure $\sigma$ be positive and supported in $(-1,1)$. According to a theorem
of Widom (see, e. g., [15, Theorem 6.2.1 and subsection 6.3.1]) in this case
the operator $\Gamma$ is bounded in $H^{2}(\mathbb{T})$ if and only if there
is a constant $b>0$ such that $\gamma_{n}<b/(n+1)$, $n\in\mathbb{Z}_{+}$. For
a complex measure $\sigma$ on $\mathbb{D}$ the property to be a Carleson
measure is sufficient for the boundedness of $\Gamma$ in $H^{2}(\mathbb{T})$
(see, e. g., [15, pp. 314–315]). As was mentioned in Theorem 3.6 if the
operator $\mathbf{A}$ is bounded it is $\mu$-Hankel where
$\mu(\chi_{k})=q^{k}$ ($k\in\mathbb{Z}_{+}$).

##  4\. $\nu$-Hankel operators on $G$

In this section we briefly discuss the class of operators that are in some
sense ”dual” to $\mu$-Hankel operators. Recall the definition of such
operators.

Let $\nu$ be a non-null homomorphism of the semigroup $X_{+}$ into a
multiplicative semigroup $\mathbb{C}$ and
$a:X_{+}\setminus\\{\chi_{0}\\}\to\mathbb{C}$. A $\nu$-Hankel operator is a
linear operator $B=B_{\nu,a}$ between the spaces $H^{2}(G)$ and $H^{2}_{-}(G)$
that is defined at least on the linear subspace $\mathrm{span}(X_{+})$ of
$H^{2}(G)$ and satisfies

$\langle B_{\nu,a}\chi,\overline{\xi}\rangle=\nu(\xi)a(\chi\xi)\ \
\forall\chi\in X_{+},\xi\in X_{+}\setminus\\{\chi_{0}\\}.$

######  Theorem 4.13.

A bounded $\nu$-Hankel operator $B:H^{2}(G)\to H^{2}_{-}(G)$ satisfies the
operator equation

$S_{\overline{\chi}}^{*}B=\nu(\chi)BS_{\chi}\ \ \forall\chi\in X_{+}.$ (4.19)

Conversely, if $\nu(\chi)\neq 0$ for all $\chi\in X_{+}$ then each bounded
operator $B:H^{2}(G)\to H^{2}_{-}(G)$ that satisfies (4.19) is $\nu$-Hankel.

###### Proof.

If a bounded operator $B:H^{2}(G)\to H^{2}_{-}(G)$ is $\nu$-Hankel then for
all $\chi,\eta\in X_{+}$, $\xi\in X_{+}\setminus\\{\chi_{0}\\}$ we have

$\displaystyle\langle S_{\overline{\chi}}^{*}B\eta,\overline{\xi}\rangle$
$\displaystyle=$ $\displaystyle\langle
B\eta,\overline{\chi\xi}\rangle=\nu(\chi\xi)a(\eta\chi\xi)$ $\displaystyle=$
$\displaystyle\nu(\chi)(\nu(\xi)a((\eta\chi)\xi))=\nu(\chi)\langle
BS_{\chi}\eta,\overline{\xi}\rangle$

and (4.19) follows.

Now let $\nu(\chi)\neq 0$ for all $\chi\in X_{+}$ and a bounded operator
$B:H^{2}(G)\to H^{2}_{-}(G)$ satisfies (4.19). If we put
$a(\xi):=\frac{1}{\nu(\xi)}\langle B\chi_{0},\overline{\xi}\rangle$ then for
all $\chi\in X_{+}$, $\xi\in X_{+}\setminus\\{\chi_{0}\\}$ one has

$\displaystyle\langle B\chi,\overline{\xi}\rangle$ $\displaystyle=$
$\displaystyle\langle
BS_{\chi}\chi_{0},\overline{\xi}\rangle=\frac{1}{\nu(\chi)}\langle
S_{\overline{\chi}}^{*}B\chi_{0},\overline{\xi}\rangle$ $\displaystyle=$
$\displaystyle\frac{1}{\nu(\chi)}\langle
B\chi_{0},\overline{\chi\xi}\rangle=\nu(\xi)\frac{1}{\nu(\chi\xi)}\langle
B\chi_{0},\overline{\chi\xi}\rangle=\nu(\xi)a(\chi\xi).$

∎

Remark 2. Theorem 4.13 enables us to apply lemmas 3.9, 3.11 to prove for
$\nu$-Hankel operators analogs of the structure theorems 3.10 and 3.12. If
necessary, the reader can easily do this work. If $\nu(\xi)\neq 0$
$\forall\xi\in X_{+}$, we have $A_{(\mu;\nu),a}=A_{\mu^{\prime},a\nu}$, where
$\mu^{\prime}=\frac{\mu}{\nu}$. Thus, analogs of theorems 3.10 and 3.12 for
$A_{(\mu;\nu),a}$ are corollaries of this theorems.

######  Theorem 4.14.

Assume that a group $X$ possesses the first positive element $\chi_{1}$ and
let $\nu\in\ell^{2}(X_{+})$. A $\nu$-Hankel operator $B_{\nu,a}$ extends from
$\mathrm{span}(X_{+})$ to a bounded operator between the spaces $H^{2}(G)$ and
$H^{2}_{-}(G)$ if and only if $a\in\ell^{2}(X_{+})$. In this case the
extension is unique and

$\|B_{\nu,a}\|\leq\|\nu\|_{\ell^{2}(X_{+}\setminus\\{\chi_{0}\\})}\|a\|_{\ell^{2}(X_{+}\setminus\\{\chi_{0}\\})}.$

###### Proof.

Let $B_{\nu,a}$ be bounded. Then for all $\chi\in X_{+}$ the numbers

$\langle
B_{\nu,a}^{*}\overline{\chi_{1}},\chi\rangle=\overline{\langle\chi,B_{\nu,a}^{*}\overline{\chi_{1}}\rangle}=\overline{\langle
B_{\nu,a}\chi,\overline{\chi_{1}}\rangle}=\overline{\nu(\chi_{1})a(\chi\chi_{1})}$

are Fourier coefficients of the function $B_{\nu,a}^{*}\overline{\chi_{1}}$.
Thus, the function $\chi\mapsto a(\chi\chi_{1})$ ($\chi\in X_{+}$) belongs to
$\ell^{2}(X_{+})$. On the other hand, $\\{\chi\chi_{1}:\chi\in
X_{+}\\}=X_{+}\setminus\\{\chi_{0}\\}$. (Indeed, $\xi\in
X_{+}\setminus\\{\chi_{0}\\}$ means that $\xi\geq\chi_{1}$. In turn, this is
equivalent to $\chi:=\xi\chi_{1}^{-1}\in X_{+}$ and so $\xi=\chi\chi_{1}$.) It
follows, that $a\in\ell^{2}(X_{+}\setminus\\{\chi_{0}\\})$.

Now let $\nu\in\ell^{2}(X_{+})$, $a\in\ell^{2}(X_{+})$ and $B=B_{\nu,a}$. As
in the proof of the Theorem 3.4 for each polynomial $f\in\mathrm{span}(X_{+})$
we have $Bf=\sum_{\chi\in X_{+}}\langle f,\chi\rangle B\chi$ (a finite sum).
It follows that

$\displaystyle Bf$ $\displaystyle=$ $\displaystyle\sum_{\xi\in
X_{+}\setminus\\{\chi_{0}\\}}\langle
Bf,\overline{\xi}\rangle\overline{\xi}=\sum_{\xi\in
X_{+}\setminus\\{\chi_{0}\\}}\left(\sum_{\chi\in X_{+}}\langle
f,\chi\rangle\langle B\chi,\overline{\xi}\rangle\right)\overline{\xi}$
$\displaystyle=$ $\displaystyle\sum_{\xi\in
X_{+}\setminus\\{\chi_{0}\\}}\left(\sum_{\chi\in X_{+}}\langle
f,\chi\rangle\nu(\xi)a(\chi\xi)\right)\overline{\xi}.$

Therefore by Parseval’s formula and Cauchy-Bunyakovskii-Schwartz inequality
one has

$\displaystyle\|Bf\|^{2}$ $\displaystyle=$ $\displaystyle\sum_{\xi\in
X_{+}\setminus\\{\chi_{0}\\}}\left|\sum_{\chi\in X_{+}}\langle
f,\chi\rangle\nu(\xi)a(\chi\xi)\right|^{2}$ $\displaystyle\leq$
$\displaystyle\sum_{\xi\in
X_{+}\setminus\\{\chi_{0}\\}}|\nu(\xi)|^{2}\left(\sum_{\chi\in X_{+}}|\langle
f,\chi\rangle|^{2}\right)\left(\sum_{\chi\in X_{+}}|a(\chi\xi)|^{2}\right)$
$\displaystyle\leq$
$\displaystyle\|\nu\|_{\ell^{2}(X_{+}\setminus\\{\chi_{0}\\})}^{2}\|a\|_{\ell^{2}(X_{+}\setminus\\{\chi_{0}\\})}^{2}\|f\|^{2}.$

As in the proof of the Theorem 3.4 it remains to note that
$\mathrm{span}(X_{+})$ is dense in $H^{2}(G)$ by [10, Lemma 1].

∎

Example 3. Let $q\in\mathbb{C}$, $|q|>1$, and $\tau$ a finite (in general
complex) regular Borel measure on the closed unit disk
$\overline{\mathbb{D}}\subset\mathbb{C}$ with sequence of moments
$\gamma=(\gamma_{n})_{n\in\mathbb{Z}_{+}}$. Consider the operator

$\mathbf{B}f(z):=\int_{\overline{\mathbb{D}}}\frac{f(\zeta)}{qz-\zeta}d\tau(\zeta)\quad(|z|=1).$

For all $k\in\mathbb{Z}_{+}$ we have

$\displaystyle\mathbf{B}\chi_{k}(z)$ $\displaystyle=$
$\displaystyle\overline{z}\int_{\overline{\mathbb{D}}}\frac{\zeta^{k}}{q-\zeta\overline{z}}d\tau(\zeta)$
$\displaystyle=$
$\displaystyle\frac{\overline{z}}{q}\int_{\overline{\mathbb{D}}}\zeta^{k}\sum_{n=0}^{\infty}\left(\frac{\zeta\overline{z}}{q}\right)^{n}d\tau(\zeta)=\sum_{n=0}^{\infty}\frac{\gamma_{n+k}}{q^{n+1}}\overline{\chi_{n+1}(z)}.$

(The term-by-term integration of the series is legal, since $1/|q|<1$ and
$(\gamma_{n})$ is bounded.) Since for all $k$ the sequence
$(\gamma_{n+k}/q^{n+1})$ belongs to $\ell^{2}$, we have $\mathbf{B}\chi_{k}\in
H^{2}_{-}(\mathbb{T})$. Moreover,

$\langle\mathbf{B}\chi_{k},\overline{\chi_{j}}\rangle=\sum_{n=0}^{\infty}\frac{\gamma_{n+k}}{q^{n+1}}\langle\overline{\chi_{n+1}},\overline{\chi_{j}}\rangle=\frac{1}{q^{j}}\gamma_{j+k-1}.$

Thus, $\mathbf{B}$ is $\nu$-Hankel with
$\nu(\chi_{j})=\left(\frac{1}{q}\right)^{j}$, $a(\chi_{k})=\gamma_{k-1}$. Now
Theorem 4.14 implies that the operator $\mathbf{B}$ is bounded if and only if
$\gamma\in\ell^{2}$ and in this case

$\|\mathbf{B}\|\leq\frac{\|\gamma\|_{\ell^{2}}}{\sqrt{|q|^{2}-1}}.$ (4.20)

Similar to the Example 1 the operator

$(J\mathbf{B})f(z)=\int_{\overline{\mathbb{D}}}\frac{f(\zeta)}{q-\zeta
z}d\tau(\zeta)$

is bounded in $H^{2}(\mathbb{T})$ if and only if $\gamma\in\ell^{2}$ and in
this case the estimate (4.20) holds for its norm. Moreover, this operator is
nuclear and

$\mathrm{tr}(J\mathbf{B})=\int_{\overline{\mathbb{D}}}\frac{d\tau(\zeta)}{q-\zeta^{2}}.$

##  5\. Acknowledgments

This work was supported in part by the State Program of Scientific Research of
the Republic of Belarus, project No. 20211776.

## References

  * [1] N. Bourbaki, Elements de mathematique. Livre VI. Integration. 2nd ed., Ch. 1–9. Hermann, Paris, 1965–1969.
  * [2] A. Defant, I. Schoolmann, $H_{p}$-theory of general Dirichlet series, J. Fourier Anal. Appl., 25 (2019), 3220–3258.
  * [3] R.V. Dyba, A. R. Mirotin, Functions of bounded mean oscillation and Hankel operators on compact Abelian groups, Tr. Inst. Mat. Mekh. UrO RAN. 20 (2) (2014), 135–144 [in Russian].
  * [4] E. Hewitt, K. A. Ross, Abstract Harmonic Analysis, Vol. I. Structure of Topological Groups. Integration Theory. Group Representations, Springer-Verlag, New York, 1979.
  * [5] L. H. Loomis,  An Introduction to Abstract Harmonic Analysis, van Nostrand, New York, 1953.
  * [6] R. Martinez-Avendaño, A Generalization of Hankel Operators, J. Funk. Anal.,  190 (2002), 418–446.
  * [7] A. R. Mirotin, Compact Hankel Operators on Compact Abelian Groups, Algebra i analiz, 33 (2021), 190–212 [in Russian]. English translation: Compact Hankel Operators on Compact Abelian Groups, St. Petersburg Math. J. (2022) (to appear).
  * [8] A. R. Mirotin,  On the general form of linear functionals on the Hardy spaces $H^{1}$ over compact Abelian groups and some of its applications, Indag. Math. 28 (2017), 451–462. https://doi.org/10.1016/j.indag.2016.11.023.
  * [9] A. R. Mirotin, On Hilbert Transform in Context of Locally Compact Abelian Groups, Int. J. Pure Appl. Math.,  51 (4) (2009), 463 – 474.
  * [10] A. R. Mirotin, Fredholm and spectral properties of Toeplitz operators on $H^{p}$ spaces over ordered groups, Sbornik: Mathematics, 202:5 (2011), 721–737.
  * [11] A. R. Mirotin, Hausdorff Operators on Compact Abelian Groups, Math. Nachr., (2022) (to appear).
  * [12] A. R. Mirotin and E. Yu. Kuzmenkova, On Hankel operators associated with linearly ordered Abelian groups, Tr. Inst. Mat. Mekh. UrO RAN., 22 (2016), 201–214 (Russian)
  * [13] A. R. Mirotin and E. Yu. Kuzmenkova, $\mu$-Hankel operators on Hilbert spaces, Opuscula Math., 41 (2021), 881–899.
  * [14] S. A. Morris, Pontryagin Duality and the Structure of Locally Compact Abelian Groups, Cambridge University Press, Cambridge, 1977.
  * [15] N.K. Nikolski, Operators, Functions, and Systems: an Easy Reading, vol. 1. Hardy, Hankel, and Toeplitz. Mathematical surveys and monographs, vol. 92, American Mathematical Society, 2002.
  * [16] W. Rudin,  Fourier Analysis on Groups, Intersciense Publishers, New York and London, 1962.
  * [17] S. Sun, On the operator equation $U^{\ast}TU=\lambda T$, Kexue Tongbao (English Ed.), 29 (1984), 298–299.
  * [18] J. Weidmann, Linear Operators in Hilbert Spaces, Springer-Verlag, New York, 1980.
  * [19] Ch. Yan, X. Chen, and K. Guo, Hankel operators and Hankel algebras, Chinese Ann. Math. Ser. B 19 (1998), 65–76. DEPARTMENT OF MATHEMATICS AND PROGRAMMING TECHNOLOGIES, FRANCISK SKORINA GOMEL STATE UNIVERSITY, GOMEL, 246019, BELARUS Email address<EMAIL_ADDRESS>