# Analysis and visualization of traceless symmetric tensors. Application to
the Hencky strain tensor for large strain tension-torsion.

Etienne Le Mire
Institut de Recherche en Génie Civil et Mécanique (GeM), UMR CNRS 6183
École Centrale de Nantes
1 rue de la Noë, 44321 Nantes, France
<EMAIL_ADDRESS>
&Erwan Verron
Institut de Recherche en Génie Civil et Mécanique (GeM), UMR CNRS 6183
École Centrale de Nantes
1 rue de la Noë, 44321 Nantes, France
<EMAIL_ADDRESS>
&Bertrand Huneau
Institut de Recherche en Génie Civil et Mécanique (GeM), UMR CNRS 6183
École Centrale de Nantes
1 rue de la Noë, 44321 Nantes, France
<EMAIL_ADDRESS>
&Nathan Selles
Laboratoire de Recherches et de Contrôle du Caoutchouc et des Plastiques
60 rue Auber, 94408 Vitry-sur-Seine, France
<EMAIL_ADDRESS>
###### Abstract

Cyclic multiaxial loadings of soft materials are usually studied throughout
experiments involving machines that prescribe a combination of uniaxial
tension and torsion. Due to the large strain framework, classical kinematic
analyses of strain in uniaxial tension-torsion are usually very complex. Based
on this observation, the present papers proposes a method to both analyze and
visualize a strain measure during a duty cycle of uniaxial tension-torsion in
large strain: based on the mathematical properties of the Hencky strain tensor
$\mathbf{h}$, the method consists in projecting $\mathbf{h}$ onto a well-
chosen tensorial basis, whose constituting elements are described in terms of
physical meaning. Thanks to this decomposition, the history of $\mathbf{h}$
reduces to the time evolution of a 3-components vector $\alpha(t)$. This
vector history can then be visualized as a path in the 3D space, rendering
very visual the complex kinematic phenomenon. As a second result, an original
definition of the mean and the amplitude of a strain path, based on the theory
of the Minimum Circumscribed Spheres, is proposed. This definition could be
useful for fatigue studies, for instance.

_K_ eywords kinematics $\cdot$ uniaxial tension-torsion $\cdot$ large strain
$\cdot$ 3D visualization

##  1 Introduction

Kinematics basic equations of the uniaxial tension-torsion of a perfect
cylinder under large strain are well-established (see for example (Ciarletta
and Destrade, 2014)), as it has now long been studied (from the work of Rivlin
(1949) in the mid twentieth century to very recent contributions of Valiollahi
et al. (2019)). However, it remains a very complex deformation to analyze,
especially in large strain (Lectez et al., 2014). As an example, one could
think about the fatigue of rubber specimens when submitted to uniaxial
tension-torsion: due to the large rotations of material planes, the definition
of a robust fatigue life criterion remains a hard task (Saintier et al., 2006;
Mars, 2001), and it is further rendered difficult when a phase shift is
applied between the uniaxial tension and the pure torsion inputs.

In this paper, we investigate a uniaxial tension-torsion duty cycle prescribed
to a solid cylinder to derive several new results, from a kinematic point of
view only (there will be no mention of stresses). Three types of loading
conditions are investigated: one where the uniaxial tension signal is in phase
with the pure torsion signal, and two where there is a phase shift between
them. In Section 2, basic kinematics is recalled, along with the types of
input signals used in this paper. Recent developments on the decomposition of
a special class of tensors onto a well-chosen tensor basis are recalled in
Section 2.3, then applied to the Hencky strain tensor in the particular case
of the uniaxial tension-torsion of a cylinder. Results are displayed in
Section 3, and consist in two new developments: the graphical visualization of
the time evolution of tensors in a 3D space, and a definition (not unique) of
both a mean tensor and an amplitude of the cycle through the computation of
Minimum Circumscribed Spheres. A physical interpretation of the decomposition
method is also proposed. Section 4 closes the paper.

##  2 Methods

The equations of the uniaxial tension-torsion of a perfect cylinder are
derived in the following; they can be found in several previous studies, e.g.
Rivlin (1949); Ogden (1997); Ciarletta and Destrade (2014); Murphy (2015)
among others.

###  2.1 Framework and input signals

The problem notations are summarized in Figure 1.

Figure 1: Simultaneous tension-torsion of a perfect cylinder.

Consider a perfect cylinder, whose dimensions are its length $L$ and radius
$A$ in the undeformed state, which become $l$ and $a$ in the deformed state,
respectively. A vertical displacement $U$ and a twist angle $\alpha$ are both
applied on the top surface, while the cylinder is fixed at its bottom surface.
The material is supposed to be homogeneous, isotropic, and incompressible. The
undeformed (or reference) configuration is denoted $(\mathcal{C}_{0})$, and
the actual deformed one is denoted $(\mathcal{C})$. The mapping $\chi$
transforms any point $P_{0}$ of $(\mathcal{C}_{0})$ into $P$ in
$(\mathcal{C})$. In what follows, capital letters ($R$, $Z$, etc.) are used
for quantities related to $(\mathcal{C}_{0})$, and lowercase letters for
quantities relative to $(\mathcal{C})$, unless otherwise mentioned.
$(\overrightarrow{e_{1}},\overrightarrow{e_{2}},\overrightarrow{e_{3}})$ is
the cartesian coordinate system, and two cylindrical coordinate systems are
used:
$(\overrightarrow{e_{R}},\overrightarrow{e_{\Theta}},\overrightarrow{e_{Z}})$
in the reference configuration and
$(\overrightarrow{e_{r}},\overrightarrow{e_{\theta}},\overrightarrow{e_{z}})$
in the deformed one.

The mapping between reference and deformed configurations is

$r=R\lambda^{-0.5}\quad,\quad\theta=\Theta+\tau\lambda Z\quad,\quad z=\lambda
Z,$ (1)

where $\lambda$ and $\tau$ are the stretch in the uniaxial tension direction
and the twist angle per unit of deformed length, respectively:

$\lambda=\frac{l}{L}\quad\textrm{and}\quad\tau=\frac{\alpha}{l}=\frac{\alpha}{\lambda
L}.$ (2)

In this study, the inputs are : the extension $\lambda(t)$, the angle per unit
length $\tau(t)$ and the phase $\phi$ (which does not depend on $t$) between
these two. Three particular cases are considered all along the paper: one in-
phase loading ($\phi=0^{\circ}$)and two out-of-phase loadings
($\phi=90^{\circ}$ and $\phi=180^{\circ}$). Figure 2 shows the corresponding
signals over a duty cycle.

(a) In-phase $\phi=0^{\circ}$.

(b) Out-of-phase $\phi=90^{\circ}$.

(c) Out-of-phase $\phi=180^{\circ}$.

Figure 2: Loading conditions: extension $\lambda(t)$ and angle by unit of
deformed length $\tau(t)$.

###  2.2 Kinematics of the tension-torsion of a perfect cylinder in large
strain

Following Equation (2), the deformation gradient tensor is

$\mathbf{F}=\frac{1}{\sqrt{\lambda}}\left(\overrightarrow{e_{r}}\otimes\overrightarrow{e_{R}}+\overrightarrow{e_{\theta}}\otimes\overrightarrow{e_{\Theta}}\right)+R\sqrt{\lambda}\tau\overrightarrow{e_{\theta}}\otimes\overrightarrow{e_{Z}}+\lambda\overrightarrow{e_{z}}\otimes\overrightarrow{e_{Z}},$
(3)

and the corresponding left Cauchy-Green strain tensor,
$\mathbf{b}=\mathbf{F}\mathbf{F}^{T}$, is

$\mathbf{b}=\frac{1}{\lambda}\overrightarrow{e_{r}}\otimes\overrightarrow{e_{r}}+\left(\frac{1}{\lambda}+\lambda^{2}\tau^{2}r^{2}\right)\overrightarrow{e_{\theta}}\otimes\overrightarrow{e_{\theta}}+\lambda^{2}\tau
r\left(\overrightarrow{e_{\theta}}\otimes\overrightarrow{e_{z}}+\overrightarrow{e_{z}}\otimes\overrightarrow{e_{\theta}}\right)+\lambda^{2}\overrightarrow{e_{z}}\otimes\overrightarrow{e_{z}}.$
(4)

The principal stretch ratios $\\{\lambda_{i}\\}_{i=1,2,3}$ are the square
roots of the eigenvalues of $\mathbf{b}$:

$\begin{cases}\begin{aligned}
\lambda_{1}=&\left[\frac{1}{2}\left(R^{2}\tau^{2}\lambda+\lambda^{2}+\frac{1}{\lambda}-\sqrt{\left(R^{2}\tau^{2}\lambda+\lambda^{2}+\frac{1}{\lambda}\right)^{2}-4\lambda}\right)\right]^{0.5}\\\
\lambda_{2}=&\lambda^{-0.5}\\\
\lambda_{3}=&\left[\frac{1}{2}\left(R^{2}\tau^{2}\lambda+\lambda^{2}+\frac{1}{\lambda}+\sqrt{\left(R^{2}\tau^{2}\lambda+\lambda^{2}+\frac{1}{\lambda}\right)^{2}-4\lambda}\right)\right]^{0.5}\end{aligned}\end{cases}$
(5)

We also define the Hencky strain tensor, which properties are used in the next
section. It can be derived from the right stretch tensor,
$\mathbf{V}=\left(\mathbf{b}\right)^{\frac{1}{2}}$:

$\mathbf{h}=\ln(\mathbf{V})=\frac{1}{2}\ln(\mathbf{b}).$ (6)

###  2.3 An original additive decomposition of $\mathbf{h}$ for tension-
torsion loadings

Remark 1: In what follows, both the concepts of tensors and their
representation as a matrix in a given basis are used. As it is really
important to differentiate them, a given tensor is written in bold letters
$\mathbf{A}$, and its representation as a matrix in a given basis
$\mathcal{B}_{\mathrm{i}}$ is written ${A}_{\mathcal{B}_{\mathrm{i}}}$.

####  2.3.1 A recent decomposition for symmetric and zero trace tensors

In order to generate a database of admissible strain tensors, Kunc and Fritzen
(2019) recently proposed a method to decompose any tensor of the space
$E=\\{\mathbf{A}\in\mathbb{M}^{3\times
3}|\mathbf{A}^{T}=\mathbf{A},\mathrm{Tr}(\mathbf{A})=0\\}$ onto a tensorial
basis. In the case of mechanical tensors (strain or stress), such a space is a
5 dimensional space. Here, we apply this idea to the Hencky strain tensor, as
described hereafter. We define the usual scalar product $<.,.>$ between two
elements of $E$. It is defined as: for two tensors $\mathbf{A}$ and
$\mathbf{B}$ of $E$, the scalar product between $\mathbf{A}$ and $\mathbf{B}$
reads

$<\mathbf{A},\mathbf{B}>=\mathbf{A}:\mathbf{B}=\textrm{tr}(\mathbf{A}^{T}\mathbf{B})$
(7)

to which is associated the Froebenius norm $||.||$ of a tensor $\mathbf{A}$

$||\mathbf{A}||=\sqrt{\mathbf{A}:\mathbf{A}}=\sqrt{\textrm{tr}(\mathbf{A}^{2})}$
(8)

Then we can define an orthonormal basis
$\mathcal{B}_{\mathrm{E}}=\\{\mathbf{Y^{(1)}},\mathbf{Y^{(2)}},\mathbf{Y^{(3)}},\mathbf{Y^{(4)}},\mathbf{Y^{(5)}}\\}$
as proposed by Kunc and Fritzen (2019), such that every element $\mathbf{Y}\in
E$ can be written as follows

$\mathbf{Y}=\sum_{i=1}^{5}\alpha_{i}\mathbf{Y^{(i)}}$ (9)

with $\forall
i\in\\{1,..,5\\},\alpha_{i}=\mathbf{Y}:\mathbf{Y^{(i)}}\in\mathbb{R}$.

####  2.3.2 Special case of the tension-torsion kinematics

By looking at the definition of the Hencky strain tensor (Eq. (6)) and the
properties of the logarithm of a tensor, one can see that the Hencky strain
tensor $\mathbf{h}$ is a real-valued, symmetric tensor, and its trace is zero
(due to the incompressibility hypothesis). Hence, the application of Equation
(9) to $\mathbf{h}$ is straightforward

$\mathbf{h}=\sum_{i=1}^{5}\alpha_{i}\mathbf{Y^{(i)}}$ (10)

This equation can also be written using matrices in the cylindrical basis
$\mathcal{B}_{\mathrm{\mathrm{cyl}}}=\\{\overrightarrow{e_{r}},\overrightarrow{e_{\theta}},\overrightarrow{e_{z}}\\}$

${h}_{\mathcal{B}_{\mathrm{\mathrm{cyl}}}}=\sum_{i=1}^{5}\alpha_{i}{Y}^{(i)}_{\mathcal{B}_{\mathrm{\mathrm{cyl}}}}$
(11)

where the ${Y}^{(i)}_{\mathcal{B}_{\mathrm{\mathrm{cyl}}}}$ can be defined as

$\displaystyle{Y}^{(1)}_{\mathcal{B}_{\mathrm{\mathrm{cyl}}}}=\sqrt{\frac{1}{6}}\left[\begin{array}[]{rrr}-1&0&0\\\
0&-1&0\\\
0&0&2\end{array}\right]_{\mathcal{B}_{\mathrm{\mathrm{cyl}}}},{Y}^{(2)}_{\mathcal{B}_{\mathrm{\mathrm{cyl}}}}=\sqrt{\frac{1}{2}}\left[\begin{array}[]{rrr}-1&0&0\\\
0&1&0\\\
0&0&0\end{array}\right]_{\mathcal{B}_{\mathrm{\mathrm{cyl}}}},{Y}^{(3)}_{\mathcal{B}_{\mathrm{\mathrm{cyl}}}}=\sqrt{\frac{1}{2}}\left[\begin{array}[]{lll}0&0&0\\\
0&0&1\\\ 0&1&0\end{array}\right]_{\mathcal{B}_{\mathrm{\mathrm{cyl}}}}$ (12)
$\displaystyle{Y}^{(4)}_{\mathcal{B}_{\mathrm{\mathrm{cyl}}}}=\sqrt{\frac{1}{2}}\left[\begin{array}[]{lll}0&0&1\\\
0&0&0\\\
1&0&0\end{array}\right]_{\mathcal{B}_{\mathrm{\mathrm{cyl}}}},{Y}^{(5)}_{\mathcal{B}_{\mathrm{\mathrm{cyl}}}}=\sqrt{\frac{1}{2}}\left[\begin{array}[]{lll}0&1&0\\\
1&0&0\\\ 0&0&0\end{array}\right]_{\mathcal{B}_{\mathrm{\mathrm{cyl}}}}$

However, the expression of the left Cauchy-Green tensor (Eq. (4)) implies
that, in the cylindrical basis, ${h}_{\mathcal{B}_{\mathrm{\mathrm{cyl}}}}$
has the following form:

${h}_{\mathcal{B}_{\mathrm{\mathrm{cyl}}}}=\left[\begin{array}[]{rrr}-(a+b)&0&0\\\
0&a&c\\\ 0&c&b\end{array}\right]_{\mathcal{B}_{\mathrm{\mathrm{cyl}}}}$ (13)

Hence, decomposing ${h}_{\mathcal{B}_{\mathrm{\mathrm{cyl}}}}$ onto the
${Y}^{(i)}_{\mathcal{B}_{\mathrm{\mathrm{cyl}}}}$ basis yields instantly
$\alpha_{4}=\alpha_{5}=0$, so

${h}_{\mathcal{B}_{\mathrm{\mathrm{cyl}}}}=\sum_{i=1}^{3}\alpha_{i}{Y}^{(i)}_{\mathcal{B}_{\mathrm{\mathrm{cyl}}}}$
(14)

where the ${Y}^{(i)}_{\mathcal{B}_{\mathrm{\mathrm{cyl}}}}$ are defined in
Equation (12(1,2,3)):

Remark 2: we draw the attention of the reader to the fact that the expression
of the matrices
${Y}^{(1)}_{\mathcal{B}_{\mathrm{\mathrm{cyl}}}},{Y}^{(2)}_{\mathcal{B}_{\mathrm{\mathrm{cyl}}}},{Y}^{(3)}_{\mathcal{B}_{\mathrm{\mathrm{cyl}}}}$
is an arbitrary choice. Indeed, one could for example invert the $1$ and $-1$
in ${Y}^{(2)}_{\mathcal{B}_{\mathrm{\mathrm{cyl}}}}$, and the corresponding
set would remain a basis.

Remark 3: the particular expression of
${h}_{\mathcal{B}_{\mathrm{\mathrm{cyl}}}}$ as described in Equation (12)
depends on the choice of the basis the problem has been expressed in. Here,
the cylindrical basis is a natural choice due to the nature of the
deformation. A provides a detailed insight on this particular issue.

Now, Equation (14) provides a powerful tool to represent the tensor
$\mathbf{h}$: instead of representing it as a matrix
${h}_{\mathcal{B}_{\mathrm{\mathrm{cyl}}}}$, it is more interesting to write
it as a 3-components vector $\\{\alpha_{1},\alpha_{2},\alpha_{3}\\}$, which
can be visualized as a point in a 3D space, as proposed by Chen et al. (2012).

Remark 4: by looking at Equation (13), one could a prima facie think about
representing the tensor $\mathbf{h}$ with the vector $\left[a,b,c\right]^{T}$
directly. However, the corresponding matrix family would not be linearly
independent (the matrices are not all orthogonal to each other), hence it
would not be a matrix basis as it is the case for the
${Y}^{(i)}_{\mathcal{B}_{\mathrm{\mathrm{cyl}}}}$.

###  2.4 Physical interpretation of the additive decomposition of $\mathbf{h}$

The new expression of ${h}_{\mathcal{B}_{\mathrm{\mathrm{cyl}}}}$, as
expressed in Equation (14) can be further analyzed. Indeed, it is possible to
analyze the ${Y}^{(i)}_{\mathcal{B}_{\mathrm{\mathrm{cyl}}}}$ in terms of
"elementary" deformations: type of deformation (tension, pure shear, etc.),
intensity and direction. From Equation (12), we propose below a kinematic
analysis of these three transformations:

  * •

${Y}^{(1)}_{\mathcal{B}_{\mathrm{\mathrm{cyl}}}}$ represents the uniaxial
tension along the $\overrightarrow{e_{z}}$ axis, with an intensity
$\lambda=\exp\left(\frac{2}{\sqrt{6}}\right)=2.26$. To highlight this, it is
possible to rewrite ${Y}^{(1)}_{\mathcal{B}_{\mathrm{\mathrm{cyl}}}}$ as
follows:

${Y}^{(1)}_{\mathcal{B}_{\mathrm{\mathrm{cyl}}}}=\left[\begin{array}[]{rrr}\ln\left(\exp\left(-\frac{1}{\sqrt{6}}\right)\right)&0&0\\\
0&\ln\left(\exp\left(-\frac{1}{\sqrt{6}}\right)\right)&0\\\
0&0&\ln\left(\exp\left(\frac{2}{\sqrt{6}}\right)\right)\end{array}\right]_{\mathcal{B}_{\mathrm{\mathrm{cyl}}}}$
(15)

  * •

${Y}^{(2)}_{\mathcal{B}_{\mathrm{\mathrm{cyl}}}}$ is the planar tension along
the $\overrightarrow{e_{\theta}}$ axis, $\overrightarrow{e_{z}}$ being
blocked, with an intensity $\lambda=\exp\left(\frac{1}{\sqrt{2}}\right)=2.03$:

${Y}^{(2)}_{\mathcal{B}_{\mathrm{\mathrm{cyl}}}}=\left[\begin{array}[]{rrr}\ln\left(\exp\left(-\frac{1}{\sqrt{2}}\right)\right)&0&0\\\
0&\ln\left(\exp\left(\frac{1}{\sqrt{2}}\right)\right)&0\\\
0&0&\ln\left(1\right)\end{array}\right]_{\mathcal{B}_{\mathrm{\mathrm{cyl}}}}$
(16)

  * •

To understand better ${Y}^{(3)}_{\mathcal{B}_{\mathrm{\mathrm{cyl}}}}$, one
can define its exponential and consider this new matrix
${V}^{(3)}_{\mathcal{B}_{\mathrm{\mathrm{cyl}}}}$ as representing the
corresponding stretch tensor $\mathbf{V}$ in the cylindrical basis:

$\displaystyle{V}^{(3)}_{\mathcal{B}_{\mathrm{\mathrm{cyl}}}}$
$\displaystyle=\exp\left({Y}^{(3)}_{\mathcal{B}_{\mathrm{\mathrm{cyl}}}}\right)$
(17) $\displaystyle=\exp\left(\left[\begin{array}[]{rrr}0&0&0\\\
0&0&\frac{1}{\sqrt{2}}\\\
0&\frac{1}{\sqrt{2}}&0\end{array}\right]\right)_{\mathcal{B}_{\mathrm{\mathrm{cyl}}}}$
$\displaystyle=\left[\begin{array}[]{rrr}1&0&0\\\ 0&1.26&0.77\\\
0&0.77&1.26\end{array}\right]_{\mathcal{B}_{\mathrm{\mathrm{cyl}}}}$

Now, as described by Thiel et al. (2019),
${V}^{(3)}_{\mathcal{B}_{\mathrm{\mathrm{cyl}}}}$ is the shear deformation in
the plane $\left(\overrightarrow{e_{\theta}},\overrightarrow{e_{z}}\right)$,
with an intensity $\alpha=\mathrm{arcsinh}^{-1}\left(0.77\right)=0.71$.

As an illustration, we propose in Figure 3 a graphical representation of the
aforementioned deformations, in 3D and in the
$\left(\overrightarrow{e_{\theta}},\overrightarrow{e_{z}}\right)$ plane.

Figure 3: Top: 3D representation of the three basis tensors
$\left(\mathbf{Y^{(1)}},\mathbf{Y^{(2)}},\mathbf{Y^{(3)}}\right)$ expressed in
the cylindrical coordinate system. Bottom: corresponding 2D representations in
the $\left(\overrightarrow{e_{\theta}},\overrightarrow{e_{z}}\right)$ plane.

Remark 5: Following the Remarks 1 and 2, the current kinematic analysis is
entirely dependent on both the basis chosen for expressing the problem (here
the cylindrical basis) and the choice of the expressions of the
${Y}^{(i)}_{\mathcal{B}_{\mathrm{\mathrm{cyl}}}}$. This is clearly highlighted
in A.

##  3 Applications

###  3.1 3D visualization of strain paths

As it has been concluded in the Section 2.3 it is now possible to visualize in
3D the time evolution of ${h}_{\mathcal{B}_{\mathrm{\mathrm{cyl}}}}$ (when
dealing with fatigue phenomena for example) as a 3D path, instead of a series
of matrices. We now directly apply this result to the uniaxial tension-torsion
of a perfect cylinder. The three loading conditions described in Figure 2 are
prescribed to the cylinder, and the Hencky strain tensor is computed at the
surface of the cylinder ($R=7$ mm, arbitrary strictly positive choice). The
output is the temporal evolution of
${h}_{\mathcal{B}_{\mathrm{\mathrm{cyl}}}}(t)$, from which are computed the
temporal evolutions of the $\alpha_{i}$:
$\\{\alpha_{1}(t),\alpha_{2}(t),\alpha_{3}(t)\\}$. The corresponding results
are shown in Figure 4.

(a) In-phase uniaxial tension-torsion - $\lambda=1.39$, $\tau=0.124\text{
rad.mm}^{-1}$, $\phi=0^{\circ}$.

(b) Out-of-phase uniaxial tension-torsion - $\lambda=1.39$, $\tau=0.124\text{
rad.mm}^{-1}$, $\phi=90^{\circ}$.

(c) Out-of-phase uniaxial tension-torsion - $\lambda=1.39$, $\tau=0.124\text{
rad.mm}^{-1}$, $\phi=180^{\circ}$.

Figure 4: Path in the 3D space
$\left({Y}^{(1)}_{\mathcal{B}_{\mathrm{\mathrm{cyl}}}},{Y}^{(2)}_{\mathcal{B}_{\mathrm{\mathrm{cyl}}}},{Y}^{(3)}_{\mathcal{B}_{\mathrm{\mathrm{cyl}}}}\right)$
of the Hencky strain tensor for different loading conditions. The arrows show
the time evolution.

As the loading conditions are cycles, all the paths are closed. The phase
between $\lambda$ and $\tau$ is clearly depicted through the shape of the
path: when in-phase ($\phi=0^{\circ}$) or in phase opposition
($\phi=180^{\circ}$), the responses are "symmetric" hence creating a curvy
line path. When a different phase is applied ($\phi=90^{\circ}$ for instance),
the path is not a curvy line anymore.

###  3.2 A definition of a mean and an amplitude for strain paths

A very useful feature used in fatigue studies is the Haigh diagram (Haigh,
1917; Gänser et al., 2007): it represents fatigue tests in a space
$S_{\mathrm{mean}}-S_{\mathrm{amp}}$, $S$ being a mechanical quantity such as
a strain measure or a stress for instance. An application to elastomers can be
found in Champy et al. (2021). From this perspective, we propose an original
definition for the mean and the amplitude of a path (or series) of tensors,
that could be used for Haigh diagrams. The new description of a tensor via
vectors as shown in Equation (9) has allowed us to visualize the paths of the
Hencky strain tensor in a 3D space (Fig. 4). The present derivation is based
on the Minimum Circumscribed Circles (MCS) considered by Dang Van et al.
(1989) for metal fatigue, who encircles the path of the shear stress $\tau(t)$
to determine a mean shear $\tau_{m}$ defined as the center of the minimum
circumscribed circle to the path $\tau(t)$. It permits the reduction of the
complex evolution of a mechanical quantity to a scalar that can be used in a
fatigue criterion, for instance. For a very clear and detailed discussion on
this subject, the reader can refer to the excellent paper of Papadopoulos
(1998).

Let us consider again a path of Hencky strain tensors
$\\{\mathbf{h}(t)\\}_{\left(0\leq t\leq T\right)}$, where $T$ is the period of
the duty cycle. All these tensors belong to a 5-dimensional space. From
Section 2.3, we have the following equivalence:

$\\{\mathbf{h}(t)\\}_{\left(0\leq t\leq
T\right)}=\\{\alpha_{1}(t),..,\alpha_{5}(t)\\}_{\left(0\leq t\leq T\right)}$
(18)

where $A(t)=\\{\alpha_{1}(t),..,\alpha_{5}(t)\\}_{\left(0\leq t\leq T\right)}$
is a vector of time dependent functions.

The MCS problem, as described by Papadopoulos (1998), consists in resolving
the min-max problem (i.e. finding both the radius $R$ and the center
$\mathbf{h}_{C}$) described by

$\mathrm{solution}=\underset{{\mathbf{Y}_{C}}}{\textrm{argmin}}\underbrace{\left\\{\underset{0\leq
t\leq T}{\max}\left\|\mathbf{h}_{C}-\mathbf{h}(t)\right\|\right\\}}_{R}$ (19)

Now, it can be adapted to our notations; by combining Eqs. (18) and (19), one
can rewrite the min-max problem using only the 5-components vector $A(t)$ and
the Euclidean distance: the MCS problem consists in resolving (i.e. finding
the radius $R$ and the center $A^{C}=\\{\alpha_{1}^{C},..,\alpha_{5}^{C}\\}$)
the following expression

$\mathrm{solution}=\underset{A^{C}}{\textrm{argmin}}\underbrace{\left\\{\underset{0\leq
t\leq T}{\max}\left\|A^{C}-A(t)\right\|\right\\}}_{R}$ (20)

We illustrate the MCS problem for a 3D case in Figure 5 by finding the
circumscribed spheres to the three paths shown in Figure 4. The corresponding
centers
$\mathbf{h}^{(C)}=\\{\alpha_{1}^{(C)},\alpha_{2}^{(C)},\alpha_{3}^{(C)}\\}$
and radii $R$ are given in Table 1.

$\phi$ | $\alpha_{1}^{(C)}$ | $\alpha_{2}^{(C)}$ | $\alpha_{3}^{(C)}$ | R
---|---|---|---|---
$0^{\circ}$ | 0.068 | 0.077 | 0.361 | 0.375
$90^{\circ}$ | 0.053 | 0.091 | 0.357 | 0.414
$180^{\circ}$ | 0.019 | 0.105 | 0.348 | 0.526
Table 1: Coordinates and radius of the center for each MCS computed for the
three paths.

(a) In-phase uniaxial tension-torsion - $\lambda=1.39$, $\tau=0.124\text{
rad.mm}^{-1}$, $\phi=0^{\circ}$.

(b) Out-of-phase uniaxial tension-torsion - $\lambda=1.39$, $\tau=0.124\text{
rad.mm}^{-1}$, $\phi=90^{\circ}$.

(c) Out-of-phase uniaxial tension-torsion - $\lambda=1.39$, $\tau=0.124\text{
rad.mm}^{-1}$, $\phi=180^{\circ}$.

Figure 5: Visualization of the MCS for the three loading conditions.

The position of $\mathbf{h}^{(C)}$ depends on the choice of the basis of
expression of the problem (here the cylindrical basis), but the radius does
not. This argument is discussed in A. Note that in the 5D case (if
$\mathbf{h}$ is represented by a matrix without zeros), the MCS and the path
exist and are well-defined, but they cannot be easily visualized.

Remark 5: By construction, the centers of the MCS are tensors, hence they are
subjected to tensor properties and calculus rules.

##  4 Conclusion

The present paper provides new tools to handle the time evolution of tensors
that are both symmetric and traceless. Through the projection of such a tensor
on a well-chosen basis, it has been shown that, in the particular case of the
uniaxial tension-torsion of a cylinder under large strain, it is possible to
visualize in 3D the time evolution of this type of tensor. Moreover, a new
definition of the mean and amplitude of a duty cycle has been proposed, based
on the idea of Minimum Circumscribed Spheres. It is believed that it could be
useful for some research fields such as fatigue life prediction, where
handling a full duty cycle is usually very complicated. A physical
interpretation of the elements constituting the tensor basis has also been
proposed: even though their definition highly depends on the choice of the
basis of expression of the problem, their analysis could help to better
understand some complex deformations by decomposing them into multiple
elementary ones. To go further, it would be interesting to study these
developments in a more general context, such as when incompressibility is not
assumed. For more classical criteria (involving a stress tensor or an energy
for instance), an extension to stress tensors can be considered, assuming that
the mathematical underlying properties are verified beforehand.

## Conflict of interest

The authors declare that they have no conflict of interest.

## References

  * Champy et al. (2021) Champy, C., Le Saux, V., Marco, Y., Glanowski, T., Charrier, P., Hervouet, W., 2021\.  Fatigue of crystallizable rubber: Generation of a Haigh diagram over a wide range of positive load ratios.  International Journal of Fatigue 150, 106313.
  * Chen et al. (2012) Chen, M.X., Tan, Y.W., Wang, B.F., 2012.  General invariant representations of the constitutive equations for isotropic nonlinearly elastic materials.  International Journal of Solids and Structures 49, 318–327.
  * Ciarletta and Destrade (2014) Ciarletta, P., Destrade, M., 2014\.  Torsion instability of soft solid cylinders.  The IMA Journal of Applied Mathematics 79, 804–819.
  * Dang Van et al. (1989) Dang Van, K., Cailletaud, G., Flavenot, J., Le Douraon, A., Lieurade, H., 1989.  Criterion for high cycle fatigue failure under multiaxial loading, in: Brown, M.W., Miller, K.J. (Eds.), Biaxial and Multiaxial Fatigue (EGF 3), pp. 459–478.
  * Gänser et al. (2007) Gänser, H.P., Leitgeb, A., Glinsner, K., Eichlseder, W., 2007\.  Computation of a modified Haigh-Goodman diagram for damage tolerant design for infinite fatigue life.  Proceedings of the Institution of Mechanical Engineers, Part C: Journal of Mechanical Engineering Science 221, 619–623.
  * Haigh (1917) Haigh, B.P., 1917.  Experiments on the fatigue of brasses.  Journal of the Institute of Metals 18, 55–86.
  * Kunc and Fritzen (2019) Kunc, O., Fritzen, F., 2019\.  Finite strain homogenization using a reduced basis and efficient sampling (and correction).  Mathematical and Computational Applications 24, 56 (and 95).
  * Lectez et al. (2014) Lectez, A.S., Verron, E., Huneau, B., 2014.  How to identify a hyperelastic constitutive equation for rubber-like materials with multiaxial tension–torsion experiments.  International Journal of Non-Linear Mechanics 65, 260–270.
  * Mars (2001) Mars, W., 2001.  Cracking energy density as a predictor of fatigue life under multiaxial conditions.  Rubber Chemistry and Technology 75, 419–424.
  * Murphy (2015) Murphy, J., 2015.  The stability of thin, stretched and twisted elastic rods.  International Journal of Non-Linear Mechanics 68, 96–100.
  * Ogden (1997) Ogden, R., 1997.  Non-linear elastic deformations.  Dover Publications, Inc. Mineola, New York (First published by Ellis Harwood Ltd. in 1984).
  * Papadopoulos (1998) Papadopoulos, I.V., 1998.  Critical plane approaches in high-cycle fatigue: on the definition of the amplitude and mean value of the shear stress acting on the critical plane.  Fatigue & Fracture of Engineering Materials & Structures 21, 269–285.
  * Rivlin (1949) Rivlin, R., 1949.  Large elastic deformations of isotropic materials VI. Further results in the theory of torsion, shear and flexure.  Philosophical Transactions of the Royal Society of London. Series A. 242, 173–195.
  * Saintier et al. (2006) Saintier, N., André, N., Cailletaud, G., Piques, R., 2006\.  Crack nucleation and propagation under multiaxial fatigue in Natural Rubber.  International Journal of Fatigue 28, 61–72.
  * Thiel et al. (2019) Thiel, C., Voss, J., Martin, R.J., Neff, P., 2019\.  Shear, pure and simple.  International Journal of Non-Linear Mechanics 112, 57–72.
  * Valiollahi et al. (2019) Valiollahi, A., Shojaeifard, M., Baghani, M., 2019.  Closed form solutions for large deformation of cylinders under combined extension-torsion.  International Journal of Mechanical Sciences 157, 336–347.

##  Appendix A Change of basis: proof and example

###  A.1 Proof

So as to compute the coefficients $\alpha_{i}$ of the decomposition of any
zero-trace symmetric tensor, we simply apply the usual formula for the
rotation of a matrix: let $\mathbf{h}\in E$, and $\mathcal{B}_{\mathrm{I}}$
and $\mathcal{B}_{\mathrm{II}}$ two bases of the cartesian space in which one
can express the tensor $\mathbf{h}$ through two matrices
${h}_{\mathcal{B}_{\mathrm{I}}}$ and ${h}_{\mathcal{B}_{\mathrm{II}}}$,
respectively. The formula that links these matrices reads as follows:

${h}_{\mathcal{B}_{\mathrm{II}}}={P}^{-1}{h}_{\mathcal{B}_{\mathrm{I}}}{P}={P}^{-1}\sum_{i=1}^{5}\alpha_{i}{Y}^{(i)}_{\mathcal{B}_{\mathrm{I}}}{P}=\sum_{i=1}^{5}\alpha_{i}\underbrace{{P}^{-1}{Y}^{(i)}_{\mathcal{B}_{\mathrm{I}}}{P}}_{={A}^{(i)}_{\mathcal{B}_{\mathrm{II}}}}$
(21)

where ${P}$ is the rotation matrix from $\mathcal{B}_{\mathrm{I}}$ to
$\mathcal{B}_{\mathrm{II}}$. Here, ${A}^{(i)}_{\mathcal{B}_{\mathrm{II}}}$ is
the representation in $\mathcal{B}_{\mathrm{II}}$ of a new tensor
$\mathbf{A}\in E$, to which one can apply again the rotation formula on the
$\mathbf{Y^{(i)}}$ basis. Its representation as a matrix in the basis
$\mathcal{B}_{\mathrm{II}}$ is:

${A}^{(i)}_{\mathcal{B}_{\mathrm{II}}}=\sum_{j=1}^{5}\beta^{(i)}_{j}{Y}^{(j)}_{\mathcal{B}_{\mathrm{II}}}$
(22)

Combining Equation (21) and (22) leads to:

$\displaystyle{h}_{\mathcal{B}_{\mathrm{II}}}=$
$\displaystyle\sum_{i=1}^{5}\left(\alpha_{i}\sum_{j=1}^{5}\beta^{(i)}_{j}{Y}^{(j)}_{\mathcal{B}_{\mathrm{II}}}\right)$
(23) $\displaystyle=$
$\displaystyle\alpha_{1}\left(\beta_{1}^{(1)}{Y}^{(1)}_{\mathcal{B}_{\mathrm{I}}}+\beta_{2}^{(1)}{Y}^{(2)}_{\mathcal{B}_{\mathrm{I}}}+...+\beta_{5}^{(1)}{Y}^{(5)}_{\mathcal{B}_{\mathrm{I}}}\right)$
$\displaystyle+...$
$\displaystyle+\alpha_{5}\left(\beta_{1}^{(5)}{Y}^{(1)}_{\mathcal{B}_{\mathrm{I}}}+\beta_{2}^{(5)}{Y}^{(2)}_{\mathcal{B}_{\mathrm{I}}}+...+\beta_{5}^{(5)}{Y}^{(5)}_{\mathcal{B}_{\mathrm{I}}}\right)$

Finally regrouping the ${Y}^{(j)}_{\mathcal{B}_{\mathrm{I}}}$ together:

$\displaystyle{h}_{\mathcal{B}_{\mathrm{II}}}=$
$\displaystyle\left(\alpha_{1}\beta_{1}^{(1)}+\alpha_{2}\beta_{1}^{(2)}+...+\alpha_{5}\beta_{1}^{(5)}\right){Y}^{(1)}_{\mathcal{B}_{\mathrm{I}}}$
(24) $\displaystyle+...$
$\displaystyle+\left(\alpha_{1}\beta_{5}^{(1)}+\alpha_{2}\beta_{5}^{(2)}+...+\alpha_{5}\beta_{5}^{(5)}\right){Y}^{(5)}_{\mathcal{B}_{\mathrm{I}}}$

Equation (24) provides a formula to compute the expression of the $\alpha_{i}$
in a rotated basis, that can be denoted $\alpha_{i}^{\mathrm{rot}}$:

$\begin{cases}{h}_{\mathcal{B}_{\mathrm{II}}}=\sum_{j=1}^{5}\alpha^{\mathrm{rot}}_{j}{Y}^{(j)}_{\mathcal{B}_{\mathrm{II}}}\\\
\alpha^{\mathrm{rot}}_{j}=\sum_{k=1}^{5}\alpha_{k}\beta^{(k)}_{j}\end{cases}$
(25)

where the $\alpha_{k}$ are the coefficients of the projection
${h}_{\mathcal{B}_{\mathrm{I}}}$ of $\mathbf{h}$ in the initial basis
$\mathcal{B}_{\mathrm{I}}$, and $\beta^{(k)}_{j}$ is the $j^{\mathrm{th}}$
coefficient of the $k^{\mathrm{th}}$ matrix
${Y}^{(k)}_{\mathcal{B}_{\mathrm{II}}}$ in the new basis
$\mathcal{B}_{\mathrm{II}}$.

###  A.2 Example on the case of the uniaxial tension-torsion of a perfect
cylinder

Let illustrate the previous Equation (25) for the uniaxial tension-torsion of
a perfect cylinder. As described in Section 2, the Hencky tensor $\mathbf{h}$
can be represented, in the cylindrical basis
$\mathcal{B}_{\mathrm{\mathrm{cyl}}}$, by a symmetric matrix with zeros in
components $\overrightarrow{e_{r}}\otimes\overrightarrow{e_{\theta}}$ and
$\overrightarrow{e_{r}}\otimes\overrightarrow{e_{z}}$. This leads to a reduced
projection on the $\mathbf{Y^{(i)}}$ basis consisting of only three
components: $\\{\alpha_{1},\alpha_{2},\alpha_{3}\\}$. Here, the cylindrical
basis is rotated three times, according to one direction at each step, so that
none of the elements of the final basis is a principal direction of the Hencky
tensor $\mathbf{h}$. The rotations are defined as follows:

  1. 1.

A first rotation by an angle $\delta_{1}$ ($=75^{\circ}$ for illustration)
around the axis $\overrightarrow{e_{z}}$ produces a new basis
$\mathcal{B}_{\mathrm{I}}=\left(\overrightarrow{e_{1}}^{(1)},\overrightarrow{e_{2}}^{(1)},\overrightarrow{e_{z}}\right)$.
The rotation matrix between $\mathcal{B}_{\mathrm{\mathrm{cyl}}}$ and
$\mathcal{B}_{\mathrm{1}}$ reads:

${P}_{\mathcal{B}_{\mathrm{cyl}}\rightarrow\mathcal{B}_{1}}=\left[\begin{array}[]{rrr}\cos(\delta_{1})&-\sin(\delta_{1})&0\\\
\sin(\delta_{1})&\cos(\delta_{1})&0\\\
0&0&1\end{array}\right]_{\mathcal{B}_{\mathrm{cyl}}}$ (26)

  2. 2.

A second rotation by an angle $\delta_{2}$ ($=110^{\circ}$ for illustration)
around the axis $\overrightarrow{e_{1}}^{(1)}$ produces a new basis
$\mathcal{B}_{\mathrm{II}}=\left(\overrightarrow{e_{1}}^{(1)},\overrightarrow{e_{2}}^{(2)},\overrightarrow{e_{3}}^{(2)}\right)$.
The rotation matrix between $\mathcal{B}_{\mathrm{1}}$ and
$\mathcal{B}_{\mathrm{2}}$ reads:

${P}_{\mathcal{B}_{1}\rightarrow\mathcal{B}_{2}}=\left[\begin{array}[]{rrr}1&0&0\\\
0&\cos(\delta_{2})&-\sin(\delta_{2})\\\
0&\sin(\delta_{2})&\cos(\delta_{2})\end{array}\right]_{\mathcal{B}_{\mathrm{1}}}$
(27)

  3. 3.

A last rotation by an angle $\delta_{3}$ ($=38^{\circ}$ for illustration)
around the axis $\overrightarrow{e_{2}}^{(2)}=\overrightarrow{e_{2}}$ produces
a new basis
$\mathcal{B}_{\mathrm{2}}=\left(\overrightarrow{e_{1}},\overrightarrow{e_{2}},\overrightarrow{e_{3}}\right)$.
The rotation matrix between $\mathcal{B}_{\mathrm{2}}$ and
$\mathcal{B}_{\mathrm{3}}$ reads:

${P}_{\mathcal{B}_{2}\rightarrow\mathcal{B}_{3}}=\left[\begin{array}[]{rrr}\cos(\delta_{3})&0&-\sin(\delta_{3})\\\
0&1&0\\\
\sin(\delta_{3})&0&\cos(\delta_{3})\end{array}\right]_{\mathcal{B}_{\mathrm{2}}}$
(28)

Combining the three rotations into one, the Hencky tensor is rotated by the
following global rotation matrix:

${P}_{\mathcal{B}_{\mathrm{cyl}}\rightarrow\mathcal{B}_{3}}={P}_{\mathcal{B}_{2}\rightarrow\mathcal{B}_{3}}{P}_{\mathcal{B}_{1}\rightarrow\mathcal{B}_{2}}{P}_{\mathcal{B}_{\mathrm{cyl}}\rightarrow\mathcal{B}_{1}}$
(29)

Figure 6 shows both the cylindrical basis
$\mathcal{B}_{\mathrm{\mathrm{cyl}}}$ and the final basis
$\mathcal{B}_{\mathrm{3}}$.

Figure 6: Representation of the cylindrical basis
$\mathcal{B}_{\mathrm{\mathrm{cyl}}}$ (black) and the rotated basis
$\mathcal{B}_{\mathrm{3}}$ (red).

In $\mathcal{B}_{\mathrm{3}}$, $\mathbf{h}$ is a priori represented by a
complete matrix, i.e. without zero coefficients. Using the same loading
conditions as in the paper ($\lambda=1.39$, $\tau=0.124\text{ rad.mm}^{-1}$,
$\phi=180^{\circ}$), both the evolutions of $\alpha_{i}$ and
$\alpha^{\mathrm{rot}}$ are displayed in Figure 7. They correspond to the
coefficients of the projection of ${h}_{\mathcal{B}_{\mathrm{\mathrm{cyl}}}}$
and ${h}_{\mathcal{B}_{\mathrm{3}}}$ respectively.

Figure 7: Evolution of the $\\{\alpha_{i}(t)\\}_{\\{i=1,..,5\\}}$ and
$\\{\alpha_{i}^{\mathrm{rot}}(t)\\}_{\\{i=1,..,5\\}}$ for out-of-phase
uniaxial tension-torsion ($\lambda=1.39$, $\tau=0.124\text{ rad.mm}^{-1}$,
$\phi=180^{\circ}$). Both lines (black and red) correspond to a different
expression of the same tensor.

Finally, Table 2 presents some results related to the computation of the MCS
applied to both ${h}_{\mathcal{B}_{\mathrm{\mathrm{cyl}}}}$ and
${h}_{\mathcal{B}_{\mathrm{3}}}$:

  * •

The MCS have the same radius.

  * •

The position $\alpha_{i}^{\mathrm{C}}$ of the center of each MCS is obviously
different, but applying the rotation formula to the center
${h}_{\mathcal{B}_{\mathrm{\mathrm{cyl}}}}^{C}$ shows the relationship between
the centers of both MCS :
${P}^{-1}\mathbf{h}^{\mathrm{C}}{P}=\mathbf{h}^{\mathrm{rot}^{\mathrm{C}}}$.

Tensor | Radius | $\alpha_{1}^{\mathrm{C}}$ | $\alpha_{2}^{\mathrm{C}}$ | $\alpha_{3}^{\mathrm{C}}$ | $\alpha_{4}^{\mathrm{C}}$ | $\alpha_{5}^{\mathrm{C}}$
---|---|---|---|---|---|---
$\mathbf{h}$ | 0.526 | 0.019 | 0.105 | 0.348 | 0 | 0
$\mathbf{h}^{\mathrm{rot}}$ | 0.526 | 0.221 | 0.061 | 0.173 | -0.223 | -0.023
${P}^{-1}\mathbf{h}^{\mathrm{C}}{P}$ | - | 0.221 | 0.061 | 0.173 | -0.223 | -0.023
Table 2: Properties of the MCS computed for each basis:
$\mathcal{B}_{\mathrm{\mathrm{cyl}}}$ et $\mathcal{B}_{\mathrm{3}}$.


# Robust Weighted Sum-Rate Maximization for Transmissive RIS Transmitter
Enabled RSMA Networks

Bojiang Li, Wen Chen, Zhendong Li, Qingqing Wu, Nan Cheng, Changle Li,
Linglong Dai This work is supported by National key project 2020YFB1807700,
NSFC 62071296, Shanghai 22JC1404000, 20JC1416502, and PKX2021-D02. B. Li, W.
Chen, Z. Li, and Q. Wu are with the Department of Electronic Engineering,
Shanghai Jiao Tong University, Shanghai 200240, China (e-mail:
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>qingqingwu@sjtu.edu.cn). N. Cheng and C. Li are with the State Key Laboratory
of Integrated Services Networks, Xidian University, Xi’an 710071, China
(e-mail<EMAIL_ADDRESS>clli@mail.xidian.edu.cn). L. Dai is with the
Department of Electronic Engineering, Tsinghua University, Beijing 100084,
China (e-mail:daill@tsinghua.edu.cn). Corresponding author: Wen Chen

###### Abstract

Due to the low power consumption and low cost nature of transmissive
reconfigurable intelligent surface (RIS), in this paper, we propose a downlink
multi-user rate-splitting multiple access (RSMA) architecture based on the
transmissive RIS transmitter, where the channel state information (CSI) is
only accquired partially. We investigate the weighted sum-rate maximization
problem by jointly optimizing the power, RIS transmissive coefficients and
common rate allocated to each user. Due to the coupling of optimization
variables, the problem is non-convex, and it is difficult to directly obtain
the optimal solution. Hence, a block coordinate descent (BCD) algorithm based
on sample average approximation (SAA) and weighted minimum mean square error
(WMMSE) is proposed to tackle it. Numerical results illustrate that the
transmissive RIS transmitter with rate-splitting architecture has advantages
over conventional space division multiple access (SDMA) and non-orthgonal
multiple access (NOMA).

###### Index Terms:

Transmissive RIS transmitter, RSMA, inaccurate CSI, WMMSE

##  I Introduction

Recently, a novel multiple access method, namely, rate splitting multiple
access (RSMA) has been proposed based on the space division multiple access
(SDMA) and non-orthogonal multiple access (NOMA). RSMA is considered as a
feasible technique which combines two decoding schemes of SDMA and NOMA, where
SDMA fully treats the interference from other users as noises while NOMA fully
decodes the interference [1]. Since both two schemes can only suit for
extremely weak or strong interference levels, and are sensitive to the
inaccuracy of channel state information at the transmitter (CSIT), RSMA is
proposed to overcome these drawback, whose excellent performance is
demonstrated by the adaptive interference management strategy and the
robustness under inaccurate CSIT [2]. Specifically, RSMA enables each user’s
message is split into common message and private message at the transmitter
side and recovered at the user side by successive interference cancellation
(SIC) [3]. Such scheme achieves a more flexible and effective interference
management by partially decoding the interference and partially treating the
remaining interference as noise.

On the other hand, the base stations of 5G network face the challenges of
higher power consumption and deployment costs. The reconfigurable intelligent
surface (RIS) is considered as an effective solution to tackle these
difficulties. RIS is an array composed of a large number of low-cost passive
elements; each element can be controlled to adjust the amplitude and phase
shift of the incident electromagnetic wave to enable beamforming [4], [5],
[6]. The application mode of RIS can be divided into reflective RIS and
transmissive RIS, where the base stations (BSs) and the users are on the same
side for reflective RIS and on the different side for transmissive RIS [7].
The communication system with reflective RIS has been well studied [8], [9].
Transmissive RIS transmitter actually outperforms reflective RIS due to less
feed blockage. [10].

Based on the previous work [4], we introduce a transmissive RIS transmitter
architecture for downlink multi-user RSMA network. The transmissive RIS
transmitter can be a good alternative to multi-antenna systems and results in
less power consumption and less cost in the RSMA architecture. [10] Based on
this system, a robust weighted sum-rate maximization problem to obtain common
rate, RIS transmissive coefficients and power allocation is formulated. In
order to make the problem solvable, we propose a block coordinate descent
(BCD) algorithm based on sample average approximation (SAA) and weighted
minimum mean square error (WMMSE) to obtain a high-quality suboptimal solution
to this problem.

##  II System Model and Problem Formulation

Figure 1: Transmissive RIS transmitter enabled RSMA architecture.

###  II-A System Model

As illustrated in Fig. 1, we consider a downlink multi-user RSMA system, where
the transmitter is composed of a feed antenna and a transmissive RIS panel
with $N$ sub-arrays serving $K$ single-antenna users, indexed by
$\mathcal{N}=\\{1,\ldots,N\\}$ and $\mathcal{K}=\\{1,\ldots,K\\}$, and each
sub-array includes $N_{e}$ elements. Besides, the RIS controller connecting
the antenna to the panel is used to control the transmissive coefficient of
RIS. It is worth noting that each sub-array is equivalent to an antenna and
works independently of each other controlled by the RIS controller. The signal
received by the $k$-th user is defined as

$y_{k}=\mathbf{h}_{k}^{H}\mathbf{x}+n_{k},\forall k\in\mathcal{K},$ (1)

where $\mathbf{h}_{k}\in\mathbb{C}^{N}$ is the channel vector bewteen the
transmitter and the $k$-th user, $\mathbf{x}\in\mathbb{C}^{N}$ is the
transmission signal and $n_{k}\sim\mathcal{C}\mathcal{N}(0,\sigma_{n,k}^{2})$
is the additive white Gaussian noise (AWGN) at the $k$-th user. Without loss
of generality, we assume that equal noise variances for all users, i.e.,
$\sigma_{n,k}^{2}=\sigma_{n}^{2}$. It is worth noting that the channel bewteen
the feed antenna and the transmissive RIS panel is not considered since the
signals are transmitted from RIS to users.

In 1-layer rate-splitting architecture, each user’s message $W_{k}$ is split
into a common part $W_{c,k}$ and a private part $W_{p,k}$ at the transmitter
side [1]. The common part of each user $W_{c,k}$ can be combined into $W_{c}$.
Hence, $K+1$ messages $W_{c},W_{p,1}\ldots,W_{p,K}$ are sent by the
transmitter instead of $K$ messages, and they are encoded into the independent
data streams
$\mathbf{s}=[s_{c},s_{1},s_{2},\ldots,s_{K}]^{H}\in\mathbb{C}^{K+1}$, where
$\mathbb{E}[\mathbf{ss}^{\mathrm{H}}]=\textbf{I}$. Let
$\mathbf{p}=[p_{c},p_{1},\ldots,p_{K}]^{T}\in\mathbb{R}^{K+1}$ denote the
transmit power vector, where $p_{c}$ and $p_{k}$ represent the power
aollocated to common stream $s_{c}$ and the $k$-th private stream $s_{k}$,
respectively.

In the proposed architecture, we assume that multiple elements of each sub-
array can serve one stream, i.e. $N_{e}\geq K+1$. Let
$\mathbf{F}=[\mathbf{f}_{c},\mathbf{f}_{1},\ldots,\mathbf{f}_{K}]\in\mathbb{C}^{N\times(K+1)}$
denote the transmissive coefficients matrix of RIS, where
$\mathbf{f}_{c}=[f_{c,1},\ldots,f_{c,N}]^{H}$ and
$\mathbf{f}_{k}=[f_{k,1},\ldots,f_{k,N}]^{H}$ represent the RIS transmissive
coefficient vector used to transmit the common stream and the $k$-th private
stream, respectively. Thus, the transmission signal from the transmitter can
be written as

$\mathbf{x}=\mathbf{f}_{c}\sqrt{p_{c}}s_{c}+\sum_{j=1}^{K}\mathbf{f}_{j}\sqrt{p_{j}}s_{j}.$
(2)

At the receiver side, each user decodes the common stream $s_{c}$ at first and
extracts the corresponding common message $\hat{W}_{c,k}$ from combined
$\hat{W}_{c}$ by treating the interference from all the private stream as
noise. SIC is later used to obtain private message $\hat{W}_{p,k}$ from the
corresponding private stream $s_{k}$ by decoding the interference stronger
than the user. Then, $\hat{W}_{c,k}$ and $\hat{W}_{p,k}$ are recombined into
$\hat{W}_{k}$ to recover the original message of the $k$-th user.
Consequently, the achievable rate of common stream $R_{c,k}$ and private
stream $R_{p,k}$ for the $k$-th user can be represented as

$R_{i,k}=\log_{2}(1+\gamma_{i,k}),\forall k\in\mathcal{K},i\in\\{c,p\\}.$ (3)

where $\gamma_{\mathrm{i},k}$ is the signal-to-interference-plus-noise ratio
(SINR) of the common or private stream at the $k$-th user, which can be
expressed as

$\displaystyle\gamma_{c,k}$
$\displaystyle=\frac{p_{c}|\mathbf{h}_{k}^{H}\mathbf{f}_{c}|^{2}}{\sum\limits_{j\in\mathcal{K}}p_{j}|\mathbf{h}_{k}^{H}\mathbf{f}_{j}|^{2}+\sigma_{n}^{2}},\forall
k\in\mathcal{K},$ (4a) $\displaystyle\gamma_{p,k}$
$\displaystyle=\frac{p_{k}|\mathbf{h}_{k}^{H}\mathbf{f}_{k}|^{2}}{\sum\limits_{j\in\mathcal{K^{\prime}}}p_{j}|\mathbf{h}_{k}^{H}\mathbf{f}_{j}|^{2}+\sigma_{n}^{2}},\forall
k\in\mathcal{K},$ (4b)

where $\mathcal{K^{\prime}}=\mathcal{K}\backslash\\{k\\}$. To ensure
$\textit{W}_{c}$ can be decoded by all users successfully, the summation of
actual common rate is required not to exceed the common rate of each user,
i.e., $R_{c}\leq R_{c,k},\forall k\in\mathcal{K}.$ Moreover, the actual common
rate of each user is $C_{k}$, which is a portion of $R_{c}$ and corresponds to
the theoretical maximum common rate $R_{c,k}$. The common rate allocation
$C_{1},\ldots,C_{K}$ satisfies $\sum_{k\in\mathcal{K}}C_{k}=R_{c}$. Hence, the
achievable sum rate of the $k$-th user is defined as
$R_{k,sum}=C_{k}+R_{p,k},\forall k\in\mathcal{K}.$

In this paper, we consider a more practical scenario where the CSI obtained by
the transmitter is not accurate. For each channel vector
$\mathbf{h}_{k},\widehat{\mathbf{h}}_{k}$ denotes the estimated instantaneous
channel and $\widetilde{\mathbf{h}}_{k}$ is the estimation error. The
relationship of them can be represented as

$\mathbf{h}_{k}=\widehat{\mathbf{h}}_{k}+\widetilde{\mathbf{h}}_{k},\forall
k\in\mathcal{K},$ (5)

where $\widehat{\mathbf{h}}_{k}$ is considered to be known in the following
discussion and $\widetilde{\mathbf{h}}_{k}$ follows the distribution of a
circularly symmetric complex Gaussian (CSCG) random vector, i.e.
$\mathcal{C}\mathcal{N}(0,\sigma_{k}^{2})$.

Due to the uncertainty of CSI, maximizing the instantaneous WSR may lead to
transmission at undecodable rates and thus impair the system performance.
Therefore, we replace WSR by the weighted ergodic sum rate (WESR) to obtain a
better evaluation of the long-term WSR performance, which is defined as

$\mathrm{WESR}\triangleq\sum\nolimits_{k\in\mathcal{K}}u_{k}\mathbb{E}_{\\{\mathbf{h}_{k},\widehat{\mathbf{h}}_{k}\\}}\\{R_{\mathrm{p},k}+C_{k}\\},$
(6)

where $u_{k}$ is the WESR weight allocated to the $k$-th user. However, due to
the non-linear relationship bewteen $\mathbf{h}_{k}$ and $R_{i,k}$, it is
difficult to get the probability density distribution of the transmission
rate. We find that the long-term ergodic rate (ER) performance can be
characterized by the short-term average rate (AR) performance when the number
of samples is large enough. The relationship between ER and AR can be found in
Eq. 8 in [11], where
$\bar{R}_{i,k}(\widehat{\mathbf{h}}_{k})\triangleq\mathbb{E}_{\mathbf{h}_{k}\mid\widehat{\mathbf{h}}_{k}}\\{R_{i,k}\mid\widehat{\mathbf{h}}_{k}\\}$
is the AR under given channel estimate $\widehat{\mathbf{h}}_{k}$.

###  II-B Problem Formulation

According to the analysis above, the weighted average sum rate (WASR)
maximization problem under all given channel estimates can be formulated as:

$\displaystyle(\text{P1}):\max_{\mathbf{p},\mathbf{F},\mathbf{c}}$
$\displaystyle\sum\limits_{k\in\mathcal{K}}u_{k}(\bar{R}_{p,k}+C_{k}),$ (7a)
s.t. $\displaystyle\sum\limits_{k\in\mathcal{K}}C_{k}\leq\bar{R}_{c,k},\forall
k\in\mathcal{K},$ (7b) $\displaystyle C_{k}+\bar{R}_{p,k}\geq
R_{k}^{th},\forall k\in\mathcal{K},$ (7c) $\displaystyle\mathbf{c}\geq 0,$
(7d) $\displaystyle p_{c}+\sum\limits_{k\in\mathcal{K}}p_{k}\leq
P_{t},\mathbf{p}\geq 0,$ (7e) $\displaystyle|f_{c,n}|\leq 1,|f_{k,n}|\leq
1,\forall k\in\mathcal{K},n\in\mathcal{N},$ (7f)

where $\mathbf{c}=[C_{1},\ldots,C_{K}]^{T}$. Constraint (7b) ensures that each
user is able to decode the common stream successfully. Constraint (7c)
guarantees that the rate of each user is no less than a certain threshold,
where $R_{k}^{th}$ is the quality of service (QoS) threshold. Constraints (7d)
and (7e) specify the range of the common rate allocation vector and power
allocation vector, where $P_{t}$ represents the maximum transmit power of the
RIS transmitter. Constraint (7f) limits the amplitude of each RIS element. It
can be seen that the problem (P1) is intractable due to the non-convexity of
the objective function Eq. (7a) and the constraints (7b), (7c). To resolve the
difficulty, we apply the BCD algorithm based on SAA and WMMSE [11], the
specific steps are detailed in next section.

##  III Solution to Optimization Problem

To obtain AR as precise as possible, a sample average approximation method as
follows is used to approach the real rates by sampling a large number of
channel estimates.

###  III-A Sample Average Approximation

Referring to the SAA method in [11], we can obtain the AR by averaging M rate
samples, which can be defined as

$\displaystyle\bar{R}_{i,k}^{(M)}(\widehat{\mathbf{h}}_{k})\triangleq\frac{1}{M}\sum_{m=1}^{M}R_{i,k}^{(m)}(\widehat{\mathbf{h}}_{k}),\forall
k\in\mathcal{K},i\in\\{c,p\\},$ (8a)
$\displaystyle\bar{R}_{i,k}(\widehat{\mathbf{h}}_{k})=\lim_{M\rightarrow\infty}\bar{R}^{(M)}_{i,k}(\widehat{\mathbf{h}}_{k}),\forall
k\in\mathcal{K},i\in\\{c,p\\}.$ (8b)

Consequently, the problem (P1) can be transformed into a more deterministic
form to solve as follows:

$\displaystyle(\text{P2}):\max_{\mathbf{p},\mathbf{F},\mathbf{c}}$
$\displaystyle\sum\limits_{k\in\mathcal{K}}u_{k}(\bar{R}_{p,k}^{(M)}+C_{k}),$
(9a) s.t.
$\displaystyle\sum\limits_{k\in\mathcal{K}}C_{k}\leq\bar{R}_{c,k}^{(M)},\forall
k\in\mathcal{K},$ (9b) $\displaystyle C_{k}+\bar{R}_{p,k}^{(M)}\geq
R_{k}^{th},\forall k\in\mathcal{K},$ (9c)
$\displaystyle(\rm{\ref{equation7d}}),(\rm{\ref{equation7e}}),(\rm{\ref{equation7f}}).$
(9d)

It is worth noting that the coupling of optimization variables still exists in
the problem (P2). Hence, we introduce a WMMSE algorithm to solve it.

###  III-B WMMSE Algorithm

To address problem (P2), the WMMSE algorithm is utilized to construct the
rate-WMMSE relationship. For any user, the estimated common stream is first
decoded and the private one is later decoded after subtracting the received
common stream, which are respectively represented as
$\hat{s}_{c,k}=g_{c,k}y_{k}$ and
$\hat{s}_{p,k}=g_{p,k}(y_{k}-\sqrt{p_{c}}\mathbf{h}_{k}^{H}\mathbf{f}_{c}s_{c})$,
where $g_{c,k}$ and $g_{p,k}$ are the equalizer of the corresponding stream.
Hence, the common and private mean square error (MSE) at the $k$-th user are
respectively denoted by

$\displaystyle\varepsilon_{c,k}$
$\displaystyle\triangleq\mathbb{E}\left\\{|\hat{s}_{c,k}-s_{c}|^{2}\right\\}$
$\displaystyle=|g_{c,k}|^{2}T_{c,k}-2\Re\\{\sqrt{p_{c}}g_{c,k}\mathbf{h}_{k}^{H}\mathbf{f}_{c}\\}+1,\forall
k\in\mathcal{K},$ (10a) $\displaystyle\varepsilon_{p,k}$
$\displaystyle\triangleq\mathbb{E}\left\\{|\hat{s}_{p,k}-s_{k}|^{2}\right\\}$
$\displaystyle=|g_{p,k}|^{2}T_{p,k}-2\Re\\{\sqrt{p_{k}}g_{p,k}\mathbf{h}_{k}^{H}\mathbf{f}_{k}\\}+1,\forall
k\in\mathcal{K},$ (10b)

where

$\displaystyle T_{c,k}$
$\displaystyle=p_{c}\left|\mathbf{h}_{k}^{H}\mathbf{f}_{c}\right|^{2}+\textstyle\sum_{j\in\mathcal{K}}p_{j}\left|\mathbf{h}_{k}^{H}\mathbf{f}_{j}\right|^{2}+\sigma_{k}^{2},\forall
k\in\mathcal{K},$ (11a) $\displaystyle T_{p,k}$
$\displaystyle=T_{c,k}-p_{c}\left|\mathbf{h}_{k}^{H}\mathbf{f}_{c}\right|^{2}=I_{c,k},\forall
k\in\mathcal{K},$ (11b) $\displaystyle I_{p,k}$
$\displaystyle=T_{p,k}-p_{k}\left|\mathbf{h}_{k}^{H}\mathbf{f}_{k}\right|^{2},\forall
k\in\mathcal{K}.$ (11c)

The optimal MSE equalizers at the $k$-th user are obtained by letting
$\frac{\partial\varepsilon_{p,k}}{\partial g_{p,k}}=0$ and
$\frac{\partial\varepsilon_{c,k}}{\partial g_{c,k}}=0$, which are

$\displaystyle g_{c,k}^{\mathrm{MSE}}$
$\displaystyle=\sqrt{p_{c}}\mathbf{h}_{k}^{H}\mathbf{f}_{c}T_{c,k}^{-1},\forall
k\in\mathcal{K},$ (12a) $\displaystyle g_{p,k}^{\mathrm{MSE}}$
$\displaystyle=\sqrt{p_{k}}\mathbf{h}_{k}^{H}\mathbf{f}_{k}T_{p,k}^{-1},\forall
k\in\mathcal{K}.$ (12b)

The minimum MSE in Eq. (10) can be rewritten by substituting Eq. (12) into it,
i.e.,

$\varepsilon_{i,k}^{\mathrm{MSE}}\triangleq\min_{g_{i,k}}\varepsilon_{i,k}=T_{i,k}^{-1}I_{i,k},\forall
k\in\mathcal{K},i\in\\{c,p\\}.$ (13)

It is not difficult to find
$\gamma_{i,k}=(1/\varepsilon_{i,k}^{\mathrm{MSE}})-1,$ and
$R_{i,k}=-\log_{2}(\varepsilon_{i,k}^{\mathrm{MSE}})$. Futhermore, the
weighted mean square errors (WMSEs) are introduced to convert the non-convex
rates into convex forms, which can be expressed as

$\xi_{i,k}=\omega_{i,k}\varepsilon_{i,k}-\log_{2}\left(\omega_{i,k}\right),\forall
k\in\mathcal{K},i\in\\{c,p\\},$ (14)

where $\omega_{i,k}$ is the weight of MSE corresponding to decoding either
common stream or private stream. The specific WMSE expressions for
$\mathbf{p}$ and $\mathbf{F}$ are respectively expressed in Eq. (15a) and Eq.
(15b).

$\displaystyle\xi_{c,k}$
$\displaystyle=\omega_{c,k}\left|g_{c,k}\right|^{2}(p_{c}\left|\mathbf{h}_{k}^{\mathrm{H}}\mathbf{f}_{c}\right|^{2}+\sum\limits_{j=1}^{K}p_{j}\left|\mathbf{h}_{k}^{H}\mathbf{f}_{j}\right|^{2}+\sigma_{k}^{2})-2\Re\\{\sqrt{p_{c}}\omega_{c,k}g_{c,k}\mathbf{h}^{H}_{k}\mathbf{f}_{c}\\}+\omega_{c,k}-\log_{2}(\omega_{c,k}),\forall
k\in\mathcal{K},$ (15a) $\displaystyle\xi_{p,k}$
$\displaystyle=\omega_{p,k}\left|g_{p,k}\right|^{2}(\sum\limits_{j=1}^{K}p_{j}\left|\mathbf{h}_{k}^{H}\mathbf{f}_{j}\right|^{2}+\sigma_{k}^{2})-2\Re\\{\sqrt{p_{k}}\omega_{p,k}g_{p,k}\mathbf{h}^{H}_{k}\mathbf{f}_{k}\\}+\omega_{p,k}-\log_{2}(\omega_{p,k}),\forall
k\in\mathcal{K},$ (15b)

The expressions in Eq. (15) are convex with respect to $\mathbf{p}$ and
$\mathbf{F}$. By optimizing WMSE weights and equalizers, the optimum WMSE can
be derived by making $\frac{\partial\xi_{i,k}}{\partial g_{i,k}}=0$ and
$\frac{\partial\xi_{i,k}}{\partial\omega_{i,k}}=0$, i.e.,

$\displaystyle\omega_{i,k}^{*}=\omega_{i,k}^{\mathrm{MSE}}\triangleq(\varepsilon_{i,k}^{\mathrm{MSE}})^{-1},\forall
k\in\mathcal{K},i\in\\{c,p\\},$ (16a) $\displaystyle
g_{i,k}^{*}=g_{i,k}^{\mathrm{MSE}},\forall k\in\mathcal{K},i\in\\{c,p\\}.$
(16b)

Substituting this back into Eq. (14) yields the relationship as follows:

$\xi_{i,k}^{\mathrm{MSE}}\triangleq\min_{\omega_{i,k},g_{i,k}}\xi_{i,k}=1-R_{i,k},\forall
k\in\mathcal{K},i\in\\{c,p\\}.$ (17)

Thus, we establish the relationship between WMMSE and rate. The SAFs
corresponding to WMSEs are shown as

$\bar{\xi}_{i,k}^{(M)}=\frac{1}{M}\sum_{m=1}^{M}\xi_{i,k}^{(m)},\forall
k\in\mathcal{K},i\in\\{c,p\\},$ (18)

which eliminates the effect of channel estimation errors to WMSE theoretically
if $M$ tends to infinity.

For problem (P2), we reformulate the problem by replacing rates into WMMSE
forms and minimizing the objective function. In detail, let
$\mathbf{G}=\big{\\{}g_{c,k}^{\mathrm{MSE}(m)},g_{p,k}^{\mathrm{MSE}(m)}\big{\\}}$,
and
$\bm{\Omega}=\big{\\{}\omega_{c,k}^{\mathrm{MSE}(m)},\omega_{p,k}^{\mathrm{MSE}(m)}\big{\\}}$,
where $m\in\mathcal{M},k\in\mathcal{K}$. The problem is reformulated as

$\displaystyle(\text{P3}):\min_{\mathbf{p},\mathbf{F},\mathbf{c},\mathbf{G},\mathbf{\Omega}}$
$\displaystyle\sum\limits_{k\in\mathcal{K}}u_{k}(\bar{\xi}_{p,k}^{(M)}-C_{k})$
(19a) s.t.
$\displaystyle\bar{\xi}_{c,k}^{(M)}+\sum\limits_{k\in\mathcal{K}}C_{k}\leq
1,\forall k\in\mathcal{K},$ (19b)
$\displaystyle\bar{\xi}_{p,k}^{(M)}-C_{k}\leq 1-R_{k}^{th},\forall
k\in\mathcal{K},$ (19c) $\displaystyle(\rm{7d}),(\rm{7e}),(\rm{7f}).$ (19d)

###  III-C Block Coordinate Descent Algorithm

Although problem (P3) is non-convex for multi optimization variables, it is
convex for a single optimization variable when the others are fixed. We
utilize BCD algorithm to decouple and alternately optimize $\mathbf{p}$ and
$\mathbf{F}$ by iteratively updating $\mathbf{G}$ and $\mathbf{\Omega}$ until
the WASR convergences. For convenience, we simplify some parameters except
$\mathbf{p}$ and $\mathbf{F}$ in Eq. (15), i.e.,

$\displaystyle t_{i,k}^{(m)}$
$\displaystyle=\omega_{i,k}^{*(m)}\left|g_{i,k}^{*(m)}\right|^{2},\bar{t}_{i,k}\triangleq\frac{1}{M}\sum_{m=1}^{M}t_{i,k}^{(m)},$
(20a) $\displaystyle\bm{\Psi}_{i,k}^{(m)}$
$\displaystyle=t_{i,k}^{(m)}\mathbf{h}_{k}^{(m)}\mathbf{h}_{k}^{H(m)},\bar{\bm{\Psi}}_{i,k}\triangleq\frac{1}{M}\sum_{m=1}^{M}\bm{\Psi}_{i,k}^{(m)},$
(20b) $\displaystyle\bm{\theta}_{i,k}^{(m)}$
$\displaystyle=\omega_{i,k}^{*(m)}g_{i,k}^{*(m)}\mathbf{h}_{k}^{H(m)},\bar{\bm{\theta}}_{i,k}\triangleq\frac{1}{M}\sum_{m=1}^{M}\bm{\theta}_{i,k}^{(m)},$
(20c) $\displaystyle v_{i,k}^{(m)}$
$\displaystyle=\omega_{i,k}^{*(m)}-\log_{2}\left(\omega_{i,k}^{*(m)}\right),\bar{v}_{i,k}\triangleq\frac{1}{M}\sum_{m=1}^{M}v_{i,k}^{(m)}.$
(20d)

By applying Eq. (20) into problem (P3), a more intuitive expression upon
optimizing $\mathbf{c}$, $\mathbf{p}$ and $\mathbf{F}$ can be formulated as

$\displaystyle(\text{P4}):\min_{\mathbf{p},\mathbf{F},\mathbf{c},\mathbf{G},\mathbf{\Omega}}$
$\displaystyle\sum\limits_{k\in\mathcal{K}}u_{k}(\Lambda_{p,k}+\bar{t}_{p,k}\sigma_{k}^{2}-2\mu_{p,k}$
$\displaystyle+\bar{v}_{p,k}-C_{k})$ (21a) s.t. $\displaystyle
p_{c}\mathbf{f}_{c}^{H}\bar{\bm{\Psi}}_{c,k}\mathbf{f}_{c}+\Lambda_{c,k}+\bar{t}_{c,k}\sigma_{k}^{2}-2\mu_{c,k}$
$\displaystyle+\bar{v}_{c,k}+\sum\limits_{k\in\mathcal{K}}C_{k}\leq 1,\forall
k\in\mathcal{K},$ (21b)
$\displaystyle\Lambda_{p,k}+\bar{t}_{p,k}\sigma_{k}^{2}-2\mu_{p,k}$
$\displaystyle+\bar{v}_{p,k}-C_{k}\leq 1-R_{k}^{th},\forall k\in\mathcal{K},$
(21c) $\displaystyle(\rm{7d}),(\rm{7e}),(\rm{7f}),$ (21d)

where
$\Lambda_{i,k}=\sum_{k^{\prime}\in\mathcal{K}}p_{k^{\prime}}\mathbf{f}_{k^{\prime}}^{H}\bar{\bm{\Psi}}_{i,k}\mathbf{f}_{k^{\prime}}$,
$\mu_{c,k}=\sqrt{p_{c}}\Re\\{\bar{\bm{\theta}}_{c,k}\mathbf{f}_{c}\\}$ and
$\mu_{p,k}=\sqrt{p_{k}}\Re\\{\bar{\bm{\theta}}_{p,k}\mathbf{f}_{k}\\}$.

The problem (P4) is convex for $\mathbf{p}$ and $\mathbf{F}$ and linear for
$\mathbf{c}$, which can be alternately solved with BCD algorithm. The details
of BCD algorithm are summarized in Algorithm 1.

Algorithm 1 SAA and WMMSE based BCD algorithm

1:the convergence threshold $\epsilon$, the QoS threshold $R_{k}^{th}$ and the
power constraint $P_{t}$.

2:the optimal power allocation $\mathbf{p}^{*}$, common rate allocation
$\mathbf{c}^{*}$ and RIS transmissive coefficients $\mathbf{F}^{*}$

3:Initialize ($\mathbf{p}^{[0]},\mathbf{F}^{[0]},\mathbf{c}^{[0]}$) and
calculate $\mathrm{WASR}^{[0]}$, $n\leftarrow 0$.

4:repeat:

5: $\mathbf{G}^{[n]}$ and $\bm{\Omega}^{[n]}$ can be obtained by Eq. (16).

6: Update $\bar{t}_{i,k}^{[n]}$, $\bar{\bm{\Psi}}_{i,k}^{[n]}$,
$\bar{\bm{\theta}}_{i,k}^{[n]}$, $\bar{v}_{i,k}^{[n]}$ based on Eq. (20).

7: Solve the problem (P4) alternately based on the parameters obtained above,
specifically:

8: P4.1: $\mathbf{p}^{[n+1]}$ is obtained by fixing $\mathbf{c}^{[n]}$,
$\mathbf{F}^{[n]}$.

9: P4.2: $\mathbf{F}^{[n+1]}$ is obtained by fixing $\mathbf{c}^{[n]}$,
$\mathbf{p}^{[n+1]}$.

10: P4.3: $\mathbf{c}^{[n+1]}$ is obtained by fixing $\mathbf{p}^{[n+1]}$,
$\mathbf{F}^{[n+1]}$.

11: Update iteration $n\leftarrow n+1$.

12:until $|\mathrm{WASR}^{[n+1]}-\mathrm{WASR}^{[n]}|<\epsilon$

13:return $\mathbf{p}^{*}=\mathbf{p}^{[n+1]}$,
$\mathbf{F}^{*}=\mathbf{F}^{[n+1]}$, $\mathbf{c}^{*}=\mathbf{c}^{[n+1]}$

In each iteration of Algorithm 1, it takes two steps to solve the problem: 1)
the original non-convex problem (P1) is transformed into the convex problem
(P4) by SAA and WMMSE operations, which has the complexity of
$\mathcal{O}(K^{2}M)$. 2) the convex problem (P4) is solved by the interior
point method, which has the complexity of $\mathcal{O}(K^{3.5})$. Hence, the
complexity of Algorithm 1 in each iteration is
$\mathcal{O}\Big{(}\text{max}(K^{2}M,K^{3.5})\Big{)}$. Since $M$ is determined
and considered as a fixed value, the total complexity of Algorithm 1 is
$\mathcal{O}\Big{(}\log(\epsilon^{-1})K^{3.5}\Big{)}$.

Suppose ${\mathbf{p}^{(r)}},{\mathbf{F}^{(r)}},{\mathbf{c}^{(r)}}$ as the r-th
iteration solution of the Algorithm 1. The objective function is denoted by
$\mathcal{P}\left(\mathbf{p}^{(r)},\mathbf{F}^{(r)},\mathbf{c}^{(r)}\right)$.
Follow the step 6,7 and 8 of the algorithm, we can get

$\displaystyle\mathcal{P}\left(\mathbf{p}^{(r)},\mathbf{F}^{(r)},\mathbf{c}^{(r)}\right)\geq\mathcal{P}\left(\mathbf{p}^{(r+1)},\mathbf{F}^{(r)},\mathbf{c}^{(r)}\right).$
(22a)
$\displaystyle\mathcal{P}\left(\mathbf{p}^{(r+1)},\mathbf{F}^{(r)},\mathbf{c}^{(r)}\right)\geq\mathcal{P}\left(\mathbf{p}^{(r+1)},\mathbf{F}^{(r+1)},\mathbf{c}^{(r)}\right).$
(22b)
$\displaystyle\mathcal{P}\left(\mathbf{p}^{(r+1)},\mathbf{F}^{(r+1)},\mathbf{c}^{(r)}\right)\geq\mathcal{P}\left(\mathbf{p}^{(r+1)},\mathbf{F}^{(r+1)},\mathbf{c}^{(r+1)}\right).$
(22c)

Based on above, we can get

$\mathcal{P}\left(\mathbf{p}^{(r)},\mathbf{F}^{(r)},\mathbf{c}^{(r)}\right)\geq\mathcal{P}\left(\mathbf{p}^{(r+1)},\mathbf{F}^{(r+1)},\mathbf{c}^{(r+1)}\right),$
(23)

which ensure the convergence of Algorithm 1 due to the fact that the objective
function has a finite lower bound.

##  IV Numercial Results

Figure 2: The convergence of the proposed RSMA-based algorithm compared with
SDMA and NOMA

Figure 3: The realtionship bewteen the maximum transmit power and WASR in
three architectures

Figure 4: The realtionship bewteen the number of users and WASR in three
architectures

Figure 5: The WASR of the proposed RSMA-based algorithms for different number
of RIS sub-arrays

In this section, we validate the effectiveness of the proposed RSMA-based
optimization algorithm by simulations with CVX. In the simulations,
$\sigma^{2}_{k}=1$. The number of channel samples is set to $M=500$. Each sub-
array includes $N_{e}=32$ elements and the total elements are 256. The channel
follows a Rayleigh distribution, where the signal attenuation at a reference
distance of 1 m and the pathloss exponents are set as 30 dB and 3. The
distance bewteen the transmissive RIS transmitter and users follows a uniform
distribution from 1 to 100m. The QoS threshold $R_{k}^{th}$ is set to 0.1
bps/Hz for each user. Each user has the same priority in communication, which
means $u_{k}=1$ for all users.

In Fig. 5, the number of RIS sub-arrays is set to $N=8$ and the number of
users is set to $K=15$. We first evaluate the convergence of the proposed
RSMA-based algorithm compared with conventional SDMA and NOMA architecture.
The initialization of each user’s power in RSMA and SDMA architectures follows
the uniform distribution at the maxium transmit power $P_{t}=10$dB, while in
NOMA it depends on the channel strength of each user [12]. That’s why the
performance of NOMA is the best in the early iterations. As the iteration
continues, the performance of NOMA suffers due to the uncertain channel and
performs worse than RSMA when coming to the convergence. It demonstrates that
our proposed RSMA-based algorithm performs better than NOMA and SDMA when the
channel acquisition is inaccurate.

In Fig. 5, we set $N=8$ and $K=15$. As a complement, we analyze how the maxium
transmit power, which is the most important factor constrainting WASR, affects
the WASR. Although the WASR grows approximately linearly with the maximum
transmit power as we expected, it may come to the peak constrainted by the
real scenarios.

In Fig. 5, we set $N=8$ and $P_{t}=10$dB. We study the realtionship bewteen
the number of users and WASR in three architectures. At the initial stage, the
actual transmit power is lower than the maximum transmit power, so the WASR
grows like Fig. 5. When the transmit power reaches the peak, the WASR won’t
continue to increase but decrease due to the increasing interference levels.
It proves that our proposed algorithm has better interference immunity
compared to others.

In Fig. 5, we set $K=15$ and $P_{t}=10$dB. Fig. 5 compares the WASR under
different number of RIS sub-arrays. It illustrates that the increase of RIS
sub-arrays significantly improves the WASR. As mentioned above, the
transmissive RIS transmitter is equivalent to a multi-antenna system, where
the system capacity and the communication performance can be improved with the
increase of sub-arrays.

##  V Conclusion

In this paper, we propose a rate-splitting multi-user access architecture with
the transmissive RIS transmitter where the CSI bewteen the users and the
transmitter is estimated inaccurately. A weighted sum-rate maximization
problem on optimizing power allocation, common rate allocation and RIS
transmissive coefficients based on RSMA architecture is mainly investivated.
To deal with the non-convexity of the problem caused by the coupling of
optimization variables, the SAA and WMMSE methods are utlized to transformed
into a convex problem and a BCD algorithm is used for the solution to the
problem. Numerical results validate that our proposed algorithm performs
better compared with SDMA and NOMA architecture, and the transmissive RIS
transmitter has more advantages than the conventional multi-antenna system.

## References

  * [1] Y. Mao, O. Dizdar, B. Clerckx, R. Schober, P. Popovski, and H. V. Poor, “Rate-splitting multiple access: Fundamentals, survey, and future research trends,” IEEE Commun. Surveys Tuts., vol. 24, no. 4, pp. 2073–2126, Jul. 2022.
  * [2] A. Mishra, Y. Mao, O. Dizdar, and B. Clerckx, “Rate-splitting multiple access for 6G-Part I: Principles, applications and future works,” IEEE Commun. Lett., vol. 26, no. 10, pp. 2232–2236, Oct. 2022.
  * [3] H. Xia, Y. Mao, B. Clerckx, X. Zhou, S. Han, and C. Li, “Weighted sum-rate maximization for rate-splitting multiple access based secure communication,” in 2022 IEEE Wireless Commun. Netw. Conf. (WCNC), pp. 19–24, Apr. 2022.
  * [4] Z. Li, W. Chen, and H. Cao, “Beamforming design and power allocation for transmissive RMS-based transmitter architectures,” IEEE Wireless Commun. Lett., vol. 11, no. 1, pp. 53–57, Jan. 2022.
  * [5] Z. Li, W. Chen, Q. Wu, H. Cao, K. Wang, and J. Li, “Robust beamforming design and time allocation for IRS-assisted wireless powered communication networks,” IEEE Trans. Commun., vol. 70, no. 4, pp. 2838–2852, Apr. 2022\.
  * [6] G. Chen, Q. Wu, W. Chen, D. W. K. Ng, and L. Hanzo, “IRS-aided wireless powered MEC systems: TDMA or NOMA for computation offloading?,” IEEE Trans.Wireless Commun., vol. 22, no. 2, pp. 1201–1218, 2023.
  * [7] S. Zeng, H. Zhang, B. Di, Y. Tan, Z. Han, H. V. Poor, and L. Song, “Reconfigurable intelligent surfaces in 6G: Reflective, transmissive, or both?,” IEEE Commun. Lett., vol. 25, no. 6, pp. 2063–2067, June. 2021\.
  * [8] Z. Li, W. Chen, Z. Zhang, Q. Wu, H. Cao, and J. Li, “Robust sum-rate maximization in transmissive RMS transceiver-enabled SWIPT networks,” IEEE Internet Things J., vol. 10, no. 8, pp. 7259–7271, Dec. 2022.
  * [9] V. Kumar, R. Zhang, M. D. Renzo, and L.-N. Tran, “A novel SCA-based method for beamforming optimization in IRS/RIS-assisted MU-MISO downlink,” IEEE Wireless Commun. Lett., vol. 12, no. 2, pp. 297–301, Feb. 2023.
  * [10] X. Bai, F. Kong, Y. Sun, G. Wang, J. Qian, X. Li, A. Cao, C. He, X. Liang, R. Jin, and W. Zhu, “High-efficiency transmissive programmable metasurface for multimode OAM generation,” Adv. Opt. Mater., vol. 8, no. 17, p. 2000570, Jun. 2020.
  * [11] H. Joudeh and B. Clerckx, “Sum-rate maximization for linearly precoded downlink multiuser MISO systems with partial CSIT: A rate-splitting approach,” IEEE Trans.Commun., vol. 64, no. 11, pp. 4847–4861, Nov. 2016\.
  * [12] Z. Li, W. Chen, Q. Wu, K. Wang, and J. Li, “Joint beamforming design and power splitting optimization in IRS-assisted SWIPT NOMA networks,” IEEE Trans. Wireless Commun., vol. 21, no. 3, pp. 2019–2033, Mar. 2022.


As an important framework for safe Reinforcement Learning, the Constrained Markov Decision Process (CMDP) has been extensively studied in the recent literature. However, despite the rich results under various on-policy learning settings, there still lacks some essential understanding of the offline CMDP problems, in terms of both the algorithm design and the information theoretic sample complexity lower bound. In this paper, we focus on solving the CMDP problems where only offline data are available. By adopting the concept of the single-policy concentrability coefficient $C^*$, we establish an $\Omega\left(\frac{\min\left\{|\mathcal{S}||\mathcal{A}|,|\mathcal{S}|+I\right\} C^*}{(1-\gamma)^3\epsilon^2}\right)$ sample complexity lower bound for the offline CMDP problem, where $I$ stands for the number of constraints. By introducing a simple but novel deviation control mechanism, we propose a near-optimal primal-dual learning algorithm called DPDL. This algorithm provably guarantees zero constraint violation and its sample complexity matches the above lower bound except for an $\tilde{\mathcal{O}}((1-\gamma)^{-1})$ factor. Comprehensive discussion on how to deal with the unknown constant $C^*$ and the potential asynchronous structure on the offline dataset are also included.

§ INTRODUCTION

Reinforcement Learning (RL) is an important tool for modeling the real world tasks that involve sequential decision making. Such RL problems are often mathematically described as a Markov Decision Process (MDP) that maximizes a cumulative sum of rewards. The safe reinforcement learning, on the other hand, not only cares the reward maximization, but also attempts to ensure a reasonable system performance with respect to certain safety constraints. Such safety constrained RL problems are often formulated as  the Constrained Markov Decision Process (CMDP) $\cM=(\cS, \cA, \PP, r, u, \gamma, \rho_0)$, where $\cS$ is a finite state space, $\cA$ is a finite action space, $\gamma \in(0,1)$ is the discount factor, $\PP\left(s^{\prime} \mid s, a\right)$ stands for the transition probability from $s$ to $s^{\prime}$ under the action $a$ for $\forall(s,a,s^{\prime})\in\cS\times\cA\times\cS$, and $r:\cS \times \cA \rightarrow[-1,1]$ is the reward function, $(u_i: \cS \times \cA \rightarrow[-1,1])_{i\in[I]}$ is a set of $I$ utility functions,  $\rho_0$ is the initial state distribution over $\cS$. The goal of CMDP is to find an optimal policy $\pi$ to maximize the cumulative reward while satisfying a group of constraints:

\begin{eqnarray}
\label{eqn:srl-obj-demo}
J(\pi)\, :=\mathbb{E}\bigg[\sum_{t=0}^{+\infty} \gamma^{t}\cdot r\left(s_{t}, a_{t}\right)\,\Big|\,s_0\sim\rho_0,\pi\bigg]\\
&\st & \!\!J^u_i(\pi)
:=\mathbb{E} \bigg[\sum_{t=0}^{+\infty} \gamma^{t}\cdot u_i\left(s_{t}, a_{t}\right)\bigg]\geq 0, \mbox{ for } i \in [I]=\left\{ 1,2,...,I\right\}.\nonumber
\end{eqnarray}

For the CMDP problem, there has been plenty of on-policy algorithms, see <cit.>. However, in real world applications such as training physical robots, where safety is an important measure of performance, the real time on-policy interaction with the environment may suffer from the potential damages to the robots.
Besides, in many non-simulating environments, the on-policy data collection may also be time-consuming. Therefore, it is crucial to design an off-policy algorithm to solve the CDMP problems, where plenty of historical data are already accumulated while real time interactions are limited. To our best knowledge, offline CMDP algorithms are rare [Le et al., 2019, Xu et al., 2021, Wu et al., 2021],  and the sample complexity guarantees are limited.
In particular, a strong uniform concentrability assumption is required in [Le et al., 2019], and the model-based method [Wu et al., 2021] mainly considers the case an empirical model is known.
Thus it is still not clear how to efficiently solve offline CMDPs with model-free approaches, and there lacks essential understanding of the information theoretic lower bound on the sample complexity of the offline CMDP.

In this paper, we propose a Deviation-controlled Primal-Dual Learning (DPDL) method to solve problem (<ref>). We adopt the primal-dual strategy developed in  <cit.> as the main algorithmic framework while several non-trivial contributions have been made beyond the existing results. Unlike the aforementioned literatures that exclusively rely on the accessibility of a generative model,  utilizes the offline data, where the distribution shift difficulties of the offline data is tackled by a novel and effective adaptive deviation control mechanism. If the considered CMDP instance has a finite (but potentially unknown) concentrability coefficient,  provably finds a policy with $\mathcal{O}(\epsilon)$-optimal reward and zero constraint violation. An information theoretical lower bound on the sample complexity of offline CMDP is also derived in this paper, which indicates
that our deviation control mechanism achieves a minimax optimal complexity dependence on $I,\nS,\nA,C^*$.

Main Contribution. We summarize the contributions in details as follows.

* We propose the  algorithm to solve the CMDP problem (<ref>). Suppose the CMDP instance satisfies the Slater's condition and certain prior knowledge on the concentrability coefficient $C^*$ is given,  provably finds an $\epsilon$-optimal policy with zero constraint violation using $\tbO{\frac{\nval C^*}{(1-\gamma)^4\epsilon^2}}$ offline samples.

* We establish an information theoretic sample complexity lower bound of $\Om{\frac{\nval C^*}{(1-\gamma)^3\epsilon^2}}$ for the offline CMDPs, indicating that DPDL is near optimal up to an $\tilde{\mathcal{O}}((1-\gamma)^{-1})$ factor. The necessity of the Slater's condition for achieving zero constraint violation is also established.

* In order to handle the practical situation where $C^*$ is unknown, an adaptive version of DPDL is designed with the same sample complexity as DPDL.

* Our analysis of  also extends to the asynchronous case, where the offline dataset consists of a sample trajectory generated by certain behavior policy. In this situation, the sample complexity of DPDL is shown to be $\tbO{\frac{t_{\mix}^2\nval C^*}{(1-\gamma)^4\epsilon^2}}$.

Related Work.
Recently, considerable efforts have been devoted to the online learning of CMDP.  Under the episodic and tabular setting, several works [Ding et al., 2021, Efroni et al., 2020, Qiu et al., 2020] have achieved the $\tbbO{\sqrt{\nS^2\nA T}}$ regret and cumulative constraint violation, with different dependence on the episode length $H$ omitted. Under proper assumptions, zero or bounded cumulative constraint violation can be achieved [Agarwal et al., 2021, Liu et al., 2021]. In terms of the number of constraints $I$, MOMA proposed in [Yu et al., 2021] achieves an $\tbbO{\!\sqrt{\min\{\nS,\!I\}I\nS\nA/T}}$ convergence on both average reward gap and constraint violation. Nevertheless, all the above results adopt the model-based approaches. Except for [Yu et al., 2021], they either consider the cases where $I\!=\!1$ or completely ignore the influence of $I$ in the sample complexity. Therefore, both deriving an efficient model-free method and obtaining the optimal dependence on $I$ remain open.

Another approach closely related to our paper is the primal-dual method in RL, see  <cit.>. Given the access to a generative model, the model-free primal-dual method developed in [Bai et al., 2021] achieves an $\tilde{\mathcal{O}}\big(\frac{I|\cS||\cA|}{(1-\gamma)^4\epsilon^2}\big)$ sample complexity to find an $\epsilon$-optimal safe policy.
The deviation control mechanism we develop enables the primal-dual approach to extend beyond the generative model.

Finally, we mention a few related works in the offline RL and safe RL. Previous offline RL algorithms with sample efficiency guarantees typically assume the uniform concentrability <cit.> or lower bounded minimum visitation $\mu_{\min}$ <cit.>. Recently, under the less restrictive assumption of the single-policy concentrability coefficient $C^*$, a minimax optimal sample complexity lower bound of $\Omega\big(\frac{\nS C^*}{(1-\gamma)^3\epsilon^2}\big)$ for discounted offline MDPs is derived in [Rashidinejad et al., 2021]. A similar $\Omega\big(\frac{H^3\nS C^*}{\epsilon^2}\big)$ lower bound is also derived for the episodic setting in [Xie et al., 2021].
Under both settings, offline algorithms with $\tilde{\mathcal{O}}(\nS C^*\epsilon^{-2})$ sample complexity (with  different $(1-\gamma)^{-1}$ or $H$ factors omitted) have been discovered with either model-based [Rashidinejad et al., 2021, Xie et al., 2021, Yin and Wang, 2021, Li et al., 2022] or model-free approaches [Shi et al., 2022, Yan et al., 2022].  In terms of the offline CMDP problem, the only existing results are [Le et al., 2019, Xu et al., 2021, Wu et al., 2021], where [Xu et al., 2021] only provides asymptotic convergence, [Le et al., 2019] relies on a much stronger uniform concentrability assumption, and [Wu et al., 2021] is a model based method that potentially suffers an $\mathcal{O}((C^*)^2)$ dependence.
Compared to these works, our method is model-free and has an optimal $\mathcal{O}(C^*)$ dependence on the concentrability coefficient.

[Agarwal et al., 2021]
Mridul Agarwal, Qinbo Bai, and Vaneet Aggarwal.
Concave utility reinforcement learning with zero-constraint
arXiv preprint arXiv:2109.05439, 2021.

[Altman, 1995]
Eitan Altman.
Constrained Markov decision processes.
PhD thesis, INRIA, 1995.

[Assouad, 1996]
Bin Yu Assouad.
Fano, and le cam.
Festschrift for Lucien Le Cam, pages 423–435, 1996.

[Bai et al., 2021]
Qinbo Bai, Amrit Singh Bedi, Mridul Agarwal, Alec Koppel, and Vaneet Aggarwal.
Achieving zero constraint violation for constrained reinforcement
learning via primal-dual approach.
arXiv preprint arXiv:2109.06332, 2021.

[Bertsimas and Tsitsiklis, 1997]
Dimitris Bertsimas and John N Tsitsiklis.
Introduction to linear optimization, volume 6.
Athena Scientific Belmont, MA, 1997.

[Chen and Teboulle, 1993]
Gong Chen and Marc Teboulle.
Convergence analysis of a proximal-like minimization algorithm using
Bregman functions.
SIAM Journal on Optimization, 30 (3):0
538–543, 1993.

[Ding et al., 2021]
Dongsheng Ding, Xiaohan Wei, Zhuoran Yang, Zhaoran Wang, and Mihailo Jovanovic.
Provably efficient safe exploration via primal-dual policy
In International Conference on Artificial Intelligence and
Statistics, pages 3304–3312. PMLR, 2021.

[Efroni et al., 2020]
Yonathan Efroni, Shie Mannor, and Matteo Pirotta.
Exploration-exploitation in constrained mdps.
arXiv preprint arXiv:2003.02189, 2020.

[Gilbert, 1952]
Edgar N Gilbert.
A comparison of signalling alphabets.
The Bell system technical journal, 310 (3):0
504–522, 1952.

[Jiang et al., 2018]
Bai Jiang, Qiang Sun, and Jianqing Fan.
Bernstein's inequality for general markov chains.
arXiv preprint arXiv:1805.10721, 2018.

[Kamoutsi et al., 2021]
Angeliki Kamoutsi, Goran Banjac, and John Lygeros.
Efficient performance bounds for primal-dual reinforcement learning
from demonstrations.
In International Conference on Machine Learning, pages
5257–5268. PMLR, 2021.

[Le et al., 2019]
Hoang Le, Cameron Voloshin, and Yisong Yue.
Batch policy learning under constraints.
In International Conference on Machine Learning, pages
3703–3712. PMLR, 2019.

[Levin and Peres, 2017]
David A Levin and Yuval Peres.
Markov chains and mixing times, volume 107.
American Mathematical Soc., 2017.

[Li et al., 2020]
Gen Li, Yuting Wei, Yuejie Chi, Yuantao Gu, and Yuxin Chen.
Sample complexity of asynchronous Q-learning: Sharper analysis and
variance reduction.
Advances in neural information processing systems,
33:0 7031–7043, 2020.

[Li et al., 2022]
Gen Li, Laixi Shi, Yuxin Chen, Yuejie Chi, and Yuting Wei.
Settling the sample complexity of model-based offline reinforcement
arXiv preprint arXiv:2204.05275, 2022.

[Li et al., 2021]
Yongfeng Li, Mingming Zhao, Weijie Chen, and Zaiwen Wen.
A stochastic composite augmented Lagrangian method for
reinforcement learning.
arXiv preprint arXiv:2105.09716, 2021.

[Liu et al., 2021]
Tao Liu, Ruida Zhou, Dileep Kalathil, Panganamala Kumar, and Chao Tian.
Learning policies with zero or bounded constraint violation for
constrained mdps.
Advances in Neural Information Processing Systems, 34, 2021.

[Munos and Szepesvári, 2008]
Rémi Munos and Csaba Szepesvári.
Finite-time bounds for fitted value iteration.
Journal of Machine Learning Research, 90 (5), 2008.

[Paulin, 2015]
Daniel Paulin.
Concentration inequalities for markov chains by marton couplings and
spectral methods.
Electronic Journal of Probability, 20:0 1–32, 2015.

[Qiu et al., 2020]
Shuang Qiu, Xiaohan Wei, Zhuoran Yang, Jieping Ye, and Zhaoran Wang.
Upper confidence primal-dual reinforcement learning for cmdp with
adversarial loss.
Advances in Neural Information Processing Systems,
33:0 15277–15287, 2020.

[Rashidinejad et al., 2021]
Paria Rashidinejad, Banghua Zhu, Cong Ma, Jiantao Jiao, and Stuart Russell.
Bridging offline reinforcement learning and imitation learning: A
tale of pessimism.
Advances in Neural Information Processing Systems, 34, 2021.

[Shi et al., 2022]
Laixi Shi, Gen Li, Yuting Wei, Yuxin Chen, and Yuejie Chi.
Pessimistic Q-learning for offline reinforcement learning: Towards
optimal sample complexity.
arXiv preprint arXiv:2202.13890, 2022.

[Tropp, 2011]
Joel Tropp.
Freedman's inequality for matrix martingales.
Electronic Communications in Probability, 16:0
262–270, 2011.

[Varshamov, 1957]
Rom Rubenovich Varshamov.
Estimate of the number of signals in error correcting codes.
Docklady Akad. Nauk, SSSR, 117:0 739–741, 1957.

[Wang, 2017]
Mengdi Wang.
Primal-dual $\pi$ learning: Sample complexity and sublinear run time
for ergodic markov decision problems.
arXiv preprint arXiv:1710.06100, 2017.

[Wang, 2020]
Mengdi Wang.
Randomized linear programming solves the markov decision problem in
nearly linear (sometimes sublinear) time.
Mathematics of Operations Research, 450 (2):0
517–546, 2020.

[Wu et al., 2021]
Runzhe Wu, Yufeng Zhang, Zhuoran Yang, and Zhaoran Wang.
Offline constrained multi-objective reinforcement learning via
pessimistic dual value iteration.
Advances in Neural Information Processing Systems, 34, 2021.

[Xie et al., 2021]
Tengyang Xie, Nan Jiang, Huan Wang, Caiming Xiong, and Yu Bai.
Policy finetuning: Bridging sample-efficient offline and online
reinforcement learning.
Advances in neural information processing systems, 34, 2021.

[Xu et al., 2021]
Haoran Xu, Xianyuan Zhan, and Xiangyu Zhu.
Constraints penalized Q-learning for safe offline reinforcement
arXiv preprint arXiv:2107.09003, 2021.

[Yan et al., 2022]
Yuling Yan, Gen Li, Yuxin Chen, and Jianqing Fan.
The efficacy of pessimism in asynchronous Q-learning.
arXiv preprint arXiv:2203.07368, 2022.

[Yin and Wang, 2021]
Ming Yin and Yu-Xiang Wang.
Towards instance-optimal offline reinforcement learning with
Advances in neural information processing systems, 34, 2021.

[Yin et al., 2020]
Ming Yin, Yu Bai, and Yu-Xiang Wang.
Near-optimal provable uniform convergence in offline policy
evaluation for reinforcement learning.
arXiv preprint arXiv:2007.03760, 2020.

[Yin et al., 2021]
Ming Yin, Yu Bai, and Yu-Xiang Wang.
Near-optimal offline reinforcement learning via double variance
Advances in neural information processing systems, 34, 2021.

[Yu et al., 2021]
Tiancheng Yu, Yi Tian, Jingzhao Zhang, and Suvrit Sra.
Provably efficient algorithms for multi-objective competitive rl.
In International Conference on Machine Learning, pages
12167–12176. PMLR, 2021.

[Zhang et al., 2021]
Junyu Zhang, Amrit Singh Bedi, Mengdi Wang, and Alec Koppel.
Cautious reinforcement learning via distributional risk in the dual
IEEE Journal on Selected Areas in Information Theory,
20 (2):0 611–626, 2021.