# Almost symmetric good semigroups

L. Casabella Dipartimento di Matematica ”Tullio Levi-Civita”, Università degli
Studi di Padova, Italy; email<EMAIL_ADDRESS>M. D’Anna
Dipartimento di Matematica e Informatica, Università degli Studi di Catania,
Italy; email<EMAIL_ADDRESS>- ORCID: 0000-0002-9943-5527

###### Abstract

The class of good semigroups is a class of subsemigroups of
$\displaystyle\mathbb{N}^{h}$, that includes the value semigroups of rings
associated to curve singularities and their blowups, and allows to study
combinatorically the properties of these rings. In this paper we give a
characterization of almost symmetric good subsemigroups of
$\displaystyle\mathbb{N}^{h}$, extending known results in numerical semigroup
theory and in one-dimensional ring theory, and we apply these results to
obtain new results on almost Gorenstein one-dimensional analytically
unramified rings.

Keywords: good semigroup, almost-symmetry, curve singularity, almost
Gorenstein rings, canonical ideal.

Mathematics Subject Classification: 13A15, 13H10, 14H99, 20M12, 20M25

Acknowledgments. The second author was partially funded by the project
“Proprietà locali e globali di anelli e di varietà algebriche”-PIACERI
2020–22, Università degli Studi di Catania.

## Introduction

Let $\displaystyle\mathbb{N}$ be the set of nonnegative integers; _good
semigroups_ are a class of subsemigroups of $\displaystyle\mathbb{N}^{h}$,
with $\displaystyle h\in\mathbb{N}$, $\displaystyle h\geq 1$ (see the formal
definition in Section 1). Their definition was given in [4] in order to
describe value semigroups of noetherian, analytically unramified, one-
dimensional, semilocal reduced rings, even though the properties of these
value semigroups were already studied in [5], [7], [8], [9], [15], [16], [17].
The class of rings mentioned above includes the local rings arising from curve
singularities (and from their blowups), possibly with more than one branch.
However, as shown in [4], there are good semigroups that are not value
semigroups of any ring.

If $\displaystyle S$ is a value subsemigroup of $\displaystyle\mathbb{N}^{h}$,
the integer $\displaystyle h$ represents the number of branches of the
singularity. In the one branch case, the value semigroup is a numerical
semigroup, and the algebraic properties of the ring can be translated and
studied at semigroup level. Some of the results in this direction have been
proved to hold for $\displaystyle h>1$ as well. For instance, the numerical
characterization of a one-dimensional analytically irreducible local
Gorenstein domain via symmetric numerical semigroups (see [22]) can be
generalized to analytically unramified local rings (see [16] and also [8]);
analogously, the numerical characterization of the canonical module in the
analytically irreducible case (see [20]) can be given also in the more general
case (see [9]).

Dealing with birational extensions of an analytically unramified ring
$\displaystyle R$, one has to work with both local and semilocal rings; this
fact is well represented at semigroup level, since it is possible to give a
satisfying definition of local good semigroup that, for value semigroups,
corresponds to the local ring case; with this definition one can show that any
good semigroup is a direct product of local good semigroups that, when
$\displaystyle S=v(R)$, correspond to the value semigroups of the
localizations of $\displaystyle R$ at its maximal ideals (see [4]).

These facts motivated the interest in studying good semigroups, and, since not
every good semigroup is a value semigroup of some ring, it is often necessary
to work on good semigroups in a purely combinatorial setting. Despite their
name, good semigroups present some problems that make their study difficult;
e.g. they are not finitely generated as monoids (but, on the other hand, they
can be completely determined by a finite set of elements; see [17] and [10])
and they are not closed under finite intersections. Moreover, if we consider
good ideals of good semigroups (e.g. the ideals arising as values of ideals of
the corresponding ring), this class is not closed under sums and differences
(see, for instance, [4] and [21]). Therefore, unlike what happens for
numerical semigroups (in analogy to analytically irreducible domains), it is
not easy to define for good semigroups some concepts that translates analogous
ring concepts, like the embedding dimension and the type (see [12] and [23]).
Finally, when working with good semigroups, technical problems often arise if
$\displaystyle h\geq 3$ and, for this reason, several papers on the subject
are limited to the very special case $\displaystyle h=2$ (see e.g. [17], [5],
[11] and [12]).

Almost Gorenstein analytically unramified rings and almost symmetric numerical
semigroups were introduced in [6] in order to find two classes of rings and
numerical semigroups, respectively, that are close to Gorenstein rings and to
symmetric numerical semigroups, in certain aspects. The notion of almost
Gorenstein ring was generalized for any one-dimensional ring in [18] and,
subsequently, for rings in any dimension admitting a canonical ideal (see
[19]). Since then, almost Gorenstein rings have been intensively studied,
being a useful intermediate class between Cohen-Macaulay and Gorenstein rings.
Similarly, the notion of almost symmetric numerical semigroups was generalized
to good semigroups in [4].

A remarkable property shown in [6] for a one-dimensional analytically
unramified local ring $\displaystyle(R,\mathfrak{m})$ is that such a ring is
almost Gorenstein of maximal embedding dimension if and only if
$\displaystyle\mathfrak{m}:\mathfrak{m}$ is Gorenstein, and an analogous
result was proved in the same paper for numerical semigroups. These two
results were the starting point for other similar investigations. In
particular, in [18] the authors generalized the first one for any one-
dimensional Cohen-Macaulay local ring admitting a canonical ideal and, in
[14], the following more general result is proved:

Proposition _[14, Proposition 3.2]. Let $\displaystyle(R,\mathfrak{m})$ be a
one-dimensional Cohen-Macaulay local ring admitting a canonical ideal. Then
$\displaystyle R$ is a one-dimensional almost Gorenstein ring if and only if
$\displaystyle\mathfrak{m}$ is a canonical ideal of
$\displaystyle\mathfrak{m}:\mathfrak{m}$._

Note that an analogous result was stated in a non-noetherian setting in [2],
but with an additional hypothesis. As for numerical semigroups, the
corresponding result of the above proposition was proved in [3].

If we consider good semigroups, fewer results of this kind are known. More
precisely, a partial result was proved in [12] for the case $\displaystyle
h=2$, assuming that $\displaystyle S$ is a local good semigroup of maximal
embedding dimension. In this context, the main goal of this paper is to prove
the following result:

Theorem 2.4. _Let $\displaystyle h\geq 2$, let $\displaystyle
S\subseteq\mathbb{N}^{h}$ be a local almost symmetric good semigroup and let
$\displaystyle M=S\setminus\\{\boldsymbol{0}\\}$ be its maximal ideal. Then
$\displaystyle S$ is almost symmetric if and only if $\displaystyle
M-M=\\{\boldsymbol{\alpha}\in\mathbb{N}^{h}\mid\boldsymbol{\alpha}+M\subseteq
M\\}$ is a good semigroup and $\displaystyle M$ is a canonical ideal of
$\displaystyle M-M$._

Hence, after recalling in Section 1 the basic notions necessary to develop our
arguments, in Section 2 we give a description of the canonical ideal of
$\displaystyle M-M$, assuming it is a good semigroup (Proposition 2.3), and we
use it to prove the main theorem, together with a characterization of local
almost symmetric good semigroups of maximal embedding dimension via
$\displaystyle M-M$ (Corollary 2.6). Afterwards, we generalize these results
to the non-local case and, in Section 3, we show how our results imply new
characterizations of almost Gorenstein analytically unramified one-dimensional
local or semilocal rings (see, e.g. Proposition 3.3).

##  1 Preliminaries on good semigroups

Let $\displaystyle\mathbb{N}$ be the set of nonnegative integers and let
$\displaystyle h\geq 2$ be an integer. As usual, $\displaystyle\leq$ stands
for the natural partial ordering on $\displaystyle\mathbb{N}^{h}$:
$\displaystyle\boldsymbol{\alpha}\leq\boldsymbol{\beta}$ if
$\displaystyle\alpha_{i}\leq\beta_{i}$ for every $\displaystyle i=1,\dots,h$.
Given $\displaystyle\boldsymbol{\alpha},\boldsymbol{\beta}\in\mathbb{N}^{h}$,
the infimum of the set
$\displaystyle\\{\boldsymbol{\alpha},\boldsymbol{\beta}\\}$ (with respect to
$\displaystyle\leq$) will be denoted by
$\displaystyle\boldsymbol{\alpha}\wedge\boldsymbol{\beta}$. Hence
$\displaystyle\boldsymbol{\alpha}\wedge\boldsymbol{\beta}=(\min(\alpha_{1},\beta_{1}),\dots,\min(\alpha_{h},\beta_{h}))$.

A submonoid $\displaystyle S\subseteq\mathbb{N}^{h}$ is said to be a _good
semigroup_ if the following three properties are satisfied:

  * (G1)

if $\displaystyle\boldsymbol{\alpha},\boldsymbol{\beta}\in S$, then
$\displaystyle\boldsymbol{\alpha}\wedge\boldsymbol{\beta}\in S$;

  * (G2)

if $\displaystyle\boldsymbol{\alpha},\boldsymbol{\beta}\in S$,
$\displaystyle\boldsymbol{\alpha}\neq\boldsymbol{\beta}$ and
$\displaystyle\alpha_{i}=\beta_{i}$ for some $\displaystyle
i\in\\{1,\dots,h\\}$, then there exists $\displaystyle\boldsymbol{\delta}\in
S$ such that $\displaystyle\delta_{i}>\alpha_{i}=\beta_{i}$ and
$\displaystyle\delta_{j}\geq\min(\alpha_{j},\beta_{j})$ for all $\displaystyle
j\neq i$ (and equality holds if $\displaystyle\alpha_{j}\neq\beta_{j}$);

  * (G3)

there exists $\displaystyle\boldsymbol{c}\in\mathbb{N}^{h}$ such that
$\displaystyle\boldsymbol{c}+\mathbb{N}^{h}\subseteq S$.

Notice that for $\displaystyle h=1$, properties (G1) and (G2) become
meaningless and property (G3), for a submonoid $\displaystyle S$ of
$\displaystyle\mathbb{N}$, means that $\displaystyle S$ is a numerical
semigroup. Hence we can consider numerical semigroups as the good semigroups
in the case $\displaystyle h=1$.

A _relative ideal_ of a good semigroup $\displaystyle
S\subseteq\mathbb{N}^{h}$ is a subset $\displaystyle E$ of
$\displaystyle\mathbb{Z}^{h}$ such that
$\displaystyle\boldsymbol{\alpha}+\boldsymbol{\beta}\in E$ for any
$\displaystyle\boldsymbol{\alpha}\in S$ and
$\displaystyle\boldsymbol{\beta}\in E$, and there is a
$\displaystyle\boldsymbol{\delta}\in S$, such that
$\displaystyle\boldsymbol{\delta}+E\subseteq S$. If $\displaystyle E\subseteq
S$ then we will simply say that $\displaystyle E$ is an _ideal_ of
$\displaystyle S$. A relative ideal $\displaystyle E$ satisfying (G1) and (G2)
is called _good ideal_. Note that a relative ideal necessarily satisfies (G3),
since it is closed up to sums with elements of $\displaystyle S$. Observe
that, given two relative ideals $\displaystyle E,F$ of $\displaystyle S$, the
following sets are relative ideals as well:

  * •

$\displaystyle
E+F:=\\{\boldsymbol{\alpha}+\boldsymbol{\beta}\mid\boldsymbol{\alpha}\in
E,\boldsymbol{\beta}\in F\\}$,

  * •

$\displaystyle
E-F:=\\{\boldsymbol{\alpha}\in\mathbb{Z}^{d}\mid\boldsymbol{\alpha}+F\subseteq
E\\}$.

However, if $\displaystyle E$ and $\displaystyle F$ are good, it is not true
in general that $\displaystyle E+F$ and $\displaystyle E-F$ are good, as shown
in [4, Example 2.10].

If the only element of $\displaystyle S$ with a zero component is
$\displaystyle\boldsymbol{0}=(0,0,\dots 0)$, we will say that $\displaystyle
S$ is _local_. In this case, the set $\displaystyle
M:=S\setminus\\{\boldsymbol{0}\\}$ is a good ideal and we will call it the
_maximal ideal_ of $\displaystyle S$. Moreover, by property (G1), a local good
semigroup has a minimum nonzero element. We will denote it by
$\displaystyle\boldsymbol{e}=(e_{1},\dots,e_{h})$ and call it the
_multiplicity_ vector of $\displaystyle S$.

In [4] it is proved that any good semigroup can be uniquely expressed as a
direct product of local ones, i.e. $\displaystyle S=S_{1}\times\dots\times
S_{r}$ and, denoting by $\displaystyle M_{i}$ the maximal ideal of
$\displaystyle S_{i}$, we will denote by $\displaystyle J$ the product
$\displaystyle M_{1}\times\dots\times M_{r}$. In this case, $\displaystyle J$
is a good ideal, that we will call the _Jacobson ideal_ of $\displaystyle S$.
Notice that any $\displaystyle S_{i}$ in this expression is a subsemigroup of
$\displaystyle\mathbb{N}^{h_{i}}$, with $\displaystyle
h_{1}+h_{2}+\dots+h_{r}=h$, and the subset of components
$\displaystyle\\{j_{1},\dots,j_{h_{i}}\\}\subseteq\\{1,\dots,h\\}$, where
$\displaystyle S_{i}$ lives, will be called the _support_ of $\displaystyle
S_{i}$.

The above notations will be fixed for the rest of the paper. In addition, we
will always assume that $\displaystyle S\neq\mathbb{N}^{h}$.

Notice that, for any relative ideal $\displaystyle E$, it holds true that
$\displaystyle E-E$ is a submonoid of $\displaystyle\mathbb{N}^{h}$, not
necessarily good, containing $\displaystyle S$. In particular, if
$\displaystyle S$ is a local good semigroup, then $\displaystyle
S-M=M-M=\\{\boldsymbol{\alpha}\in\mathbb{N}^{h}\mid\boldsymbol{\alpha}+M\subseteq
M\\}$ is both a monoid such that $\displaystyle S\subsetneq
M-M\subseteq\mathbb{N}^{h}$ and a relative ideal of $\displaystyle S$. The
same holds, in general, for $\displaystyle J-J$.

In the next section, we will need the following lemma in order to deal with
the non-local case. Its proof is straightforward.

######  Lemma 1.1.

Let $\displaystyle S\subset\mathbb{N}^{h}$ be a good semigroup and let
$\displaystyle S=S_{1}\times\dots\times S_{r}$ be its representation as direct
product of local good semigroups. Then $\displaystyle
J-J=(M_{1}-M_{1})\times\dots\times(M_{r}-M_{r})$.

A relative ideal that plays a central role in the theory of good semigroups is
the standard canonical ideal. In order to give its definition, we need to
introduce some more notation.

######  Definition 1.2.

Let $\displaystyle\boldsymbol{\alpha}\in\mathbb{Z}^{d}$, $\displaystyle
i\in\\{1,\dots,d\\}$. We define:

  * $\displaystyle\Delta_{i}(\alpha):=\\{\beta\in\mathbb{Z}^{d}\mid\alpha_{i}=\beta_{i}\text{, }\alpha_{j}<\beta_{j}\enspace\forall j\neq i\\}$;

  * $\displaystyle\Delta(\alpha):=\\{\beta\in\mathbb{Z}^{d}\mid\exists\,i\text{ such that }\alpha_{i}=\beta_{i}\text{, }\alpha_{j}<\beta_{j}\enspace\forall j\neq i\\}=\bigcup_{i}\Delta_{i}(\alpha)$;

  * $\displaystyle\Delta_{i}^{S}(\alpha):=\Delta_{i}(\alpha)\cap S$;

  * $\displaystyle\Delta^{S}(\alpha):=\Delta(\alpha)\cap S$.

The minimum $\displaystyle\boldsymbol{c}\in\mathbb{N}^{h}$ satisfying (G3)
will be called the _conductor_ of $\displaystyle S$. We additionally set
$\displaystyle\boldsymbol{\gamma}:=\boldsymbol{c}-\boldsymbol{1}$ (where
$\displaystyle\boldsymbol{1}=(1,1,\dots,1)$) and call it the _Frobenius
vector_ of $\displaystyle S$. Observe that, by properties (G1) and (G2),
$\displaystyle\Delta^{S}(\boldsymbol{\gamma})=\varnothing$.

######  Definition 1.3.

The _standard canonical ideal_ of $\displaystyle S$ is

$\displaystyle
K(S):=\\{\boldsymbol{\alpha}\in\mathbb{Z}^{d}\mid\Delta^{S}(\boldsymbol{\gamma}-\boldsymbol{\alpha})=\varnothing\\}.$

This relative ideal was firstly defined in the local case in [9]. In [4] it
was noticed that its definition can be stated in the general case and that, if
$\displaystyle S=S_{1}\times\dots\times S_{r}$, then $\displaystyle
K(S)=K(S_{1})\times\dots\times K(S_{r})$. It is not difficult to see that
$\displaystyle K(S)$ is a good relative ideal of $\displaystyle S$ such that
$\displaystyle S\subseteq K(S)\subseteq\mathbb{N}^{h}$.

A good relative ideal $\displaystyle K^{\prime}$ of $\displaystyle S$ is
called _canonical ideal_ of $\displaystyle S$ if there exists $\displaystyle
x\in\mathbb{Z}^{d}$ such that $\displaystyle K^{\prime}=K(S)+x$.

The class of canonical ideals can be characterized via a duality property that
we are going to state. This property was proved in [21] for the local case,
but it can be stated for any good semigroup, using the representation of
$\displaystyle K(S)$ as a direct product of standard canonical ideals.

######  Lemma 1.4.

[21, Theorem 5.2.7] Given a good relative ideal $\displaystyle K^{\prime}$ of
$\displaystyle S$, the following conditions are equivalent:

  * (i)

$\displaystyle K^{\prime}$ is a canonical ideal of $\displaystyle S$;

  * (ii)

$\displaystyle K^{\prime}-(K^{\prime}-I)=I$ for every good relative ideal
$\displaystyle I$ of $\displaystyle S$.

In the same paper, the authors proved the following key result, that, again,
holds true in the non-local case as well.

######  Lemma 1.5.

[21, Proposition 5.2.7] If $\displaystyle E$ is a good relative ideal of
$\displaystyle S$, then $\displaystyle K(S)-E$ is a good relative ideal of
$\displaystyle S$.

We list below three key properties of canonical ideals that will be useful in
the next section. Let $\displaystyle K^{\prime}$ be a canonical ideal of
$\displaystyle S$ and let $\displaystyle E,F$ be two relative ideals. Then:

  * •

$\displaystyle E\subseteq F$ if and only if $\displaystyle
K^{\prime}-F\subseteq K^{\prime}-E$;

  * •

$\displaystyle E=F$ if and only if $\displaystyle K^{\prime}-F=K^{\prime}-E$;

  * •

$\displaystyle K^{\prime}-K^{\prime}=S$.

These properties can be easily deduced by Lemma 1.4, keeping in mind, as for
the third one, that $\displaystyle K^{\prime}-K^{\prime}$ is independent of
translations of $\displaystyle K^{\prime}$ and that $\displaystyle
K(S)=K(S)-S$.

An important class of good semigroups is the class of symmetric semigroups,
which corresponds to the class of Gorenstein rings.

######  Definition 1.6.

A good semigroup $\displaystyle S$ is called _symmetric_ if, for every
$\displaystyle\boldsymbol{\alpha}\in\mathbb{N}^{d}$,
$\displaystyle\boldsymbol{\alpha}\in S$ if and only if
$\displaystyle\Delta^{S}(\boldsymbol{\gamma}-\boldsymbol{\alpha})=\varnothing$.

Notice that, for any good semigroup $\displaystyle S$,
$\displaystyle\boldsymbol{\alpha}\in S$ implies
$\displaystyle\Delta^{S}(\boldsymbol{\gamma}-\boldsymbol{\alpha})=\varnothing$,
but in general the converse is not true.

The definition of symmetry is clearly equivalent to the condition
$\displaystyle S=K(S)$; hence a good semigroup $\displaystyle S$ is symmetric
if and only if every ideal of $\displaystyle S$ is bidual (or reflexive), i.e.
$\displaystyle S-(S-E)=E$. In the non-local case, the symmetry of a good
semigroup is also equivalent to the property that every component
$\displaystyle S_{i}$ of $\displaystyle S$ is symmetric. This fact was proved
explicitly in [12] for the case $\displaystyle h=2$, but the same proof holds
in general.

Good semigroups are not finitely generated in general. Therefore, it is not
possible to define the embedding dimension of a local good semigroup just by
generalizing the definition given for local rings or for numerical semigroups.
A definition of embedding dimension of a local good semigroup is given in
[23], but it is very technical and we will not need to use it in this paper.
In any case, keeping in mind that a one-dimensional noetherian local ring
$\displaystyle(R,\mathfrak{m})$ is of maximal embedding dimension (i.e. its
embedding dimension is the same as its multiplicity) if and only if
$\displaystyle\mathfrak{m}:\mathfrak{m}=x^{-1}\mathfrak{m}$ (where
$\displaystyle x$ is a minimal reduction of $\displaystyle\mathfrak{m}$) and
the same holds for numerical semigroups, we can say that a local good
semigroup is of _maximal embedding dimension_ if $\displaystyle
M-M=M-\boldsymbol{e}$. Notice that this condition implies that $\displaystyle
M-M$ is a good semigroup (not necessarily local). This definition is
consistent with the one given in [23].

##  2 Almost symmetric good semigroups

###  2.1 The local case

######  Definition 2.1.

A local good semigroup $\displaystyle S$ is called _almost symmetric_ if
$\displaystyle K(S)+M=M$.

Notice that this is equivalent to saying that $\displaystyle K(S)+M\subseteq
M$, since the reverse inclusion always holds, because
$\displaystyle\boldsymbol{0}\in K(S)$.

Moreover, for any local good semigroup, it holds true that $\displaystyle
M-M\subseteq K(S)\cup\Delta(\boldsymbol{\gamma})$: if
$\displaystyle\boldsymbol{\alpha}\in(M-M)\setminus\Delta(\gamma)$ and
$\displaystyle\boldsymbol{\alpha}\not\in K(S)$, then there would exist a
nonzero
$\displaystyle\boldsymbol{\beta}\in\Delta^{S}(\boldsymbol{\gamma}-\boldsymbol{\alpha})$;
therefore
$\displaystyle\boldsymbol{\alpha}+\boldsymbol{\beta}\in\Delta^{S}(\boldsymbol{\gamma})$,
contradiction. Notice that, in the case of numerical semigroups, the above
inclusion is reduced to $\displaystyle M-M\subseteq K(S)\cup F$, where
$\displaystyle F$ is the Frobenius number of $\displaystyle S$.

In [4] it is shown that almost symmetric local good semigroups can be
characterized by the equality of these two sets:

######  Proposition 2.2.

[4, Lemma 3.5] A local good semigroup $\displaystyle S$ is almost symmetric if
and only if $\displaystyle K(S)\cup\Delta(\boldsymbol{\gamma})=M-M$.

Proposition 2.2 shows that, if $\displaystyle S$ is an almost symmetric local
good semigroup, then $\displaystyle M-M$ is a good relative ideal of
$\displaystyle S$, because it is the union of a good relative ideal and
$\displaystyle\Delta(\gamma)$. Hence, in this case, $\displaystyle M-M$ is a
(not necessarily local) good semigroup.

In order to prove Theorem 2.4, we firstly need the following.

######  Proposition 2.3.

Let $\displaystyle S$ be a local good semigroup, of multiplicity vector
$\displaystyle\boldsymbol{e}$, such that $\displaystyle M-M$ is a good
semigroup. Then the standard canonical ideal of $\displaystyle M-M$ is

$\displaystyle K(M-M)=(K(S)-(M-M))-\boldsymbol{e}.$

###### Proof.

For the sake of simplicity we set $\displaystyle T=M-M$ and $\displaystyle
K=K(S)$. Our aim is to show that $\displaystyle K(T)=(K-T)-\boldsymbol{e}$.

Firstly we prove that $\displaystyle K-T$ is a canonical ideal of
$\displaystyle T$. In fact, $\displaystyle K-T$ is not only a good relative
ideal of $\displaystyle S$, but also of $\displaystyle T$, because if
$\displaystyle\boldsymbol{\alpha},\boldsymbol{\delta}\in T=M-M$ and
$\displaystyle\boldsymbol{\beta}\in K-T$, we get
$\displaystyle(\boldsymbol{\alpha}+\boldsymbol{\beta})+\boldsymbol{\delta}=(\boldsymbol{\alpha}+\boldsymbol{\delta})+\boldsymbol{\beta}\in
T+(K-T)\subseteq K$, that is
$\displaystyle\boldsymbol{\alpha}+\boldsymbol{\beta}\in K-T$. By Lemmas 1.4
and 1.5, it is sufficient to show that, for every good relative ideal
$\displaystyle E$ of $\displaystyle T$, it holds
$\displaystyle(K-T)-((K-T)-E)=E$. Using the fact that for any three relative
ideals $\displaystyle A,B,C$ it holds that $\displaystyle(A-B)-C=A-(B+C)$, we
obtain

$\displaystyle(K-T)-((K-T)-E)=(K-T)-(K-(T+E)).$

Since $\displaystyle E$ and $\displaystyle K-E$ are both relative ideals of
$\displaystyle T$, then $\displaystyle T+E=E$ and $\displaystyle T+(K-E)=K-E$,
hence we get

$\displaystyle\displaystyle(K-T)-(K-(T+E))=(K-T)-(K-E)$
$\displaystyle\displaystyle=$ $\displaystyle\displaystyle
K-(T+(K-E))=K-(K-E).$

Being $\displaystyle E$ a good relative ideal of $\displaystyle T\supset S$,
it is also a good relative ideal of $\displaystyle S$, therefore
$\displaystyle K-(K-E)=E$, again by Lemma 1.4.

It remains to prove that $\displaystyle
T\subseteq(K-T)-\boldsymbol{e}\subseteq\mathbb{N}^{d}$. As for the first
inclusion, this is equivalent to saying that $\displaystyle
T+\boldsymbol{e}\subseteq K-T$. But $\displaystyle
T+\boldsymbol{e}=(M-M)+\boldsymbol{e}\subseteq M$ and $\displaystyle
M+T=M+(M-M)\subseteq M\subset K$; thus $\displaystyle
T+\boldsymbol{e}\subseteq M\subseteq K-T$, as desired.

We prove now the second inclusion,
$\displaystyle(K-T)-\boldsymbol{e}\subseteq\mathbb{N}^{d}$. First of all, we
show that $\displaystyle S-\mathbb{N}^{h}\subseteq T+\boldsymbol{e}$. Let
$\displaystyle\boldsymbol{\alpha}\in S-\mathbb{N}^{h}$; we have to prove that
$\displaystyle\boldsymbol{\alpha}-\boldsymbol{e}\in T$. This is equivalent to
proving that for any $\displaystyle\boldsymbol{\delta}\in M$,
$\displaystyle\boldsymbol{\alpha}-\boldsymbol{e}+\boldsymbol{\delta}\in M$.
But $\displaystyle\boldsymbol{\delta}-\boldsymbol{e}\geq 0$, i.e.
$\displaystyle\boldsymbol{\delta}-\boldsymbol{e}\in\mathbb{N}^{h}$, hence
$\displaystyle\boldsymbol{\alpha}+(\boldsymbol{\delta}-\boldsymbol{e})\in S$.
Since $\displaystyle\boldsymbol{0}\notin S-\mathbb{N}^{d}$, because
$\displaystyle S\subsetneq\mathbb{N}^{h}$, and
$\displaystyle\boldsymbol{\alpha}+(\boldsymbol{\delta}-\boldsymbol{e})\geq\boldsymbol{\alpha}$,
we must have
$\displaystyle\boldsymbol{\alpha}+\boldsymbol{\delta}-\boldsymbol{e}\in M$.

Finally notice that $\displaystyle K-\mathbb{N}^{h}=S-\mathbb{N}^{h}$: indeed,
$\displaystyle\boldsymbol{0}\in K$ and so $\displaystyle
K+\mathbb{N}^{h}=\mathbb{N}^{h}$; therefore $\displaystyle
S-\mathbb{N}^{h}=(K-K)-\mathbb{N}^{h}=K-(K+\mathbb{N}^{h})=K-\mathbb{N}^{h}$.
Hence $\displaystyle K-\mathbb{N}^{h}\subseteq T+\boldsymbol{e}$ and dualizing
with $\displaystyle K$ we obtain $\displaystyle
K-(T+\boldsymbol{e})\subseteq\mathbb{N}^{d}$, that is our claim. ∎

We observe that the fact that $\displaystyle K(S)-(M-M)$ is a canonical ideal
of $\displaystyle M-M$ can be deduced by [21, Corollary 5.2.12], where the
proof uses an explicit computation of $\displaystyle K-E$ for any relative
ideal $\displaystyle E$ of $\displaystyle S$. We included our proof of this
part in the above proposition, for the sake of completeness.

We now use Proposition 2.3 to prove Theorem 2.4

######  Theorem 2.4.

A local good semigroup $\displaystyle S$ is almost symmetric if and only if
$\displaystyle M-M$ is a good relative ideal of $\displaystyle S$ and
$\displaystyle M-\boldsymbol{e}$ is the standard canonical ideal of
$\displaystyle M-M$.

###### Proof.

For the sake of simplicity we set $\displaystyle K=K(S)$, $\displaystyle
T=M-M$ and $\displaystyle K^{\prime}=K(T)$.

$\displaystyle(\Leftarrow)$

Since the inclusion $\displaystyle T\subseteq
K\cup\Delta(\boldsymbol{\gamma})$ holds for every good semigroup, it is
sufficient to show that $\displaystyle K\subseteq T$. By assumption
$\displaystyle K^{\prime}=M-\boldsymbol{e}$, and by Proposition 2.3
$\displaystyle K^{\prime}=(K-T)-\boldsymbol{e}$. Hence $\displaystyle M=K-T$,
that, dualizing with $\displaystyle K$, is equivalent to $\displaystyle
K-M=K-(K-T)=T$. But since $\displaystyle K$ is a relative ideal of
$\displaystyle S$, we have $\displaystyle K\subseteq K-M=T$, as desired.

$\displaystyle(\Rightarrow)$

We have already noticed that $\displaystyle S$ almost symmetric implies that
$\displaystyle T$ is good. In order to prove the second assertion we firstly
need the following:

Claim: every good relative ideal $\displaystyle I$ of $\displaystyle T$ is
bidual as an ideal of $\displaystyle S$.
Let $\displaystyle I$ be a good relative ideal of $\displaystyle T$. We have
$\displaystyle K+I\subseteq T+I=I$, where the first inclusion is true because
$\displaystyle S$ is almost symmetric and the second one because
$\displaystyle I$ is an ideal of $\displaystyle T$. Conversely, $\displaystyle
I\subseteq K+I$ because $\displaystyle 0\in K$. Hence

(2.1) $K+I=I.$

Now suppose that $\displaystyle I$ is not bidual as ideal of $\displaystyle
S$, then $\displaystyle I\subsetneq S-(S-I)\subseteq K-(S-I)$. Dualizing with
$\displaystyle K$, this is equivalent to $\displaystyle S-I\subsetneq K-I$.
Therefore

$\displaystyle S-I\subsetneq K-I=K-(K+I)=(K-K)-I=S-I$

that is a contradiction. Hence the claim holds true.

We are now ready to prove that $\displaystyle M$ is a canonical ideal of
$\displaystyle T$. By Lemma 1.4, it is sufficient to show that $\displaystyle
M-(M-I)=I$ for every good ideal $\displaystyle I$ of $\displaystyle T$.
Firstly, we show that $\displaystyle M-I=S-I$. If not, there exists an element
$\displaystyle x\in S-I$ such that $\displaystyle x\not\in M-I$. Then
$\displaystyle x+I$ is an ideal of $\displaystyle S$ not contained in
$\displaystyle M$, i.e. $\displaystyle x+I=S$; and this means that
$\displaystyle S$ is itself a relative ideal of $\displaystyle T$, a
contradiction. Hence $\displaystyle M-I=S-I$.

Since, by the claim, $\displaystyle I$ is bidual as an ideal of $\displaystyle
S$,

$\displaystyle M-(M-I)\subseteq S-(M-I)=S-(S-I)=I.$

The other inclusion is always true, hence we get $\displaystyle M-(M-I)=I$, as
desired. Finally, it is straightforward to check that $\displaystyle
T\subseteq M-\boldsymbol{e}\subseteq\mathbb{N}^{h}$, hence $\displaystyle
M-\boldsymbol{e}$ is the standard canonical ideal of $\displaystyle T$.

∎

######  Example 2.5.

Let $\displaystyle S$ be the semigroup represented by the black dots in the
figure on the left; then its standard canonical ideal $\displaystyle K(S)$ is
obtained by adding to $\displaystyle S$ the white dots in the same figure, and
it is not difficult to check that $\displaystyle M+K(S)\subseteq M$, i.e.
$\displaystyle S$ is almost symmetric. In the figure on the right,
$\displaystyle M-M$ is depicted with black dots and $\displaystyle
M-\boldsymbol{e}$ is obtained by adding the white dots to it. As prescribed by
the previous theorem, $\displaystyle M-\boldsymbol{e}$ is the standard
canonical ideal of $\displaystyle M-M$.

01122334455667788990112233445566778899

The following result was proved in [12] for the special case $\displaystyle
h=2$. We will give a proof that holds in general, using Theorem 2.4.

######  Corollary 2.6.

A local good semigroup $\displaystyle S$ is almost symmetric of maximal
embedding dimension if and only if $\displaystyle M-M$ is a symmetric good
semigroup.

###### Proof.

As usual, we set $\displaystyle K=K(S)$ and $\displaystyle T=M-M$.

$\displaystyle(\Rightarrow)$

If $\displaystyle S$ is almost symmetric, by Theorem 2.4, $\displaystyle
M-\boldsymbol{e}$ is the standard canonical ideal of $\displaystyle T$.
Remembering that, by definition, $\displaystyle S$ is of maximal embedding
dimension if and only if $\displaystyle T=M-\boldsymbol{e}$, we get that
$\displaystyle T$ coincides with its standard canonical ideal, that is
equivalent to saying that $\displaystyle T$ is symmetric.

$\displaystyle(\Leftarrow)$

Assume that $\displaystyle T$ is good and symmetric. Therefore, by Proposition
2.3, $\displaystyle T=K(T)=(K-T)-\boldsymbol{e}$. It follows that
$\displaystyle T=K-(T+\boldsymbol{e})\supseteq K-M\supseteq K-S=K$. Hence
$\displaystyle K+M\subseteq M$, which means that is $\displaystyle S$ almost
symmetric.

Now we can apply Theorem 2.4 to get $\displaystyle K(T)=M-\boldsymbol{e}$;
therefore, since $\displaystyle T$ is symmetric, $\displaystyle T=K(T)$, i.e.
$\displaystyle M-M=M-\boldsymbol{e}$, that means that $\displaystyle S$ is of
maximal embedding dimension.

∎

######  Example 2.7.

Let $\displaystyle S$ be the semigroup represented by the black dots in the
figure on the left; then $\displaystyle M-M$ is the semigroup represented by
the black dots in the figure on the right. In this case, $\displaystyle M-M$
coincides with $\displaystyle M-\boldsymbol{e}$ and it is a symmetric good
semigroup. Hence, by the previous corollary, $\displaystyle S$ is almost
symmetric of maximal embedding dimension. The standard canonical ideal
$\displaystyle K(S)$ is obtained by adding to $\displaystyle S$ the white dots
in the figure on the left, and it is immediately seen that $\displaystyle
K(S)\cup\Delta(\boldsymbol{\gamma})=M-M$: this is another way to check that
$\displaystyle S$ is almost symmetric.

01122334455660112233445566

###  2.2 The non-local case

If $\displaystyle S=S_{1}\times\dots\times S_{r}$ is a non-local good
semigroup, using the facts that $\displaystyle J=M_{1}\times\dots\times
M_{r}$, $\displaystyle J-J=(M_{1}-M_{1})\times\dots\times(M_{r}-M_{r})$ and
$\displaystyle K(S)=K(S_{1})\times\dots\times K(S_{r})$, it is straightforward
to generalize the definitions of almost symmetry (as it has been done in [4])
and the previous theorems. More precisely, we will say that $\displaystyle S$
is _almost symmetric_ if its components are almost symmetric for every
$\displaystyle i=1,\dots,r$. Moreover, denoting by
$\displaystyle\boldsymbol{e_{i}}$ the multiplicity vector of $\displaystyle
S_{i}$, we can state the following results. Their proofs are straightforward
since, if $\displaystyle E,F$ are relative ideals of $\displaystyle S$ of the
form $\displaystyle E_{1}\times\dots\times E_{r}$ and $\displaystyle
F_{1}\times\dots\times F_{r}$, where the supports of $\displaystyle E_{i}$ and
$\displaystyle F_{i}$ are the same as the supports of $\displaystyle S_{i}$,
then $\displaystyle E-F=(E_{1}-F_{1})\times\dots\times(E_{r}-F_{r})$ and hence
we can reduce our proofs to the local case.

######  Proposition 2.8.

Preserving the above notations, let $\displaystyle S_{1}\times\dots\times
S_{r}$ be a good semigroup and let
$\displaystyle\boldsymbol{e}=(\boldsymbol{e_{1}},\dots,\boldsymbol{e_{r}})$
and assume that $\displaystyle J-J$ is a good semigroup. Then the standard
canonical ideal of $\displaystyle J-J$ is

$\displaystyle K(J-J)=(K(S)-(J-J))-\boldsymbol{e}.$

######  Theorem 2.9.

A good semigroup $\displaystyle S$ is almost symmetric if and only if
$\displaystyle J-J$ is a good relative ideal of $\displaystyle S$ and
$\displaystyle J-\boldsymbol{e}$ is the standard canonical ideal of
$\displaystyle J-J$.

######  Corollary 2.10.

A good semigroup $\displaystyle S$ is almost symmetric and $\displaystyle
J-J=J-\boldsymbol{e}$ if and only if $\displaystyle J-J$ is a symmetric good
semigroup.

##  3 Almost Gorenstein rings

We conclude this paper by applying the previous results to noetherian,
analytically unramified, residually rational, one-dimensional, reduced,
semilocal rings.

As it is described in [4], under these hypotheses, the integral closure
$\displaystyle\overline{R}$ of $\displaystyle R$ is a direct product of
discrete valuation rings $\displaystyle V_{1}\times\dots\times V_{h}$, and
therefore it is possible to associate with such a ring $\displaystyle R$ a
value semigroup $\displaystyle v(R)$ that is a good subsemigroup of
$\displaystyle\mathbb{N}^{h}$. Note that the residually rational hypothesis
means that all the residue fields of $\displaystyle R$ and
$\displaystyle\overline{R}$ are isomorphic; if moreover the cardinality of
these residue fields is bigger than $\displaystyle h$, then the connection
between the ring and its value semigroup is particularly meaningful and we
will assume these two hypotheses till the end of the paper. One of the reasons
for this connection is the fact that, if $\displaystyle I\subseteq J$ are two
fractional ideals of $\displaystyle R$, then the length
$\displaystyle\ell(J/I)$ of $\displaystyle J/I$, as an $\displaystyle
R$-module, can be computed looking at the value ideals $\displaystyle v(I)$
and $\displaystyle v(J)$ of $\displaystyle v(R)$; in particular, we will use
the fact that $\displaystyle I=J$ if and only if $\displaystyle v(I)=v(J)$
(see [9, Corollary 2.5]). We also recall that algebroid curves belong to this
class of rings and that, in this case, the integer $\displaystyle h$
represents the number of branches of the singularity.

If $\displaystyle R$ is semilocal with maximal ideals
$\displaystyle\mathfrak{m}_{1},\dots,\mathfrak{m}_{j}$ (with $\displaystyle
j\leq h$), then $\displaystyle R\cong R_{\mathfrak{m}_{1}}\times\dots\times
R_{\mathfrak{m}_{j}}$ (see [4, Corollary 3.2]). From this, it follows
immediately that $\displaystyle v(R)=v(R_{\mathfrak{m}_{1}})\times\dots\times
v(R_{\mathfrak{m}_{j}})$ is the unique representation of $\displaystyle v(R)$
as a direct product of local good semigroups. Moreover, the Jacobson radical
$\displaystyle J(R)$ coincides with
$\displaystyle\mathfrak{m}_{1}R_{\mathfrak{m}_{1}}\times\dots\times\mathfrak{m}_{j}R_{\mathfrak{m}_{j}}$
and hence $\displaystyle v(J(R))=J$, where $\displaystyle J$ is the Jacobson
ideal of $\displaystyle v(R)$.

In this context, we give the following definition.

######  Definition 3.1.

A noetherian, analytically unramified, one-dimensional, reduced local ring is
said to be _almost Gorenstein_ if
$\displaystyle\mathfrak{m}\omega_{R}=\mathfrak{m}$, where
$\displaystyle\omega_{R}$ is a canonical ideal of $\displaystyle R$ with the
property that $\displaystyle R\subseteq\omega_{R}\subseteq\overline{R}$.

If $\displaystyle R$ is semilocal, following again [4], we will say that
$\displaystyle R$ is almost Gorenstein if $\displaystyle R_{\mathfrak{m}_{i}}$
is almost Gorenstein for any maximal ideal $\displaystyle\mathfrak{m}_{i}$ of
$\displaystyle R$.

In the local case, $\displaystyle(R,\mathfrak{m})$ is almost Gorenstein if and
only if $\displaystyle v(R)$ is almost symmetric and the Cohen-Macaulay type
of $\displaystyle R$ equals the type of $\displaystyle v(R)$ ([4, Proposition
3.7]). In fact, setting $\displaystyle v(R)=S$ and $\displaystyle
M=v(\mathfrak{m})$, in that paper it is proved that if $\displaystyle R$ is
almost Gorenstein then $\displaystyle
v(\mathfrak{m}:\mathfrak{m})=M-M=K(S)\cup\Delta(\boldsymbol{\gamma})$ and,
therefore, we could restate that result saying that
$\displaystyle(R,\mathfrak{m})$ is almost Gorenstein if and only if
$\displaystyle v(R)$ is almost symmetric and $\displaystyle
v(\mathfrak{m}:\mathfrak{m})=M-M$ (notice that, in general, only the inclusion
$\displaystyle v(\mathfrak{m}:\mathfrak{m})\subseteq M-M$ holds). In light of
Theorem 2.4 we can further reformulate this fact in the following way:

######  Corollary 3.2.

Preserving the above assumptions, set $\displaystyle S=v(R)$ and
$\displaystyle M=v(\mathfrak{m})$. Then $\displaystyle(R,\mathfrak{m})$ is
almost Gorenstein if and only if $\displaystyle M-\boldsymbol{e}$ is the
standard canonical ideal of $\displaystyle M-M$ and $\displaystyle
v(\mathfrak{m}:\mathfrak{m})=M-M$.

This result is not particularly useful from a computational point of view,
since it is not easy at all to determine
$\displaystyle\mathfrak{m}:\mathfrak{m}$ and its value semigroup. However, if
we restrict ourselves to the maximal embedding dimension case, we get a
stronger result.

######  Proposition 3.3.

Preserving the above assumptions, set $\displaystyle S=v(R)$, $\displaystyle
M=v(\mathfrak{m})$ and assume that $\displaystyle R$ is of maximal embedding
dimension. The following conditions are equivalent:

  * (i)

$\displaystyle(R,\mathfrak{m})$ is almost Gorenstein;

  * (ii)

$\displaystyle M-\boldsymbol{e}$ is a good symmetric semigroup;

  * (iii)

$\displaystyle S$ is almost symmetric.

###### Proof.

$\displaystyle(i)\Leftrightarrow(ii)$

It is well known that $\displaystyle R$ is of maximal embedding dimension if
and only if $\displaystyle\mathfrak{m}:\mathfrak{m}=\mathfrak{m}x^{-1}$, where
$\displaystyle x\in\mathfrak{m}$ is a minimal reduction of
$\displaystyle\mathfrak{m}$, i.e., in our context, an element of minimal non-
zero value (that is $\displaystyle\boldsymbol{e}$). Hence $\displaystyle
v(\mathfrak{m}x^{-1})=M-\boldsymbol{e}$. Moreover, the equality
$\displaystyle\mathfrak{m}:\mathfrak{m}=\mathfrak{m}x^{-1}$ holds if and only
if $\displaystyle\mathfrak{m}x^{-1}$ is a ring.

Applying [6, Proposition 25], we get that $\displaystyle R$ is almost
Gorenstein if and only if
$\displaystyle\mathfrak{m}:\mathfrak{m}=\mathfrak{m}x^{-1}$ is a Gorenstein
ring, that implies immediately that $\displaystyle
M-\boldsymbol{e}=v(\mathfrak{m}x^{-1})$ is a good symmetric semigroup.

Conversely, if $\displaystyle M-\boldsymbol{e}$ is a good symmetric semigroup,
since by hypothesis
$\displaystyle\mathfrak{m}x^{-1}=\mathfrak{m}:\mathfrak{m}$ is a ring, it has
to be Gorenstein, having a symmetric value semigroup. The thesis follows again
applying [6, Proposition 25].

$\displaystyle(ii)\Leftrightarrow(iii)$

We need only to observe that $\displaystyle
v(\mathfrak{m}:\mathfrak{m})\subseteq M-M\subseteq
M-\boldsymbol{e}=v(\mathfrak{m}x^{-1})$ and so the hypothesis that
$\displaystyle R$ is of maximal embedding dimension implies that
$\displaystyle\subseteq M-M\subseteq M-\boldsymbol{e}$, i.e. $\displaystyle S$
is of maximal embedding dimension. Now the thesis follows by Corollary 2.6

∎

######  Example 3.4.

The semigroup considered in Example 2.7 is the value semigroup of the ring
$\displaystyle R=k[[x,y,z,w]]/P_{1}\cap P_{2}$, where $\displaystyle k$ is any
field of with at least $\displaystyle 3$ elements, $\displaystyle
P_{1}=(x^{3}-y^{2},z-x^{2},w-xy)$ and $\displaystyle P_{2}=(x^{3}-y^{2},z,w)$.
This fact can be seen by considering the homomorphism $\displaystyle
k[[x,y,z,w]]\rightarrow k[[t]]\times k[[u]]$ induced by $\displaystyle
x\mapsto(t^{2},u^{2})$, $\displaystyle y\mapsto(t^{3},u^{3})$, $\displaystyle
z\mapsto(t^{4},0)$ and $\displaystyle w\mapsto(t^{5},0)$, whose kernel is
$\displaystyle P_{1}\cap P_{2}$. The ring $\displaystyle R$ is of maximal
embedding dimension, since also its multiplicity is $\displaystyle 4$ (i.e.
the sum of the components of the minimal non-zero value), and as explained in
Example 2.7, $\displaystyle M-\boldsymbol{e}$ is a symmetric good semigroup.
Hence $\displaystyle R$ is almost Gorenstein.

In light of the above discussion on the semilocal case, we can easily
generalize the previous results.

######  Corollary 3.5.

Let $\displaystyle R$ be a one-dimensional, noetherian, analytically
unramified, residually rational, reduced semilocal ring. Set $\displaystyle
S=v(R)$ and let $\displaystyle J=v(J(R))$ be its Jacobson ideal. The following
conditions are equivalent:

  * (i)

$\displaystyle R$ is almost Gorenstein;

  * (ii)

$\displaystyle J(R)$ is a canonical ideal of $\displaystyle J(R):J(R)$;

  * (iii)

$\displaystyle S$ is almost symmetric and $\displaystyle v(J(R):J(R))=J-J$;

  * (iv)

$\displaystyle J$ is a canonical ideal of $\displaystyle J-J$ and
$\displaystyle v(J(R):J(R))=J-J$.

###### Proof.

$\displaystyle(i)\Leftrightarrow(ii)$

By definition $\displaystyle R$ is almost Gorenstein if and only if
$\displaystyle R_{\mathfrak{m}_{i}}$ is almost Gorenstein for every
$\displaystyle i=1,\dots,j$. By [14, Proposition 3.2] this is equivalent to
saying that $\displaystyle\mathfrak{m}_{i}R_{\mathfrak{m}_{i}}$ is a canonical
ideal of
$\displaystyle({\mathfrak{m}_{i}R_{\mathfrak{m}_{i}}}:{\mathfrak{m}_{i}R_{\mathfrak{m}_{i}}})$.
Since $\displaystyle
J(R)=\mathfrak{m}_{1}R_{\mathfrak{m}_{1}}\times\dots\times\mathfrak{m}_{j}R_{\mathfrak{m}_{j}}$
and $\displaystyle
J(R):J(R)=({\mathfrak{m}_{1}R_{\mathfrak{m}_{1}}}:{\mathfrak{m}_{1}R_{\mathfrak{m}_{1}}})\times\dots\times({\mathfrak{m}_{j}R_{\mathfrak{m}_{j}}}:{\mathfrak{m}_{j}R_{\mathfrak{m}_{j}}})$,
the thesis follows immediately.

$\displaystyle(iii)\Leftrightarrow(iv)$

In both conditions it is assumed that $\displaystyle v(J(R):J(R))=J-J$, hence
$\displaystyle J-J$ is good and the equivalence follows applying Theorem 2.9.

$\displaystyle(i)\Leftrightarrow(iii)$

By [4] we know that, for every $\displaystyle i=1,\dots,j$, the ring
$\displaystyle R_{\mathfrak{m}_{i}}$ is almost Gorenstein if and only if
$\displaystyle S_{i}$ is almost symmetric and $\displaystyle
v(\mathfrak{m}_{i}R_{\mathfrak{m}_{i}}:\mathfrak{m}_{i}R_{\mathfrak{m}_{i}})=M_{i}-M_{i}$.
The thesis is now straightforward.

∎

Like in the local case, the previous result is not very useful from a
computational point of view, but we can state a semilocal version of
Proposition 3.3.

######  Corollary 3.6.

We preserve the above notations. Let $\displaystyle R$ be a one-dimensional,
noetherian, analytically unramified, residually rational, reduced semilocal
ring; identify $\displaystyle R$ with $\displaystyle
R_{\mathfrak{m}_{1}}\times\dots\times R_{\mathfrak{m}_{j}}$ and set
$\displaystyle x=(x_{1},\dots,x_{j})$, where $\displaystyle x_{i}$ is a
minimal reduction of $\displaystyle\mathfrak{m}_{i}R_{\mathfrak{m}_{i}}$ (for
any $\displaystyle i=1,\dots,j$); set $\displaystyle S=v(R)$ and let
$\displaystyle J=v(J(R))$ be its Jacobson ideal. Assume that $\displaystyle
R_{\mathfrak{m}_{i}}$ is of maximal embedding dimension for every
$\displaystyle i=1,\dots,j$ (or, equivalently, that $\displaystyle
J(R)x^{-1}=J(R):J(R)$). Then the following conditions are equivalent:

  * (i)

$\displaystyle R$ is almost Gorenstein;

  * (ii)

$\displaystyle J-\boldsymbol{e}$ is a good symmetric semigroup (where
$\displaystyle\boldsymbol{e}=(\boldsymbol{e_{1}},\dots,\boldsymbol{e_{r}})$
and $\displaystyle\boldsymbol{e}_{i}$ is the multiplicity vector of
$\displaystyle S_{i}=v(R_{\mathfrak{m}_{i}}$).

Statements and Declarations. On behalf of all authors, the corresponding
author states that there is no conflict of interest.

## References

  * [1]
  * [2] V. Barucci, Local rings of minimal length, J. Pure Appl. Algebra 213 (2009), no. 6, 991–996.
  * [3] V. Barucci, On propinquity of numerical semigroups and one-dimensional local Cohen Macaulay rings, in: Commutative Algebra and its Applications (M. Fontana, S. Kabbaj, O. Olberding, I. Swanson eds.) de Gruyter, Berlin, New York (2009), 49–60.
  * [4] V. Barucci, M. D’Anna, R. Fröberg, _Analytically unramified one-dimensional semilocal rings and their value semigroups_ , J. Pure Appl. Alg. 147 (2000), 215-254.
  * [5] V. Barucci, M. D’Anna, R. Fröberg, _The semigroup of values of a one-dimensional local ring with two minimal primes_ , Comm. Algebra 28(8) (2000), 3 607-3633.
  * [6] V. Barucci, R. Fröberg, One-dimensional almost Gorenstein rings, J. of Algebra 188 (1997), 418–442.
  * [7] A. Campillo, F. Delgado, S. M. Gusein-Zade, _On generators of the semigroup of a plane curve singularity_. J. London Math. Soc. (2) 60 (1999), 420-430.
  * [8] A. Campillo, F. Delgado, K. Kiyek, _Gorenstein properties and symmetry for one-dimensional local Cohen-Macaulay rings_ , Manuscripta Math. 83 (1994), 405-423.
  * [9] M. D’Anna, The canonical module of a one-dimensional reduced local ring, Comm. Algebra 25 (1997), 2939–2965.
  * [10] M. D’Anna, P. Garcia Sanchez, V. Micale, L. Tozzo, _Good semigroups of $\displaystyle N^{n}$_. International Journal of Algebra and Computation, vol. 28 (2018), p. 179-206.
  * [11] M. D’Anna, L. Guerrieri, V. Micale, _The Apéry Set of a Good Semigroup_. (2018) arXiv:1812.02064.
  * [12] M. D’Anna, L. Guerrieri and V. Micale, The type of a good semigroup and the almost symmetric condition, Mediterranean Journal of Mathematics 17 (2020).
  * [13] M. D’Anna and F. Strazzanti, Almost Canonical Ideals and GAS Numerical Semigroups, arXiv:2003.13061.
  * [14] M. D’Anna and F. Strazzanti, When is $\displaystyle\mathfrak{m}:\mathfrak{m}$ almost Gorenstein?, arXiv:2003.13061.
  * [15] F. Delgado, _The semigroup of values of a curve singularity with several branches_ , Manuscripta Math. 59 (1987), 347-374.
  * [16] F. Delgado, _Gorenstein curves and symmetry of the semigroup of value_ , Manuscripta Math. 61 (1988), 285-296. Perspectives, Springer.
  * [17] A. García, _Semigroups associated to singular points of plane curves_ , J. Reine Angew. Math. 336 (1982), 165-184.
  * [18] S. Goto, N. Matsuoka, T.T. Phuong, Almost Gorenstein rings, arXiv:1106.1301v2 (2011).
  * [19] S. Goto, R. Takahashi, N. Taniguchi, Almost Gorenstein rings - towards a theory of higher dimension, J. Pure Appl. Algebra 219 (2015), no. 7, 2666–2712.
  * [20] J. Jäger, _Längenberechnung und kanonische Ideale in eindimensionalen Ringen_ , Arch. Math. (Basel) 29 (1977), no. 5, 504-512.
  * [21] P. Korell, M. Schulze and L. Tozzo, Duality on value semigroups, Comm. Algebra 11 (2019), 81–129.
  * [22] E. Kunz, _The value-semigroup of a one-dimensional Gorenstein ring_ , Proc. Amer. Math. Soc. 25 (1970), 748-751.
  * [23] N. Maugeri, G. Zito, _Embedding dimension of a good semigroup_ , (2019) arXiv:1903.02057


# Attention-based sequential recommendation system using multimodal data

Hyungtaik Oh<EMAIL_ADDRESS>Wonkeun Jo<EMAIL_ADDRESS>Dongil Kim
<EMAIL_ADDRESS>
###### Abstract

Sequential recommendation systems that model dynamic preferences based on a
user’s past behavior are crucial to e-commerce. Recent studies on these
systems have considered various types of information such as images and texts.
However, multimodal data have not yet been utilized directly to recommend
products to users. In this study, we propose an attention-based sequential
recommendation method that employs multimodal data of items such as images,
texts, and categories. First, we extract image and text features from pre-
trained VGG and BERT and convert categories into multi-labeled forms.
Subsequently, attention operations are performed independent of the item
sequence and multimodal representations. Finally, the individual attention
information is integrated through an attention fusion function. In addition,
we apply multitask learning loss for each modality to improve the
generalization performance. The experimental results obtained from the Amazon
datasets show that the proposed method outperforms those of conventional
sequential recommendation systems.

###### keywords:

Recommendation system, Sequential recommendation, Multimodal data, Attention
mechanism, Deep learning

††journal: Expert Systems with Applications

[label1]organization=Department of Computer Science and Engineering, Chungnam
National University,addressline=99 Daehak–ro, city=Yuseong–gu, postcode=34134,
state=Daejeon, country=Republic of Korea [label2]organization=Department of
Data Science, Ewha Womans University,addressline=52 Ewhayeodae–gil,
city=Seodaemun–gu, postcode=03760, state=Seoul, country=Republic of Korea

##  1 Introduction

Recommendation systems provide information on items preferred by users on
e-commerce websites (Shao et al., 2021). Although they change dynamically,
user preferences can be deduced from the purchase history of users. For
example, a user who purchases a shirt may subsequently purchase a jacket or
pants. In this manner, a sequential recommendation system models dynamic
preferences from the past purchase behavior of users and recommends the next
item to purchase (Kang and McAuley, 2018).

As shown in Fig. 1, users usually check various sources of information, such
as item image, description, and satisfaction scores, on an e-commerce website
before purchasing an item. Such multimodal data contain important information
about user behavior and can be used to train recommendation models (Chen et
al., 2019). For example, various types of information, such as thumbnails,
movie storylines, and casts, can play a key role in video recommendation (Wei
et al., 2019). However, utilizing multimodal data is challenging because they
have different forms that require preprocessing steps.

Figure 1: A sequence of items purchased by the user, which contains multimodal
data such as images, text, and categories.

Neural networks have been applied to sequential recommendation systems to
improve their performance by utilizing various types of data. Convolution
neural networks (CNNs) were employed to learn the user sequence patterns as
image data, whereas recurrent neural networks (RNNs) were employed to learn
the user sequence patterns with recurrent hidden states (Hidasi et al., 2015;
Tang and Wang, 2018). In addition, some studies showed that using an RNN can
extract useful information from multimodal data such as images and text (Cui
et al., 2018; Yan and Zhang, 2019). Meanwhile, the attention mechanism is a
popular calculation module in neural networks and shows promising results when
applied to sequential recommendation systems (Vaswani et al., 2017). For
example, self-attention networks can solve long-range dependency problems by
adaptively assigning weights to the previous items in a sequence. Recent
research has also attempted to reflect side information, such as categories
and positions, in attention-based sequential recommendation systems (Xie et
al., 2022; Zhang et al., 2019; Zhou et al., 2020). These methods resulted in a
significant improvement in performance. However, they only used a limited
amount of structured information, and not unstructured multimodal data such as
images and text.

In this study, we propose a novel attention-based sequential recommendation
system using multimodal data. Using a Multimodal Attention Fusion (MAF)
method, the proposed system applies multimodal information extracted from
images, texts, and categories to predict the next purchase items of users
through attention-based sequential recommendation. The multimodal data of
users’ sequential behavior were collected from the purchase history, item
image, item description text, and item category information, and the image and
text features were extracted using pre-trained models. These data were,
subsequently, used as input data for neural networks with attention modules,
and attention operations were performed independently of item sequence and
multimodal representation. The attention fusion function combines each
attention result to adjust the different weights of the sequential multimodal
representations. In the training process, multitask learning was employed to
better represent each data modality. The experimental results show that the
proposed method improved the recommendation performance on the Amazon
datasets. The main contributions of this study are summarized as follows:

  1. $\bullet$

The proposed method utilizes multimodal data from images, text, and
categories. Features are extracted directly from each modality and the
attention operations are performed, following which, the information is
combined by the MAF.

  2. $\bullet$

Multitask learning is employed for a better generalization performance. In the
training process, the loss function comprises each task-specific loss for
predicting the item ID, image, text, and category.

  3. $\bullet$

Experiments on four datasets collected from Amazon show that our model
outperforms other recently proposed models.

##  2 Related work

###  2.1 Sequential recommendation

The recommendation system have been widely studied based on collaborative
filtering (Sarwar et al., 2001), matrix factorization (Koren et al., 2009;
Mnih and Salakhutdinov, 2007), top–N recommendation (Hu et al., 2008; Pan et
al., 2008), and their hybrid method (Walek and Fojtik, 2020); however, most of
them cannot easily model the sequential purchasing patterns of users. To
overcome this drawback of the conventional methods, sequential recommendation
system has recently been studied to model the purchasing patterns of users.
Moreover, deep learning techniques have enabled us to apply sequential
recommendation to neural networks to more effectively model user-item
interaction. For example, the convolutional sequence embedding recommendation
model (CASER) (Tang and Wang, 2018), which uses a convolutional filter,
considers the item sequence of users as an image and learns sequential
information through convolutional operations. RNNs have also been utilized
with a gated recurrent unit (GRU) to learn the sequential patterns of users
(GRU4Rec) (Hidasi et al., 2015). The RNN unit enables us to input the hidden
state and the current input at each point in time, while considering the
sequential dependencies between items purchased by the user. In addition,
self-attentive sequential recommendation (SASRec) methods perform extremely
well owing e to their ability to capture long-range dependencies between
items. SASRec excels in the unidirectional learning of user sequential
patterns through self-attention technique, and computes results faster than
previously proposed CNN/RNN-based sequential models (Kang and McAuley, 2018).
Finally, sequential recommendation with bidirectional encoder representations
from transformer (BERT4Rec) is an improved model that performs bidirectional
self-attention by applying random masking to the input sequence (Sun et al.,
2019).

###  2.2 Attention-based sequential recommendation to reflect side information

Previous studies have proved that images and text are useful features for
recommending new items (Han et al., 2021; He and McAuley, 2016; Iqbal et al.,
2018; Liu et al., 2019). Recently, attempts have been made to integrate the
side information of items such as category and location into attention-based
sequential recommendations. SASRecF, which is an improved version of SASRec,
uses multimodal item information. The embedding layer of SASRecF directly
employs the side information of the item with the sequence to be used as the
input for the model. The feature-level deeper self-attention (FDSA) method
integrates several types of side information via a vanilla attention mechanism
and performs self-attention operations on the sequences and side information
(Zhang et al., 2019). It subsequently concatenates the outputs of sequences
and multimodal blocks into fully connected layers for next-item
recommendations. The decoupled side information fusion for sequential
recommendation (DIF-SR) is another method that reflects side information. It
decouples the information between attention operations and learns the relative
importance of each type of side information. Consequently, this method has
high attention representation capacity and flexibility because the decoupled
attention improves the final representation by breaking the rank bottleneck
(Xie et al., 2022). Moreover, a multi-task training method was used to further
activate side formation. All the above-mentioned methods consider side
information in attention-based sequential recommendation. However, the side
information does not include complicated information such as images and texts;
only simple data such as categories and positions are considered.

##  3 Proposed method

###  3.1 Task definition and overall framework

Figure 2: Proposed model’s structure: 1) Features are extracted with pre-
trained VGG, BERT, and etc., 2) the attention operation with multimodal
representations, and 3) multi-task learning for better generalization.

The problem addressed in this study can be formulated with the following
notation: Let $I$ and $U$ denote the sets of items and users, respectively.
For multimodal data, we used $V,T,$ and $C$ to represent sets of images, text,
and categories, respectively. For each user $u\in U$, we used
$i_{1:n}=\\{i_{1},i_{2},\cdots,i_{n}\\}$ to represent the purchased item ID
sequence where $n$ represents the sequence length. Similarly,
$v_{1:n}=\\{v_{1},v_{2},\cdots,v_{n}\\},t_{1:n}=\\{t_{1},t_{2},\cdots,t_{n}\\}$,
and $c_{1:n}=\\{c_{1},c_{2},\cdots,c_{n}\\}$ denote the corresponding visual,
text and category, respectively. We use $i_{k}$ to represent the items that
user $u$ interacted with at the $k$-th time step, and $v_{k}$, $t_{k}$, and
$c_{k}$ are the corresponding visuals, texts, and categories of $i_{k}$. Based
on these notations, the task of our sequential recommendations is to learn an
objective function $f(\cdot)$ that recommends the possible purchase item after
the $n$-th item that the user $u$ will interact with at time $i_{n+1}$ given
multimodal data from $i_{1:n}\in\mathbb{R}^{n\times d}$,
$v_{1:n}\in\mathbb{R}^{n\times d_{v}}$, $t_{1:n}\in\mathbb{R}^{n\times
d_{t}}$, and $c_{1:n}\in\mathbb{R}^{n\times d_{c}}$ as shown in Eq. 1. Here,
$d$ indicates the latent (hidden) size of item ID, and $d_{v},d_{t}$, and
$d_{c}$ denotes the latent size of each multimodal piece of information. The
overall model structure is illustrated in Fig. 2 and comprises three main
modules: Embedding module, Multimodal fusion module, Prediction module.

$f(i_{1:n},v_{1:n},t_{1:n},c_{1:n})\to i_{n+1}$ (1)

###  3.2 Embedding module

The embedding module extracts the embedding vector for each data point from
the input sequence of the user. To use multimodal data, image and text
features are extracted using a pre-trained VGG network, denoted as
$v_{i}\in\mathbb{R}^{1\times d_{v}}$ and a pre-trained BERT network, denoted
as $t_{i}\in\mathbb{R}^{1\times d_{t}}$, respectively. Subsequently, the
category is transformed into a multi-label form, denoted as
$c_{i}\in\mathbb{R}^{1\times d_{c}}$. Because each user has different
interaction times with items, the data is pre-processed using a fixed length
$n$ for a neural network. If the length of the sequence is less than $n$, it
replaces the value with padding; otherwise, it is truncated by $n$. The user
item embeddings $E_{ID}$ and multimodal data embeddings $E_{v},E_{t},E_{c}$
are obtained via the embedding module and can be represented by Eq. 2.

$\begin{split}E_{ID}&=\mathrm{concat}([i_{1},i_{2},\cdots,i_{n}])\\\
E_{v}&=\mathrm{concat}([v_{1},v_{2},\cdots,v_{n}])\\\
E_{t}&=\mathrm{concat}([t_{1},t_{2},\cdots,t_{n}])\\\
E_{c}&=\mathrm{concat}([c_{1},c_{2},\cdots,c_{n}])\end{split}$ (2)

###  3.3 Multimodal fusion module

The multimodal fusion module comprises an MAF and feed-forward layer. The MAF
layer contains independent attention operations and an attention fusion
function that combines sequential multimodal representations. For the
attention operation, multimodal data of different dimensions were processed to
obtain data with the same dimension through linear transformations. Because
images, texts, and categories extracted from each item contains information
related to each other, they are added and expressed as one multimodal, as
shown in Eq. 3, where the $W_{v}$, $W_{t}$, $W_{c}$, and $d^{\prime}$ indicate
the linear embedding matrix for each data point and the dimension for
connecting multimodal data (Cui et al., 2018).

$\begin{split}E_{m}=E_{v}W_{v}+E_{t}W_{t}+E_{c}W_{c}\\\
W_{v}\in\mathbb{R}^{d_{v}\times d^{\prime}},W_{t}\in\mathbb{R}^{d_{t}\times
d^{\prime}},W_{c}\in\mathbb{R}^{d_{c}\times d^{\prime}}\end{split}$ (3)

When performing the attention operation, the SASRec’s attention calculation
process was employed (Kang and McAuley, 2018). Additionally, multimodal
representations are added directly block by block, reducing the computational
complexity and preventing overfitting (Xie et al., 2022). The formula for the
overall structure of the fusion module can be expressed as shown in Eq. 4:
$R^{k}$ denotes the updated sequence that has passed through the $k$–th block
from the sequence and the multimodal input, $FFN$ is a fully connected feed-
forward network, and $LN$ is the layer normalization. When $k=1$, $R^{1}$
refers to the $E_{ID}$.

$R^{k+1}=LN(FFN(MAF(R^{k},E_{m})))$ (4)

In the case of $MAF(\cdot)$, independent attention operations have a flexible
gradient and higher rank, thus improving the representation of the model,
enabling it to adaptively learn sequences and multimodality (Xie et al.,
2022). The item ID attention matrix $att_{ID}^{i}$ is calculated by
$W_{Q}^{i},W_{K}^{i},W_{V}^{i}\in\mathbb{R}^{d_{i}\times d_{h}}$ that is
divided by the size $d_{h}=d/h$ for $h$’s multi heads among all dimensions $d$
from the sequence $R^{k}$. Similarly, $W_{Q}^{m}$,
$W_{K}^{m}\in\mathbb{R}^{d^{\prime}\times d^{\prime}_{h}}$ is used to compute
the multimodal attention matrix $att_{m}^{i}$. In this case, the multimodal
dimension $d^{\prime}$ should be smaller than or equal to the sequence
dimension $d$ to prevent overfitting and to reduce the amount of computation.
Attention scores for item ID representation and multimodal representation are
calculated as follows:

$\begin{split}att_{ID}^{i}&=(R^{k}W_{Q}^{i})(R^{k}W_{K}^{i})^{\mathsf{T}}\\\
att_{m}^{i}&=(E_{m}W_{Q}^{m})(E_{m}W_{K}^{m})^{\mathsf{T}}\end{split}$ (5)

The Sum, Concat, and Gate functions proposed in previous studies were used to
combine the sequences and multimodal attention fusion function $F(\cdot)$ (Liu
et al., 2021). $MAF_{att}^{i}$ is obtained with the $W_{V}^{i}$ from the
attention fusion function $F$ for mixing two attention matrices. $\sqrt{d}$ is
a scale factor that avoids overgrowth values for the inner products. Finally,
the result outputs of all the attention heads are concatenated and fed into
the feed-forward layers and layer normalization. This process is represented
as follows:

$\begin{split}MAF_{att}^{i}&=F(att_{ID}^{i},att_{m}^{i})\\\
MAF_{head}^{i}&=softmax(\frac{MAF_{att}^{i}}{\sqrt{d}})(R^{k}W_{V}^{i})\\\
MAF(R^{k},E_{m})&=\mathrm{concat}([MAF_{head}^{1},\cdots,MAF_{head}^{i}])\end{split}$
(6)

###  3.4 Prediction module

To recommend a new item, the last element $R_{n}^{L}\in\mathbb{R}^{1\times d}$
is taken from the final representation $R^{L}$. $R_{n}^{L}$ computes to the
inner product with the item embedding table
$M_{id}\in\mathbb{R}^{d\times|I|}$, Where $|I|$ represents the total number of
distinct items. In addition, the softmax function can be used to derive the
probability value of the item that the user will purchase. According to Eq. 7,
a cross-entropy loss with softmax helps to decrease the difference in
probabilities between the actual user-purchased items and the predicted items.

$\begin{split}\hat{i}_{n+1}&=softmax(R_{n}^{L}M_{id})\\\
L_{id}&=-\sum_{i=1}^{|l|}i_{n+1}\log\hat{i}_{n+1}\end{split}$ (7)

Furthermore, the use of multi-task learning can improve the generalization of
the model and obtain more diverse informations from multimodal data (Han et
al., 2021). For images and text in multimodal data, the reconstruction loss
function can be used to compute the difference between the original
representation $v_{n+1},t_{n+1}$ and the representation output from the
attention fusion module $\hat{v}_{n+1},\hat{t}_{n+1}$, as shown in Eq. 8.
$W_{v}^{\prime}\in\mathbb{R}^{d\times d_{v}}$,
$b_{v}^{\prime}\in\mathbb{R}^{1\times d_{v}}$ and
$W_{t}^{\prime}\in\mathbb{R}^{d\times d_{t}}$,
$b_{t}^{\prime}\in\mathbb{R}^{1\times d_{t}}$ are learnable parameters that
aim to remove noise generated through layers and enhance the representations
of image and text data (Chen et al., 2012). $|V|$ and $|T|$ represent the
total number of distinct images and the total number of distinct texts.

$\begin{split}\hat{v}_{n+1}&=R_{n}^{L}W^{\prime}_{v}+b_{v}^{\prime}\qquad
L_{v}=\sum_{v=1}^{|V|}\left\|v_{n+1}-\hat{v}_{n+1}\right\|^{2}\\\
\hat{t}_{n+1}&=R_{n}^{L}W^{\prime}_{t}+b_{t}^{\prime}\qquad
L_{t}=\sum_{t=1}^{|T|}\left\|t_{n+1}-\hat{t}_{n+1}\right\|^{2}\end{split}$ (8)

For categories that show multiple label forms, the real category $c_{n+1}$ of
items and the predicted category $\hat{c}_{n+1}$ from the attention fusion
module are learned using a binary cross-entropy loss, as expressed in Eq. 9.
$W_{c}^{\prime}\in\mathbb{R}^{d\times d_{c}}$ and
$b_{c}^{\prime}\in\mathbb{R}^{1\times d_{c}}$ are learnable parameters,
$\sigma$ is sigmoid. Where $|C|$ represents the total number of item-specific
categories.

$\begin{split}\hat{c}_{n+1}&=\sigma(R_{n}^{L}W_{c}^{\prime}+b_{c}^{\prime})\\\
L_{c}&=-\sum_{c=1}^{|C|}c_{n+1}\log\hat{c}_{n+1}+(1-c_{n+1})\log(1-\hat{c}_{n+1})\end{split}$
(9)

The final loss function comprises the sum of the sequence loss and the
multimodal loss, as in Eq. 10. $\lambda$ is a hyperparameter that adjusts for
the influence of each loss function.

$L=L_{id}+\lambda\sum_{j\in{v,t,c}}L_{j}$ (10)

##  4 Experiments

###  4.1 Datasets and evaluation metrics

We used the Video Games and Clothing&Shoes datasets provided by Amazon to
evaluate the proposed model. The Clothing&Shoes dataset contained three
categories: Boys&Girls, Women, and Men. To consider user trends, we selected
only user transaction data from 2016 to 2017. We removed items that did not
have multimodal features such as images, descriptions, and categories. In
addition, users and items with fewer than 5 sequential interactions were
excluded. To use multimodal data, we utilized the pre-trained VGG and BERT to
obtain 4096 and 768 dimensional representations for image and text,
respectively, and categorical data were converted into a multi-label form.
Table 1 summarizes the dataset used in the experiment. In an experimental
evaluation method, the proposed model was evaluated using a leave-one-out
strategy as in previous studies (Kang and McAuley, 2018; Sun et al., 2019).
The last item in the user-purchased sequence is used as the test data, and the
item before the last one is used as the validation data. We used Recall@k and
NDCG@k as the evaluation metrics as shown in Eq. 11. Recall@k indicates
whether the user includes recommended item k among the items of interest.
NDCG@k applies weight values differently in the order of preference, assigning
higher weights to higher orders ($G_{i}$ and $G_{i}^{ideal}$ denote a relevant
item and an ideal sequence of relevant items, respectively). In our
experiments, the value of k was set to 10 and 20 to compare the results of
Recall@k and NDCG@k.

Table 1: Statistics of datasets for experiments.

Dataset | Video Games | Boys&Girls | Women | Men
---|---|---|---|---
#Users | 4,940 | 1,405 | 105,558 | 51,892
#Items | 11,002 | 15,603 | 122,701 | 56,325
#Avg. Actions/User | 7.4 | 6.8 | 7.7 | 7.4
#Avg. Actions/Item | 16.3 | 35 | 22.4 | 35.8
#Interactions | 36,752 | 9,587 | 810,245 | 383,148

$\begin{split}Recall@k=\frac{\\#\mathrm{~{}of~{}recommendations~{}that~{}are~{}relevant}}{\\#\mathrm{~{}of~{}all~{}possible~{}relevant~{}items}}\\\
NDCG@k=\frac{\mathrm{DCG@k}}{\mathrm{IDCG@k}}\\\
where~{}DCG@k=\sum_{i=1}^{k}\frac{G_{i}}{log_{2}(i+1)},IDCG@k=\sum_{i=1}^{k_{ideal}}\frac{G_{i}^{ideal}}{log_{2}(i+1)}\end{split}$
(11)

###  4.2 Hyperparameter settings

The proposed and baseline models used the RecBole (Zhao et al., 2021)
framework and summarized the results with five random seed changes. The
hyperparameters of the experimental model were trained with a learning rate of
1e-4 and 200 epochs using the Adam optimizer. The hidden size $d$ was set to
256 for the proposed as well as all the attention-based models. Similarly, the
attention layer $L$ was set to 4, and the attention head $h$ was set to 8.
Other hyperparameters use grid search to find the best settings for the model.
The searching space was: hidden size for multimodal $d^{\prime}$ $\in$ {32,
64, 128, 256}, $\lambda$ $\in$ {5, 10, 15, 20, 25}, $F$ $\in$ {Gate, Concat,
and Sum}.

###  4.3 Benchmark methods

The model shown below is the baseline models that were used for the
experiments. There are two types of models: basic sequential recommendation
methods that use only item sequences (1–4) and sequential recommendation
methods that use multimodal information (5 and 6). The hyperparameters of the
baseline model used a default setting and the attention-based model used the
same layer, head and hidden sizes as the proposed model.

  1. 1.

GRU4Rec (Hidasi et al., 2015): A sequential recommendation model using RNNs.

  2. 2.

Caser (Tang and Wang, 2018): A CNN-based sequential recommendation model that
uses horizontal and vertical convolution filters.

  3. 3.

SASRec (Kang and McAuley, 2018): An attention-based model that learns
sequential patterns using self-attention.

  4. 4.

BERT4Rec (Sun et al., 2019): A bidirectional attention-based model that masks
a user’s sequential patterns for learning.

  5. 5.

SASRecF: An extension of the SASRec model that fuses items and their
corresponding side information into embedding layers.

  6. 6.

FDSA (Zhang et al., 2019): A self-attention model that learns the sequential
patterns and side information of users through independent self-attention
blocks.

Table 2: Performance comparison between the benchmark model and the proposed
model.

Dataset | Metric | GRU4Rec | Caser | SASRec | BERT4Rec | SASRecF | FDSA | Proposal
---|---|---|---|---|---|---|---|---
Video Games | Recall@10 | 0.1120 | 0.0733 |  0.1689 | 0.0705 | 0.1513 | 0.1518 | 0.1714
Recall@20 | 0.1733 | 0.1136 |  0.2429 | 0.1263 | 0.2154 | 0.2120 | 0.2499
NDCG@10 | 0.0615 | 0.0406 | 0.0904 | 0.0344 | 0.0907 | 0.0959 |  0.0941
NDCG@20 | 0.0769 | 0.0507 | 0.1090 | 0.0483 | 0.1069 |  0.1111 | 0.1138
Boys&Girls | Recall@10 | 0.6859 | 0.7372 |  0.7877 | 0.6774 | 0.7791 | 0.7688 | 0.7906
Recall@20 | 0.7123 | 0.7675 | 0.8056 | 0.7251 |  0.8107 | 0.7893 | 0.8262
NDCG@10 | 0.6430 | 0.6702 | 0.7112 | 0.3292 | 0.7151 | 0.7092 |  0.7125
NDCG@20 | 0.6496 | 0.6774 | 0.7157 | 0.3412 | 0.7230 | 0.7144 |  0.7216
Women | Recall@10 | 0.2048 | 0.1817 |  0.2323 | 0.1806 | 0.2283 | 0.2306 | 0.2325
Recall@20 | 0.2109 | 0.1889 |  0.2419 | 0.1926 | 0.2370 | 0.2395 | 0.2422
NDCG@10 | 0.1932 | 0.1661 | 0.2162 | 0.0927 | 0.2151 | 0.2171 | 0.2171
NDCG@20 | 0.1948 | 0.1679 | 0.2186 | 0.0957 | 0.2173 |  0.2193 | 0.2196
Men | Recall@10 | 0.4561 | 0.4285 | 0.4857 | 0.4193 | 0.4863 |  0.4865 | 0.4884
Recall@20 | 0.4659 | 0.4398 |  0.4978 | 0.4354 | 0.4965 | 0.4966 | 0.5001
NDCG@10 | 0.4390 | 0.4060 | 0.4661 | 0.2291 | 0.4658 |  0.4677 | 0.4691
NDCG@20 | 0.4414 | 0.4088 | 0.4692 | 0.2332 | 0.4684 |  0.4703 | 0.4720

###  4.4 Experimental results

Table 2 compares the proposed model with various sequential recommendations on
four datasets collected from Amazon. Bold text in the table indicates the best
result of the comparative models, and underlined text denotes the second-best
result. First, the result for the GRU4Rec, Caser, SASRec, and BERT4Rec models,
which cannot receive multimodal information, show that the attention-based
SASRec perform best, which demonstrates that the attention mechanism is often
a useful structure for learning sequential patterns fo users. Attention-based
sequential recommendations SASRecF and FDSA that can receive multimodal
information often perform worse than SASRec which uses only user-item
information. In the case of SASRecF, which integrates multimodal data via an
embedding layer, the model receives diverse information and has difficulty
learning sequential patterns of the users. In contrast, the FDSA model learns
user sequence patterns and multimodal data via an independent self-attention
block. Neverthelss, FDSA consistently performs worse when it used complex
multimodal data than when it used only user sequences. In all the datasets,
the proposed model mostly outperforms the baselines in terms of the metrics.
Furthermore, various types of data contribute to different performance
improvements depending on the dataset size. For large-scale datasets such as
the Women and Men datasets, the use of Recall@20 improves the performance of
the proposed model by 0.12% and 0.46%, respectively, compared with the SASRec
model, which only considers sequences. For small-scale datasets, such as Video
Games and Boys & Girls the corresponding performance increased by 2.9% and
2.6%, respectively, indicating that multimodal data has a significant effect
on relatively small datasets.

###  4.5 Analysis on attention fusion functions

Table 3 compares the overall performance when applying different attention
fusion functions to the four experimental datasets. Bold in the table
indicates the best result among the models. Three types of attention fusion,
viz. Gate, Concat, and Sum, were used to aggregate sequence representation and
multimodal representation after performing query-key attention operations
independently. The results in Table 3 show small performance differences
between attention fusions. However, Concat and Gate perform poorly on most of
the datasets. the simple attention fusion function Sum performs best on the
four datasets without increasing the capacity of the model.

Table 3: Attention fusion function comparison experiment result. Dataset | Metric | Sum | Concat | Gate
---|---|---|---|---
Video Games | Recall@10 | 0.1714 | 0.1729 | 0.1725
Recall@20 | 0.2499 | 0.2479 | 0.2469
NDCG@10 | 0.0941 | 0.0929 | 0.0937
NDCG@20 | 0.1138 | 0.1117 | 0.1124
Boys & Girls | Recall@10 | 0.7883 | 0.7855 | 0.7910
Recall@20 | 0.8208 | 0.8180 | 0.8187
NDCG@10 | 0.7099 | 0.7074 | 0.7134
NDCG@20 | 0.7182 | 0.7156 | 0.7204
Women | Recall@10 | 0.2327 | 0.2314 | 0.2319
Recall@20 | 0.2421 | 0.2410 | 0.2414
NDCG@10 | 0.2163 | 0.2153 | 0.2151
NDCG@20 | 0.2186 | 0.2177 | 0.2175
Men | Recall@10 | 0.4880 | 0.4867 | 0.4875
Recall@20 | 0.4995 | 0.4978 | 0.4987
NDCG@10 | 0.4685 | 0.4682 | 0.4655
NDCG@20 | 0.4714 | 0.4710 | 0.4683

###  4.6 Analysis on utilizing multimodal data

Fig. 3 shows a comparative experiment using images, text, and categories to
determine the contribution of various multimodal data in the proposed model.
The results show that each dataset and evaluation metric yielded different
results. In Recall@20, item categories of the video game and women datasets
perform best among the multimodal data. For Boys & Girls and Men, images
containing visual information about clothing perform the best. In the case of
NDCG@20, the image data show the best results for the Women and Men datasets.
In addition, textual data performs best on the Video Games dataset, and
category data performs best on the Boys & Girls dataset. Both results indicate
that multimodal data have different levels of importance depending on the
characteristics of the dataset. However, on combining images, text, and
categories to form multimodal data, they complement each other, resulting in
improved model performance on most experimental datasets.

Figure 3: Comparison of multimodal data performance of four datasets.

###  4.7 Analysis on multi-task loss

To examine the effect of the multitask learning method applied to the proposed
model, comparative experiments were performed with and without multitask loss,
as shown in Table 4. In most cases, the use of multitask loss resulted in
better model performance because it can activate beneficial interactions of
multimodal data.

Table 4: Performance comparison with and without multi-task learning (MTL) applied to the proposed model: $\%$ Improved indicates the percentage of performance improved with MTL. Dataset | Metric | w/ MTL | w/o MTL |  $\%$ Improved
---|---|---|---|---
Video Games | Recall@10 | 0.1714 | 0.1732 | -1.05%
Recall@20 | 0.2499 | 0.2465 | 1.38%
NDCG@10 | 0.0941 | 0.0934 | 0.75%
NDCG@20 | 0.1138 | 0.1119 | 1.7%
Boys&Girls | Recall@10 | 0.7906 | 0.7926 | -0.25%
Recall@20 | 0.8262 | 0.8225 | 0.45%
NDCG@10 | 0.7125 | 0.7123 | 0.03%
NDCG@20 | 0.7216 | 0.7199 | 0.24%
Women | Recall@10 | 0.2325 | 0.2321 | 0.17%
Recall@20 | 0.2422 | 0.2410 | 0.5%
NDCG@10 | 0.2171 | 0.2157 | 0.65%
NDCG@20 | 0.2196 | 0.2179 | 0.78%
Men | Recall@10 | 0.4884 | 0.4873 | 0.23%
Recall@20 | 0.5001 | 0.4985 | 0.32%
NDCG@10 | 0.4691 | 0.4680 | 0.24%
NDCG@20 | 0.4720 | 0.4709 | 0.23%

###  4.8 Attention weight visualization

Fig. 4. shows the attention weight, which provides an interpretation of the
model. The Figure shows the results visualized through the last attention
layer of the video game dataset extracted from the randomly sampled attention
heads. Because the MAF focused on the interactions between the past purchased
items and multimodal features, we applied the masked attention operation. The
last $15^{th}$ time and position of the length of the entire sequence are
represented in the Figure; a darker color indicates a higher attention weight.
The item attention matrix and multimodal attention matrix represent the
different attention weights, which demonstrates that the sequence and
multimodal data have different impacts when the model performs operations. In
addition, the fusion attention matrix coordinates the contributions of
sequence and multimodal data and integrates them into significant patterns.

Figure 4: Video Games attention weight visualization.

##  5 Conclusion

In this study, we proposed an attention-based sequential recommendation system
model that utilizes multimodal data such as images, texts, and categories. The
image and text features of the items were extracted from pre-trained VGG and
BERT, and the category was converted into a multi-label form to add multimodal
information. In the proposed method, as each feature information is combined
with one multimodal representation, the attention operation is performed by
separating the item and multimodal representations. In addition, through
multi-task learning, both item representation and multimodal representation
were learned to refine multimodal information effectively. The experiments
were conducted on a total of four datasets: Video Games, Boys&Girls, Women,
and Men, which were collected from Amazon. The use of the multimodal
information in the sequential recommendation system results in a significant
performance improvement. Through experiments and attention weight
visualization, the MAF of the proposed method aids in complementing the ID
attention with multimodal information.

However, the applicability of the proposed method is limited by its relatively
high computational cost. Therefore, in future work, we plan to improve the
efficiency of the proposed method using a compressed attention operation to
implement an online real-time recommendation system. Furthermore, in this
study, experiments were conducted on only four categories of the Amazon
dataset. We plan to expand our experiments to include other datasets.

## CRediT authorship contribution statement

Hyuyngtaik Oh: Conceptualization, Software, Validation, Writing – Original
Draft. Wonkeun Jo: Methodology, Software, Writing – Original Draft. Dongil
Kim: Conceptualization, Methodology, Supervision, Writing – Review & Editing.

## Declaration of Competing Interest

The authors declare that they have no known competing financial interests or
personal relationships that could have appeared to influence the work reported
in this paper.

## Acknowledgments

This work was supported by the Ewha Womans University Research Grant of 2023.
This work was partially supported by Institute of Information $\&$
communications Technology Planning $\&$ Evaluation (IITP) grant funded by the
Korea government(MSIT) (No. RS-2022-00155966, Artificial Intelligence
Convergence Innovation Human Resources Development (Ewha Womans University)),
Research-centric Hospital R$\&$D grant funded by the Korean government (MOHW)
(No. HI21C0198), and Basic Science Research Program through the National
Research Foundation of Korea(NRF) funded by the Ministry of Education(No.
2022R1A6A3A13056750).

## References

  * Chen et al. (2012) Chen, M., Xu, Z., Weinberger, K., Sha, F., 2012\.  Marginalized denoising autoencoders for domain adaptation.  arXiv preprint arXiv:1206.4683 .
  * Chen et al. (2019) Chen, W., Huang, P., Xu, J., Guo, X., Guo, C., Sun, F., Li, C., Pfadler, A., Zhao, H., Zhao, B., 2019.  Pog: personalized outfit generation for fashion recommendation at alibaba ifashion, in: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pp. 2662–2670.
  * Cui et al. (2018) Cui, Q., Wu, S., Liu, Q., Zhong, W., Wang, L., 2018\.  Mv-rnn: A multi-view recurrent neural network for sequential recommendation.  IEEE Transactions on Knowledge and Data Engineering 32, 317–331.
  * Han et al. (2021) Han, T., Niu, S., Wang, P., 2021.  Multimodal-adaptive hierarchical network for multimedia sequential recommendation.  Pattern Recognition Letters 152, 10–17.
  * He and McAuley (2016) He, R., McAuley, J., 2016.  Vbpr: visual bayesian personalized ranking from implicit feedback, in: Proceedings of the AAAI conference on artificial intelligence.
  * Hidasi et al. (2015) Hidasi, B., Karatzoglou, A., Baltrunas, L., Tikk, D., 2015\.  Session-based recommendations with recurrent neural networks.  arXiv preprint arXiv:1511.06939 .
  * Hu et al. (2008) Hu, Y., Koren, Y., Volinsky, C., 2008.  Collaborative filtering for implicit feedback datasets, in: 2008 Eighth IEEE international conference on data mining, Ieee. pp. 263–272.
  * Iqbal et al. (2018) Iqbal, M., Kovac, A., Aryafar, K., 2018.  A multimodal recommender system for large-scale assortment generation in e-commerce.  arXiv preprint arXiv:1806.11226 .
  * Kang and McAuley (2018) Kang, W.C., McAuley, J., 2018\.  Self-attentive sequential recommendation, in: 2018 IEEE international conference on data mining (ICDM), IEEE. pp. 197–206.
  * Koren et al. (2009) Koren, Y., Bell, R., Volinsky, C., 2009.  Matrix factorization techniques for recommender systems.  Computer 42, 30–37.
  * Liu et al. (2021) Liu, C., Li, X., Cai, G., Dong, Z., Zhu, H., Shang, L., 2021.  Noninvasive self-attention for side information fusion in sequential recommendation, in: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 4249–4256.
  * Liu et al. (2019) Liu, T., Wang, Z., Tang, J., Yang, S., Huang, G.Y., Liu, Z., 2019.  Recommender systems with heterogeneous side information, in: The world wide web conference, pp. 3027–3033.
  * Mnih and Salakhutdinov (2007) Mnih, A., Salakhutdinov, R.R., 2007\.  Probabilistic matrix factorization.  Advances in neural information processing systems 20\.
  * Pan et al. (2008) Pan, R., Zhou, Y., Cao, B., Liu, N.N., Lukose, R., Scholz, M., Yang, Q., 2008\.  One-class collaborative filtering, in: 2008 Eighth IEEE International Conference on Data Mining, IEEE. pp. 502–511.
  * Sarwar et al. (2001) Sarwar, B., Karypis, G., Konstan, J., Riedl, J., 2001\.  Item-based collaborative filtering recommendation algorithms, in: Proceedings of the 10th international conference on World Wide Web, pp. 285–295.
  * Shao et al. (2021) Shao, B., Li, X., Bian, G., 2021.  A survey of research hotspots and frontier trends of recommendation systems from the perspective of knowledge graph.  Expert Systems with Applications 165, 113764.
  * Sun et al. (2019) Sun, F., Liu, J., Wu, J., Pei, C., Lin, X., Ou, W., Jiang, P., 2019.  Bert4rec: Sequential recommendation with bidirectional encoder representations from transformer, in: Proceedings of the 28th ACM international conference on information and knowledge management, pp. 1441–1450.
  * Tang and Wang (2018) Tang, J., Wang, K., 2018.  Personalized top-n sequential recommendation via convolutional sequence embedding, in: Proceedings of the eleventh ACM international conference on web search and data mining, pp. 565–573.
  * Vaswani et al. (2017) Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I., 2017\.  Attention is all you need.  Advances in neural information processing systems 30\.
  * Walek and Fojtik (2020) Walek, B., Fojtik, V., 2020\.  A hybrid recommender system for recommending relevant movies using an expert system.  Expert Systems with Applications 158, 113452.
  * Wei et al. (2019) Wei, Y., Wang, X., Nie, L., He, X., Hong, R., Chua, T.S., 2019.  Mmgcn: Multi-modal graph convolution network for personalized recommendation of micro-video, in: Proceedings of the 27th ACM International Conference on Multimedia, pp. 1437–1445.
  * Xie et al. (2022) Xie, Y., Zhou, P., Kim, S., 2022.  Decoupled side information fusion for sequential recommendation.  arXiv preprint arXiv:2204.11046 .
  * Yan and Zhang (2019) Yan, C., Zhang, Q., 2019.  Merging visual features and temporal dynamics in sequential recommendation.  Neurocomputing 362, 11–18.
  * Zhang et al. (2019) Zhang, T., Zhao, P., Liu, Y., Sheng, V.S., Xu, J., Wang, D., Liu, G., Zhou, X., 2019.  Feature-level deeper self-attention network for sequential recommendation., in: IJCAI, pp. 4320–4326.
  * Zhao et al. (2021) Zhao, W.X., Mu, S., Hou, Y., Lin, Z., Chen, Y., Pan, X., Li, K., Lu, Y., Wang, H., Tian, C., et al., 2021\.  Recbole: Towards a unified, comprehensive and efficient framework for recommendation algorithms, in: Proceedings of the 30th ACM International Conference on Information & Knowledge Management, pp. 4653–4664.
  * Zhou et al. (2020) Zhou, K., Wang, H., Zhao, W.X., Zhu, Y., Wang, S., Zhang, F., Wang, Z., Wen, J.R., 2020.  S3-rec: Self-supervised learning for sequential recommendation with mutual information maximization, in: Proceedings of the 29th ACM International Conference on Information & Knowledge Management, pp. 1893–1902.