# Automorphisms of valued Hahn groups

Salma Kuhlmann and Michele Serra  lifting property

###### Abstract.

Let $\Gamma$ be a totally ordered set and for all $\gamma\in\Gamma$ let
$A_{\gamma}$ be an abelian group. The support of an element
$a=(a_{\gamma})_{\gamma\in\Gamma}\in\prod_{\Gamma}A_{\gamma}$ is the set
$\mathrm{supp}(a)=\\{\gamma:a_{\gamma}\neq 0\\}.$ The Hahn sum
$\coprod_{\Gamma}A_{\gamma}$ (resp. Hahn product
$\mathbf{H}_{\Gamma}A_{\gamma}$) is the subgroup consisting of elements with
finite (resp. well ordered support). By a Hahn group we mean a group $G$ such
that $\coprod_{\Gamma}A_{\gamma}\leq G\leq\mathbf{H}_{\Gamma}A_{\gamma}$. If
for all $\gamma\in\Gamma$, $A_{\gamma}$ is an ordered abelian group we endow
$\mathbf{H}_{\Gamma}A_{\gamma}$ with the lexicographic order. Hahn groups,
endowed with the canonical valuation $v=v_{\min}$, play a fundamental role in
the classiﬁcation of valued groups. In this paper we study the group
$v\text{-}\mathrm{Aut}G$ (resp. $o\text{-}\mathrm{Aut}G$) of valuation (resp.
order) preserving automorphisms of a Hahn group $G$. Under the assumption that
$G$ satisﬁes the canonical lifting property we prove a structure theorem
decomposing $v\text{-}\mathrm{Aut}G$ into a semidirect product of two notable
subgroups: the subgroups of internal and external automorphisms. We
characterise a class of groups satisfying the aforementioned lifting property.
For some special cases we provide matrix descriptions.

##  1\. Introduction

In [Con58], Conrad describes the group of order preserving automorphisms of a
Hahn sum as a certain group of matrices. Droste and Göbel [DG97] extend this
approach to balanced Hahn sum (see Definition 2.1). Hofberger [Hof91]
describes the group of automorphisms of Hahn fields as a semidirect product of
so-called internal and external automorphisms. Building up on Hofberger’s work
we completed the description of the group of valuation preserving
automorphisms of Hahn fields [KS22].

In this paper we extend the approach of [KS22] to the study of
$v\text{-}\operatorname{Aut}G$ and $o\text{-}\operatorname{Aut}G$ for a Hahn
group $G$. We introduce the notion of (canonical) lifting property for a
general Hahn group and, based on [KKS22], we prove that this is satisfied by a
relatively large class of Hahn groups. We then decompose
$v\text{-}\operatorname{Aut}G$ (resp. $o\text{-}\operatorname{Aut}G$) for Hahn
groups satisfying the canonical lifting property. This extends the main result
obtained in [DG97] (for the special case of balanced Hahn sums), to this large
class of Hahn groups.

The paper is organised as follows. In Section 2 we provide the definitions and
notations that will be needed. In Section 3 we introduce the important notion
of . Subsection 3.1 focusses on order preserving automorphisms and on the
lifting property with respect to the rank (Definition 3.2). Theorem 3.7
characterises automorphisms of the rank that lift by means of principal convex
subgroups. We use this result to show that the lifting property with respect
to the rank is not preserved by taking lexicographic sums (Example 3.8). In
Subsection 3.2 we study valuation preserving automorphisms and we introduce
the lifting property with respect to the skeleton (Definition 3.12). The main
result of the section is Theorem 3.15, which provides a semidirect product
decomposition of the $v\text{-}\operatorname{Aut}G$ of a Hahn group $G$
satisfying the lifting property w.r.t. the skeleton. In Subsection 3.3, based
on [KKS22], we introduce Rayner groups (Definition 3.18) and characterise
those that satisfy the lifting property w.r.t. the skeleton (Theorem 3.19).
Section 4 focusses on Hahn sums, providing a description of the group of order
preserving automorphism as a group of matrices, generalising results of
[Con58] and [DG97]. Theorem 4.5 gives a decomposition of the subgroup of order
preserving automorphisms that induce the identity on the rank while Theorem
4.6 provides the matrix group representation.

In [KS23] we further investigate the subgroups of internal and external
automorphisms of $o\text{-}\operatorname{Aut}G$ of an ordered Hahn group $G$,
providing a finer decomposition of those groups. This analysis is of
particular importance, since $o\text{-}\operatorname{Aut}G$ appears as a
factor of the automorphism group $v\text{-}\operatorname{Aut}K$ of a valued
Hahn field $K$ [KS22]. Furthermore we focus on the subgroup of strongly
additive (i.e., commuting with formal infinite sums) automorphisms. This
latter subgroup is of particular importance for the study of derivations on
valued Hahn fields (cf. [Bag+23]).

##  2\. Definitions and notation

Let $\Gamma$ be a (totally) ordered set and for all $\gamma\in\Gamma$ let
$A_{\gamma}$ be an abelian group. For an element
$a=(a_{\gamma})_{\gamma\in\Gamma}\in\prod_{\gamma\in\Gamma}A_{\gamma}$ we
define the _support_ of $a$ to be the set
$\operatorname*{supp}(a)=\\{\gamma\in\Gamma:a_{\gamma}\neq 0\\}$.

######  Definition 2.1 (Hahn group).

  1. (i)

The _Hahn sum_ of the $A_{\gamma}$’s, denoted by
$\coprod_{\gamma\in\Gamma}A_{\gamma}$ is the set of elements of the full
cartesian product with finite support:

$\coprod_{\gamma\in\Gamma}A_{\gamma}=\left\\{a\in\prod_{\gamma\in\Gamma}A_{\gamma}:|\operatorname*{supp}(a)|<\infty\right\\}.$

  2. (ii)

The _Hahn product_ of the $A_{\gamma}$’s, denoted by
$\operatorname*{\mathbf{H}}_{\gamma\in\Gamma}A_{\gamma}$ is the set of
elements of the full direct product with well ordered support:

$\operatorname*{\mathbf{H}}_{\gamma\in\Gamma}A_{\gamma}=\left\\{a\in\prod_{\gamma\in\Gamma}A_{\gamma}:\operatorname*{supp}(a)\text{
is well ordered}\right\\}.$

The Hahn sum is a subgroup of the Hahn product.

  3. (iii)

A group $G$ comprised between them will be called a _Hahn group_ and the pair
$S(G):=[\Gamma;\\{A_{\gamma}:\gamma\in\Gamma\\}]$ will be called the
_skeleton_ of $G$.

  4. (iv)

If $G$ is balanced, with $A_{\gamma}\simeq A$ for all $\gamma\in\Gamma$, we
will write $S(G)=[\Gamma;A]$.

  5. (v)

In the sequel we will also refer to $\Gamma$ as the _rank of_ $G$ (sometimes
denoted by $\operatorname{rk}G$) and to the groups $A_{\gamma}$ as the
_components of_ $G$.

  6. (vi)

If for all $\gamma\in\Gamma$ we have $A_{\gamma}\simeq A$ for a given abelian
group $A$ we say that $G$ is a _balanced Hahn group_.

######  Notation 2.2.

We will use the following notation for the elements of $G$:

$\sum_{\gamma\in\Gamma}a_{\gamma}\mathbbm{1}_{\gamma}:=(a_{\gamma})_{\gamma\in\Gamma}.$

######  Definition 2.3.

Let $G_{1},G_{2}$ be Hahn groups and let
$S(G_{i})=[\Gamma_{i}:\\{A^{i}_{\gamma}:\gamma\in\Gamma_{i}\\}],\ i=1,2$ be
the respective skeletons. An _isomorphism_ $\tau:S(G_{1})\rightarrow S(G_{2})$
consists of an isomorphism $\tau_{\Gamma}:\Gamma_{1}\rightarrow\Gamma_{2}$ of
ordered sets, and for all $\gamma\in\Gamma_{1}$ an isomorphism
$\tau_{\gamma}\colon A^{1}_{\gamma}\rightarrow A^{2}_{\tau_{\Gamma}(\gamma)}$
of ordered groups. We will also write
$\tau=[\tau_{\Gamma};\\{\tau_{\gamma}:\gamma\in\Gamma_{1}\\}].$
An automorphism of $S(G)$, for a Hahn group $G$, is hence defined in the
obvious way and we will denote by $\operatorname{Aut}S(G)$ the group of
automorphisms of $S(G)$ under composition.

Notice that, an automorphism of the skeleton is, in particular, an
automorphism of the rank. However, there can be automorphism of the rank that
are not part of any automorphism of the skeleton:

######  Example 2.4.

Let $G=\coprod_{\mathbb{Z}}A_{n}$ where $A_{2n}=\mathbb{Q}$ and
$A_{2n+1}=\mathbb{R}$ for all $n\in\mathbb{Z}$. Then
$\sigma_{\mathbb{Z}}\colon n\mapsto n+1$ is an automorphism of the chain
$\mathbb{Z}$ that cannot appear in any automorphism of the skeleton, as there
is no isomorphism between $A_{n}$ and $A_{n+1}$, for any $n$.

For the remaining of this section, let us fix a Hahn group $G$ with skeleton
$S(G)=[\Gamma;\\{A_{\gamma}:\gamma\in\Gamma\\}]$.

######  Definition 2.5.

Since all elements of $G$ have well ordered support, we can define a valuation
$v_{\min}$ on $G$ with value set $\Gamma$ by

$v_{\min}\colon G\longrightarrow\Gamma\cup\\{\infty\\},\quad
v_{\min}(g)=\begin{cases}\infty&\text{if }g=0\\\
\min\operatorname*{supp}(g)&\text{if }g\neq 0\end{cases}$

We will call $v_{\min}$ the _canonical valuation on_ $G$ and if the context is
clear we will simply denote it by $v$.

Unless otherwise stated we will always consider $G$ endowed with its canonical
valuation $v$.

######  Definition 2.6.

An automorphism $\sigma$ of $G$ is said to be _valuation preserving_ if there
exists a chain automorphism $\sigma_{\Gamma}$ of $\Gamma$ such that, for all
$g\in G$, we have $\sigma_{\Gamma}(v(g))=v(\sigma(g))$.
The valuation preserving automorphisms for ma group under composition that we
will denote by $v\text{-}\operatorname{Aut}G$.

######  Remark 2.7 ([Kuh00, Lemma 0.1]).

An automorphism $\sigma\in\operatorname{Aut}G$ is valuation preserving if and
only if the map $\sigma_{\Gamma}\colon\Gamma\rightarrow\Gamma,\ v(x)\mapsto
v(\sigma(x))$ is a well defined automorphism of chains. We will say that
$\sigma_{\Gamma}$ is the chain automorphism _induced on $\Gamma$ by
$\sigma$_.∎

######  Definition 2.8.

More generally, let $G_{1},G_{2}$ be Hahn groups with skeletons
$S(G_{i})=[\Gamma_{i}:\\{A^{i}_{\gamma}:\gamma\in\Gamma_{i}\\}],\ i=1,2$ and
let $v_{1},v_{2}$ be the canonical valuations on $G_{1}$ and $G_{2}$,
respectively. An isomorphism $\rho\colon G_{1}\to G_{2}$ is said to be
_valuation preserving_ if there exists an isomorphism of chains
$\rho_{\Gamma}\colon\Gamma_{1}\to\Gamma_{2}$ such that, for all $g\in G_{1}$
we have $\rho_{\Gamma}(v_{1}(g))=v_{2}(\rho(g))$.

######  Lemma 2.9 ([Kuh00, Lemma 0.2]).

Let us keep the notation of Definition 2.8. A valuation preserving isomorphism
$\sigma\colon G_{1}\rightarrow G_{2}$ of Hahn groups naturally induces an
isomorphism of the skeletons
$[\sigma_{\Gamma};\\{\sigma_{\gamma}:\gamma\in\Gamma_{1}\\}]\colon
S(G_{1})\rightarrow S(G_{2})$ as follows:

  * •

for any $\gamma\in\Gamma_{1}$ choose $0\neq a\in A_{\gamma}$ and set
$\sigma_{\Gamma}(\gamma):=v_{2}(\sigma(a\mathbbm{1}_{\gamma}))$ (the fact that
$\sigma$ is valuation preserving implies that
$v_{2}(\sigma(a\mathbbm{1}_{\gamma}))$ does not depend on the choice of $a$);

  * •

for any $\gamma\in\Gamma_{1}$ and each $g_{\gamma}\in A^{1}_{\gamma}$ set
$\sigma_{\gamma}(g_{\gamma})=\sigma(g_{\gamma}\mathbbm{1}_{\gamma})_{\sigma_{\Gamma}(\gamma)}$.

In particular, a valuation preserving automorphism $\sigma\in
v\text{-}\operatorname{Aut}G$ induces an automorphism
$\sigma_{S}=[\sigma_{\Gamma};\\{\sigma_{\gamma}:\gamma\in\Gamma\\}]\in\operatorname{Aut}S(G)$
of the skeleton where $\sigma_{\Gamma}\in o\text{-}\operatorname{Aut}\Gamma$
is defined as in Remark 2.7 and $\sigma_{\gamma}\colon A_{\gamma}\to
A_{\sigma_{\Gamma}(\gamma)}$ is given by
$\sigma_{\gamma}(a)=(\sigma(a\mathbbm{1}_{\gamma}))_{\sigma_{\Gamma}(\gamma)}$.
∎

######  Definition 2.10.

If all the components $A_{\gamma}$ are ordered abelian groups and we denote by
$<_{\gamma}$ the order relation on $A_{\gamma}$, then we can order $G$
lexicographically by setting, for all $g\in G$,

$g>_{\operatorname{lex}}0\iff g_{v(g)}>_{v(g)}0.$

An automorphism $\sigma$ is said to be _order preserving_ if for all $g\in G$
we have $g>0\Rightarrow\sigma(g)>0$. The order preserving automorphisms also
form a group under composition which will be denoted by
$o\text{-}\operatorname{Aut}G$.

######  Remark 2.11.

Assume, moreover, that all the components $A_{\gamma}$ be ordered archimedean
groups. Then an order preserving automorphism of $G$ is automatically
valuation preserving. So we have $o\text{-}\operatorname{Aut}G\leq
v\text{-}\operatorname{Aut}G\leq\operatorname{Aut}G.$∎

##  3\. The lifting property

Let $G$ be a Hahn group with skeleton
$S(G)=[\Gamma;\\{A_{\gamma}:\gamma\in\Gamma\\}]$. In Section 2 we saw that a
valuation preserving automorphism $\sigma\in v\text{-}\operatorname{Aut}G$
induces an automorphism $\sigma_{S}\in\operatorname{Aut}S(G)$ of the skeleton
(Lemma 2.9) and, in particular, an automorphism $\sigma_{\Gamma}\in
o\text{-}\operatorname{Aut}\Gamma$ of the rank (Remark 2.7). In this section
we will study under what conditions, given an automorphism of the skeleton
(resp. rank) we can find an automorphism of the group that induces it. In
Section 3.1 we will study automorphisms of the chain $\Gamma$ that lift to
_order preserving_ automorphisms of $G$ and in Section 3.2 automorphisms of
$S(G)$ that lift to _valuation preserving_ automorphisms of $G$.

###  3.1. Lifting from the rank to $o\text{-}\operatorname{Aut}G$

Let $G$ be a Hahn group with skeleton
$S(G)=[\Gamma;\\{A_{\gamma}:\gamma\in\Gamma\\}]$ and let
${\mathbb{G}}=\operatorname*{\mathbf{H}}_{\Gamma}A_{\gamma}$ be the
corresponding Hahn product. In this section we will assume that all the
components $A_{\gamma}$ of the skeleton of $G$ be archimedean ordered abelian
groups and we will consider the induced lexicographic order on $G$ (Definition
2.10). Let $\sigma\in o\text{-}\operatorname{Aut}G$. Then, by Remark 2.11,
$\sigma\in v\text{-}\operatorname{Aut}G$ and therefore, it induces an
automorphism $\sigma_{\Gamma}\in o\text{-}\operatorname{Aut}\Gamma$ given by
$\sigma_{\Gamma}(v(g))=v(\sigma(g))$ (Remark 2.7). This gives rise to a group
homomorphism

$\Phi_{\Gamma}:o\text{-}\operatorname{Aut}G\longrightarrow
o\text{-}\operatorname{Aut}\Gamma,\ \sigma\longmapsto\sigma_{\Gamma}.$ (1)

It is straightforward to verify that this is a group homomorphism.

######  Definition 3.1.

The kernel $\ker\Phi_{\Gamma}$ of the map (1) is a normal subgroup of
$o\text{-}\operatorname{Aut}G$ that we call the _subgroup of internal
o-automorphisms_ of $G$. We will denote it by
$\operatorname{Int}o\text{-}\operatorname{Aut}G$.

######  Definition 3.2.

If the map $\Phi_{\Gamma}$ defined in (1) admits a section, i.e., an injective
group homomorphism $\Psi_{\Gamma}\colon
o\text{-}\operatorname{Aut}\Gamma\hookrightarrow o\text{-}\operatorname{Aut}G$
such that
$\Phi_{\Gamma}\circ\Psi_{\Gamma}=\operatorname{id}_{o\text{-}\operatorname{Aut}\Gamma}$,
we say that $G$ has the _lifting property with respect to $\Psi_{\Gamma}$_.
For an automorphism $\tau\in o\text{-}\operatorname{Aut}\Gamma$ we call
$\Psi_{\Gamma}(\tau)$ the _lift of $\tau$ with respect to $\Psi_{\Gamma}$_.

######  Definition 3.3.

Assume $G$ has the lifting property with respect to a fixed section
$\Psi_{\Gamma}$ of $\Phi_{\Gamma}$. The subgroup
$\Psi_{\Gamma}(o\text{-}\operatorname{Aut}G)$ of
$o\text{-}\operatorname{Aut}G$ will be called the subgroup of
$\Psi_{\Gamma}$_-external automorphisms of $G$_ and denoted by
${\Psi_{\Gamma}}\text{-}\operatorname{Ext}o\text{-}\operatorname{Aut}G$.

######  Definition 3.4.

Assume $G$ is balanced, say $A_{\gamma}=A$ for all $\gamma$. Then for the Hahn
product ${\mathbb{G}}=\operatorname*{\mathbf{H}}_{\Gamma}A$ there exists
always the _canonical section_

$\Psi^{c}_{\Gamma}\colon o\text{-}\operatorname{Aut}\Gamma\to
o\text{-}\operatorname{Aut}{\mathbb{G}},\ \tau\longmapsto\tilde{\tau}$

where $\tilde{\tau}$ is defined by

$\tilde{\tau}\left(\sum_{\gamma}g_{\gamma}\mathbbm{1}_{\gamma}\right)=\sum_{\gamma}g_{\gamma}\mathbbm{1}_{\tau(\gamma)}.$

We call $\tilde{\tau}$ the _canonical lift of $\tau$ to ${\mathbb{G}}$._

If, moreover, $\tilde{\tau}(G)=G$ we say that $\tau$ _lifts canonically_ to an
automorphism of $G$ and call $\tilde{\tau}|_{G}$ the _canonical lift_ of
$\tau$ to $G$. If all automorphisms of $\Gamma$ lift canonically we say that
$G$ has the _canonical lifting property with respect to_ $\Gamma$.

Now we give a characterisation of the automorphisms of the rank $\Gamma$ that
lift to an order-preserving automorphism or $G$ by means of its principal
convex subgroups (Definition 3.5). We do this in Theorem 3.7, after proving a
preliminary lemma.

######  Definition 3.5.

For an element $\gamma\in\Gamma$, the _principal convex subgroup of $G$
associated to $\gamma$_ is the subgroup $C_{\gamma}:=\\{x\in
G:v(x)\geq\gamma\\}$.

######  Lemma 3.6.

Let $\sigma\in v\text{-}\operatorname{Aut}G$ and let $g\in G$ and
$C_{g}:=\\{x\in G:\ v(x)\geq v(g)\\}$. Then we have:

$\sigma(C_{g})=C_{\sigma(g)}=\\{x:\ v(x)\geq v(\sigma(g))\\}.$

###### Proof.

Let $\sigma_{\Gamma}$ be the automorphism of the chain $\Gamma$ induced by
$\sigma$. Let $x\in C_{g}$. Then

$v(x)\geq v(g)\Rightarrow
v(\sigma(x))=\sigma_{\Gamma}(v(x))\geq\sigma_{\Gamma}(v(g))=v(\sigma(g))$

so $\sigma(x)\in C_{\sigma(g)}$, that is $\sigma(C_{g})\subseteq
C_{\sigma(g)}$.

Vice versa, let $x\in C_{\sigma(g)}$. Then

$v(x)\geq v(\sigma(g))\Rightarrow
v(\sigma^{-1}(x))=\tilde{\sigma}^{-1}(v(x))\geq\sigma_{\Gamma}^{-1}(v(\sigma(g)))=v(\sigma^{-1}(\sigma(g)))=v(g)$

because $\sigma_{\Gamma}^{-1}$ is also an automorphism of $\Gamma$. So
$\sigma^{-1}(x)\in C_{g}$, that is $x\in\sigma(C_{g})$. Hence
$\sigma(C_{g})\supseteq C_{\sigma(g)}$, which completes the proof. ∎

######  Theorem 3.7.

Let $\tau\in o\text{-}\operatorname{Aut}\Gamma$. Then $\tau$ lifts to an
automorphism $\sigma\in o\text{-}\operatorname{Aut}G$ if and only if, for all
$\gamma\in\Gamma$ there is an isomorphism

$\sigma_{\gamma}\colon
C_{\gamma}\overset{\sim}{\longrightarrow}C_{\tau(\gamma)}$

such that

$\gamma>\delta\Rightarrow\sigma_{\delta}|_{C_{\gamma}}=\sigma_{\gamma}.$ (2)

###### Proof.

Assume that $\tau$ lifts to $\sigma\in o\text{-}\operatorname{Aut}G$. Let
$\gamma\in\Gamma$ and choose $g\in G$ such that $v(g)=\gamma$. Recall that, by
definition, $v(\sigma(g))=\tau(v(g))$. Then Lemma 3.6 gives

$C_{\gamma}=C_{g}=\\{x:\ v(x)\geq\gamma=v(g)\\}\simeq C_{\sigma(g)}=\\{x:\
v(x)\geq\tilde{\sigma}(v(g))=\tau(\gamma)\\}=C_{\tau(\gamma)}.$

where the isomorphism is simply given by $\sigma$, which is order preserving,
so condition (2) is satisfied.

Vice versa, suppose that for all $\gamma\in\Gamma$ there is an isomorphism

$\sigma_{\gamma}\colon
C_{\gamma}\overset{\sim}{\longrightarrow}C_{\tau(\gamma)}$

such that
$\gamma>\delta\Rightarrow\sigma_{\gamma}(C_{\gamma})\subset\sigma_{\delta}(C_{\delta})$.
We want to construct a lift $\sigma\in o\text{-}\operatorname{Aut}G$ of
$\tau$. We define it as follows: for $x\in G$ with $v(x)=\gamma$, set
$\sigma(x):=\sigma_{\gamma}(x)$. First let us show that $\sigma$ is a well
defined group morphism. Let $x,y\in G$. We have two cases:
Case 1. $v(x)\neq v(y)$, and we can assume $v(x)<v(y)$. Then
$v(x+y)=v(x)=:\gamma$. So we have

$\sigma(x-y)=\sigma_{\gamma}(x-y)=\sigma_{\gamma}(x)-\sigma_{\gamma}(y)=\sigma(x)-\sigma_{v(y)}(y)=\sigma(x)-\sigma(y)$

where in the second-last equality we used condition (2).
Case 2. $v(x)=v(y)=\delta$. Then $v(x+y)=:\gamma\geq\delta$. Then

$\sigma(x-y)=\sigma_{\gamma}(x-y)=\sigma_{\delta}(x-y)=\sigma_{\delta}(x)-\sigma_{\delta}(y)=\sigma(x)-\sigma(y).$

Again, for the second equality we used condition (2).

To show that $\sigma$ is an automorphism, we define its inverse. Similarly to
what we did to define $\sigma$ we define $\sigma^{-1}$ as follows: for $x\in
G$ with $v(x)=\tau(\gamma)$ we set $\sigma^{-1}(x)=\sigma_{\gamma}^{-1}(x)$.
Checking that this is a well defined group morphism is done the same way as it
was done for $\sigma$. To see that this is indeed the inverse of $\sigma$, let
$x\in G$ with $v(x)=\gamma$. Recall that $\sigma_{\gamma}(x)\in
C_{\tau(\gamma)}$ and $\sigma_{\gamma}^{-1}:C_{\tau(\gamma)}\rightarrow
C_{\gamma}$. So

$\sigma^{-1}(h(x))=\sigma^{-1}(\sigma_{\gamma}(x))=\sigma_{\gamma}^{-1}(\sigma_{\gamma}(x))=x$

and similarly $\sigma(\sigma^{-1}(x))=x$. So $\sigma$ is an automorphism of
$G$, which completes the proof. ∎

The following example shows that, in general, the lifting property is not
preserved after taking lexicographic sums.

######  Example 3.8.

Let $\Gamma=\mathbb{Q}$ and
$G=(\operatorname*{\mathbf{H}}_{q<0}\mathbb{Q})\amalg(\coprod_{q\geq
0}\mathbb{Q})$. So $\Gamma=\operatorname{rk}G=\mathbb{Q}$. Consider the
automorphism of the chain $\mathbb{Q}$ defined by $\tilde{h}(q)=q+2$ and
assume that $\tilde{h}$ lifts to an automorphism $h$ of $G$. Let $g\in G$ be
such that $v(g)=-1/2$. So $v(h(g))=\tilde{h}(-\frac{1}{2})=\frac{3}{2}>1$. By
Lemma 3.6 we have $C_{g}\simeq C_{h(g)}$. Now $C_{h(g)}=\\{x:\
v(x)\geq\frac{3}{2}\\}\subseteq\coprod_{q>0}\mathbb{Q}$ is countable. But
$C_{g}=\\{x:\ v(x)\geq-\frac{1}{2}\\}$ is uncountable. To prove this, we show
that the uncountable set $\\{0,1\\}^{\mathbb{N}}$ of functions
$f:\mathbb{N}\rightarrow\\{0,1\\}$ can be embedded into $C_{g}$. We construct
the embedding as follows: to each function $f:\mathbb{N}\rightarrow\\{0,1\\}$
we associate the element
$f^{*}=(f^{*}_{q})_{q<0}\in\operatorname*{\mathbf{H}}_{q<0}\mathbb{Q}$ defined
by

$f^{*}_{q}=\begin{cases}f(n)\text{ if }q=-\frac{1}{2+n}\\\ 0\text{
otherwise.}\end{cases}$

Notice that, for all $f\in\\{0,1\\}^{\mathbb{N}}$, the element $f^{*}$ belongs
to $C_{g}$: indeed we have
$\operatorname*{supp}(f^{*})\subseteq\\{-\frac{1}{2+n}|n\in\mathbb{N}\\}$
which is well ordered, so $f^{*}\in\operatorname*{\mathbf{H}}_{q<0}\mathbb{Q}$
and $v(f^{*})\geq-\frac{1}{2}$. Moreover, the map $f\mapsto f^{*}$ is
injective, for if $f\neq g$ then there exists $n\in\mathbb{N}$ such that

$f^{*}_{-\frac{1}{2+n}}=f(n)\neq g(n)=g^{*}_{-\frac{1}{2+n}}.$

∎

Last example shows that, even if two Hahn groups have the lifting property,
after we take the lexicographic sum not all automorphism of the rank will
lift. The next proposition characterises those that do.

######  Proposition 3.9.

Let $G_{1},G_{2}$ be two groups with the lifting property (with respect to the
rank) and consider $G=G_{1}\amalg G_{2}$. Then
$\operatorname{rk}G=\operatorname{rk}G_{1}+\operatorname{rk}G_{2}$. Let $\tau$
be an automorphism of $\operatorname{rk}G$ such that
$\tau|_{\operatorname{rk}G_{i}}\in
o\text{-}\operatorname{Aut}(\operatorname{rk}G_{i})$ for $i=1,2$. Then $\tau$
lifts to an automorphism of $G$.

###### Proof.

Let $\Gamma:=\operatorname{rk}G$ and $\Gamma_{i}:=\operatorname{rk}G_{i}$ for
$i=1,2$. Let $\tau\in o\text{-}\operatorname{Aut}\Gamma$ be such that
$\tau_{i}:=\tau|_{\Gamma_{i}}\in o\text{-}\operatorname{Aut}\Gamma_{i}$ for
$i=1,2$. Since both $G_{i}$’s have the lifting property, let $\sigma_{i}\in
v\text{-}\operatorname{Aut}G_{i}$ be a lift of $\tau_{i}$ and define
$\sigma\colon G\rightarrow G$ through
$\sigma(g_{1}\mathbbm{1}_{1}+g_{2}\mathbbm{1}_{2}):=\sigma_{1}(g_{1})\mathbbm{1}_{1}+\sigma_{2}(g_{2})\mathbbm{1}_{2}$.
It is straightforward to verify that this is an automorphism of $G$. ∎

###  3.2. Lifting from the skeleton to $v\text{-}\operatorname{Aut}G$

In this section we consider the _lifting property with respect to the
skeleton_ , and ask under what conditions an automorphism of the skeleton of a
Hahn group lifts to an automorphism of the group. Recall that (Lemma 2.9) a
valuation preserving automorphism $\sigma\in v\text{-}\operatorname{Aut}G$
induces an automorphism $\sigma_{S}\in\operatorname{Aut}S(G)$ of the skeleton.
This gives rise to a map, that we will denote by $\Phi_{S}$, in analogy to the
one in (1):

$\Phi_{S}\colon
v\text{-}\operatorname{Aut}G\longrightarrow\operatorname{Aut}S(G),\
\sigma\longmapsto[\sigma_{\Gamma};\\{\sigma_{\gamma}:\gamma\in\Gamma\\}].$ (3)

It is straightforward to verify that $\Phi_{S}$ is a group homomorphism.

######  Definition 3.10.

The kernel of the map $\Phi_{S}$ defined in (3) is a normal subgroup of
$v\text{-}\operatorname{Aut}G$ called _the group of internal v-automorphisms_
and denoted by $\operatorname{Int}v\text{-}\operatorname{Aut}G$.

If the context is clear we will drop the “$v$” from the notation and
terminology.

######  Remark 3.11.

An automorphism $\sigma\in v\text{-}\operatorname{Aut}G$ is internal if and
only if, for all $g\in G$, we have

$v(g)=v(\sigma(g))\quad\text{ and }\quad g_{v(g)}=\sigma(g)_{v(g)}.$ (4)

Indeed, conditions (4) are equivalent to
$\Phi(\sigma)=\operatorname{id}_{S(G)}$. ∎

######  Definition 3.12.

If the map $\Phi_{S}$ defined in (3) admits a section, i.e., an injective
group homomorphism $\Psi_{S}\colon\operatorname{Aut}S(G)\hookrightarrow
v\text{-}\operatorname{Aut}G$ such that
$\Phi_{S}\circ\Psi_{S}=\operatorname{id}_{\operatorname{Aut}S(G)}$, we say
that $G$ has the _lifting property with respect to $\Psi_{S}$_. For
$\tau\in\operatorname{Aut}(S(G))$, the automorphism $\Psi_{S}(\tau)\in
v\text{-}\operatorname{Aut}G$ is called _the lift of $\tau$ with respect to
$\Psi_{S}$_.

######  Definition 3.13.

Assume $G$ has the lifting property with respect to a fixed section $\Psi_{S}$
of $\Phi_{S}$. The subgroup $\Psi_{S}(\operatorname{Aut}S(G))$ of
$v\text{-}\operatorname{Aut}G$ will be called the subgroup of
$\Psi_{S}$_-external automorphisms of $G$_ and denoted by
${\Psi_{S}}\text{-}\operatorname{Ext}\operatorname{Aut}G$. If no confusion can
arise we will just write $\operatorname{Ext}\operatorname{Aut}G$.

######  Remark 3.14.

If $G$ has the lifting property with respect to a given section $\Psi_{S}$
then the homomorphism $\Phi_{S}$ defined in (3) is surjective and every
$\tau\in\operatorname{Aut}S(G)$ lifts to an automorphism $\sigma\in
v\text{-}\operatorname{Aut}G$. The first isomorphism theorem thus yields

$\operatorname{Aut}S(G)\simeq\frac{v\text{-}\operatorname{Aut}G}{\operatorname{Int}\operatorname{Aut}G}.$

∎

In [KS22, Section 3] we provide a decomposition of the group of valuation
preserving automorphisms of a Hahn field with the lifting property into the
semidirect product of its subgroups of internal and external automorphisms.
Here we generalise this result to Hahn groups with the lifting property.

######  Theorem 3.15.

Let $G$ be a Hahn group with the lifting property with respect to a given
section $\Psi_{S}$ of $\Phi_{S}$. Then we can decompose
$v\text{-}\operatorname{Aut}G$ into the (inner) semidirect product of its
subgroups of internal and external automorphisms:

$v\text{-}\operatorname{Aut}G=\operatorname{Int}\operatorname{Aut}G\rtimes\operatorname{Ext}\operatorname{Aut}G.$

###### Proof.

Let us consider the sequence

$\textstyle{\operatorname{Int}\operatorname{Aut}G\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{\iota}$$\textstyle{v\text{-}\operatorname{Aut}G\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{\Phi_{S}}$$\textstyle{\operatorname{Aut}S(G)\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{\Psi_{S}}$

where $\iota$ is the canonical embedding. By definition of
$\operatorname{Int}\operatorname{Aut}G$, we have
$\operatorname{im}\iota=\ker\Phi$, so the sequence is exact. Therefore (see
[Con, Theorem 3.3]) we have

$v\text{-}\operatorname{Aut}G=\operatorname{im}\iota\rtimes\operatorname{im}\Psi=\ker\Phi\rtimes\operatorname{im}\Psi=\operatorname{Int}\operatorname{Aut}G\rtimes\operatorname{Ext}\operatorname{Aut}G.$

∎

Analogously to the previous section, we can define the canonical lifting
property with respect to the skeleton:

######  Definition 3.16.

For the Hahn product
${\mathbb{G}}=\operatorname*{\mathbf{H}}_{\Gamma}A_{\gamma}$ there exists
always the _canonical section_

$\Psi^{c}_{S}\colon\operatorname{Aut}S(G)\to
v\text{-}\operatorname{Aut}{\mathbb{G}},\
\tau=[\tau_{\Gamma};\\{\tau_{\gamma}:\gamma\in\Gamma\\}]\longmapsto\tilde{\tau}$

where $\tilde{\tau}$ is defined by

$\tilde{\tau}\left(\sum_{\gamma}g_{\gamma}\mathbbm{1}_{\gamma}\right)=\sum_{\gamma}\tau_{\gamma}(g_{\gamma})\mathbbm{1}_{\tau_{\Gamma}(\gamma)}.$

We call $\tilde{\tau}$ the _canonical lift of $\tau$ to ${\mathbb{G}}$._

If, moreover, $\tilde{\tau}(G)=G$ we say that $\tau$ _lifts canonically_ to an
automorphism of $G$ and call $\tilde{\tau}|_{G}$ the _canonical lift of $\tau$
to $G$_. If all automorphisms of $S(G)$ lift canonically we say that $G$ has
the _canonical lifting property with respect to_ $S(G)$.

In the next section we introduce an interesting class fo Hahn groups, that we
will call _Rayner groups_ (Definition 3.18) in which we can characterise those
groups that have the canonical lifting property.

###  3.3. Rayner groups

A family ${\mathcal{F}}$ of subsets of $\Gamma$ is said to be a _group family_
(cf. [Ray68, Paragraph 2]) if the following properties are satisfied:

  1. (i)

the elements of ${\mathcal{F}}$ are well ordered subsets of $\Gamma$;

  2. (ii)

$A,B\in{\mathcal{F}}\Rightarrow A\cup B\in{\mathcal{F}}$;

  3. (iii)

$A\in{\mathcal{F}}\wedge B\subseteq A\Rightarrow B\in{\mathcal{F}}$.

######  Proposition 3.17 ([Ray68, Theorem 1]).

If ${\mathcal{F}}$ is a group family then the set
${\mathbb{G}}({\mathcal{F}})$ of elements of ${\mathbb{G}}$ whose supports
belong to ${\mathcal{F}}$ is a subgroup of ${\mathbb{G}}$.

###### Proof.

By property (i) all elements of ${\mathcal{F}}$ are well ordered so they can
occur as supports of elements of ${\mathbb{G}}$. Since for all
$g\in{\mathbb{G}}$ we have $\operatorname*{supp}(g)=\operatorname*{supp}(-g)$
then $g\in{\mathbb{G}}({\mathcal{F}})$ implies
$-g\in{\mathbb{G}}({\mathcal{F}})$. If $g,h\in{\mathbb{G}}({\mathcal{F}})$,
then
$\operatorname*{supp}(g+h)\subseteq\operatorname*{supp}(g)\cup\operatorname*{supp}(h)$:
the latter is in ${\mathcal{F}}$ by (ii) and hence by (iii) the former is too.
So $g+h\in{\mathbb{G}}({\mathcal{F}})$. It follows then that also
$0\in{\mathbb{G}}({\mathcal{F}})$ so ${\mathbb{G}}({\mathcal{F}})$ is indeed a
subgroup of ${\mathbb{G}}$. ∎

######  Definition 3.18.

Groups obtained as in Proposition 3.17 will be called _Rayner groups_. These
generalise an analogous notion (Rayner fields) introduced by Rayner in [Ray68]
in the context of Hahn fields. Further _Rayner structures_ are introduced and
studied in [KKS22].

The next result gives a necessary and sufficient condition for a Rayner group
to satisfy the canonical lifting property.

######  Theorem 3.19.

Let ${\mathbb{G}}$ be the Hahn product over the skeleton
$[\Gamma;\\{A_{\gamma}:\gamma\in\Gamma\\}]$. Let ${\mathcal{F}}$ be a group
family with respect to $\Gamma$. Then the Rayner group
$G={\mathbb{G}}({\mathcal{F}})$ has the canonical lifting property with
respect to the skeleton if and only if ${\mathcal{F}}$ is stable under the
action of $\operatorname{Aut}S(G)$, that is, for all
$[\bar{\tau};\\{\tau_{\gamma}:\gamma\in\Gamma\\}]\in\operatorname{Aut}S(G)$
and for all $T\in{\mathcal{F}}$ we have $\bar{\tau}(T)=T$.

###### Proof.

Assume that ${\mathcal{F}}$ be stable under $\operatorname{Aut}S(G)$ and let
$g=\sum g_{\gamma}\mathbbm{1}_{\gamma}\in G$. Let also
$\tau=[\bar{\tau};\tau_{\gamma}]$ be an automorphism of $S(G)$. Then
$\operatorname*{supp}\left(\sum\tau_{\gamma}(g_{\gamma})\mathbbm{1}_{\bar{\tau}(\gamma)}\right)=\\{\bar{\tau}(\gamma):\gamma\in\operatorname*{supp}(g)\\}=\bar{\tau}(\operatorname*{supp}(g))$
which, by our assumption, belongs to ${\mathcal{F}}$. Thus
$\sum\tau_{\gamma}(g_{\gamma})\mathbbm{1}_{\bar{\tau}(\gamma)}\in G$ and so
$G$ has the canonical lifting property with respect to the skeleton.

Vice versa, assume that $G$ has the canonical lifting property and let
$T\in{\mathcal{F}}$. Let $g=\sum g_{\gamma}\mathbbm{1}_{\gamma}\in G$ be such
that $\operatorname*{supp}(g)=T$. Then, for all $[\bar{\tau};\tau_{\gamma}]$
we have $h:=\sum\tau_{\gamma}(g_{\gamma})\mathbbm{1}_{\bar{\tau}(\gamma)}\in
G$, so in particular $\operatorname*{supp}(h)\in{\mathcal{F}}$. So
${\mathcal{F}}$ is stable under action of $\operatorname{Aut}S(G)$. ∎

######  Definition 3.20.

Let $\kappa$ be an infinite regular cardinal and let ${\mathcal{F}}_{\kappa}$
be the family of all well ordered subsets of $\Gamma$ of cardinality smaller
than $\kappa$. The $\kappa$-bounded subgroup of ${\mathbb{G}}$ consists of all
elements of ${\mathbb{G}}$ with support of cardinality less than $\kappa$, and
will be denoted by ${\mathbb{G}}_{\kappa}$.

Notice that the Hahn sum and the Hahn product are special cases of
$\kappa$-bounded subgroups, with $\kappa=\aleph_{0}$ and
$\kappa=|{\mathbb{G}}|^{+}$ respectively.

######  Corollary 3.21.

Let $\kappa$ be an infinite regular cardinal. Then ${\mathbb{G}}_{\kappa}$ has
the canonical lifting property. In particular, ${\mathbb{G}}$ and
$\coprod_{\Gamma}A_{\gamma}$ have the canonical lifting property.

###### Proof.

By Proposition 3.19 it suffices to show that the family
${\mathcal{F}}_{\kappa}$ of well ordered subsets of $\Gamma$ is stable under
$\operatorname{Aut}S(G)$. Let
$[\bar{\tau};\\{\tau_{\gamma}:\gamma\in\Gamma\\}]\in\operatorname{Aut}S(G)$
and let $T$ be a well ordered subset of $\Gamma$ of cardinality less than
$\kappa$. Then $\bar{\tau}(T)$ will have the same cardinality as $T$, because
$\bar{\tau}$ is an automorphism of $\Gamma$. Since it is also order preserving
then $\bar{\tau}(T)$ will also be well ordered, hence
$\bar{\tau}(T)\in{\mathcal{F}}_{\kappa}$. ∎

Analogous results to the ones contained in this section hold for Rayner
fields: see [KS22, Section 3.2].

##  4\. Automorphism groups as groups of matrices

In this section we are going to generalise and sharpen results of Conrad
[Con58] and Droste-Göbel [DG97]. The first describes the group of order
preserving automorphisms of a Hahn sum as a certain group of triangular
matrices. The latter obtain a semidirect product decomposition, which is
different from the one we gave in Theorem 3.15 – we will explain how the two
relate.

Let $\Gamma$ be a chain, $\\{A_{\gamma}:\gamma\in\Gamma\\}$ a family of
ordered abelian groups and $G=\coprod_{\gamma\in\Gamma}A_{\gamma}$ the Hahn
sum. Let $\operatorname{End}A_{\gamma}$ be the ring of endomorphisms of
$A_{\gamma}$ (with pointwise addition and composition). For all
$\alpha,\beta\in\Gamma$ let
$H_{\alpha\beta}=\operatorname{Hom}(A_{\beta},A_{\alpha})$ be the group of
homomorphisms from $A_{\beta}$ into $A_{\alpha}$ (with pointwise addition).
Let $\Delta$ be the set of all $\Gamma{\times}\Gamma$-matrices
$(\sigma_{\alpha\beta})$ where

  1. (i)

$\sigma_{\alpha\alpha}\in\operatorname{End}A_{\alpha}$;

  2. (ii)

$\sigma_{\alpha\beta}\in H_{\alpha\beta}$;

  3. (iii)

for every $\beta$ and for all $a\in A_{\beta}$ we have
$\sigma_{\alpha\beta}(a)=0$ for all but finitely many $\alpha$.

Then $\Delta$ forms a ring with respect to the usual matrix addition and
multiplication (condition (iii) ensures that the product be well defined).

######  Remark 4.1.

There is an isomorphism between the rings $\operatorname{End}G$ and $\Delta$.
Indeed, for $a=\sum a_{\gamma}\mathbbm{1}_{\gamma}\in G$ and
$(\sigma_{\alpha\beta})\in\Delta$ we can consider the row vector
$(a_{\gamma})$ and multiply it on the left to get

$(a_{\gamma})(\sigma_{\alpha\beta})=\left(\sum_{\alpha\in\operatorname*{supp}(a)}\sigma_{\alpha\beta}(a_{\alpha})\right)_{\beta\in\Gamma}=:(b_{\beta})_{\beta\in\Gamma}$
(5)

where the sum on the right hand side makes sense because the
$\operatorname*{supp}(a)$ is finite. Then it is clear that the map
$\sigma:\sum a_{\gamma}\mathbbm{1}_{\gamma}\mapsto\sum
b_{\gamma}\mathbbm{1}_{\gamma}$ is an endomorphism of $G$ induced by
$(\sigma_{\alpha\beta})$.

Vice versa, if $\sigma\in\operatorname{End}G$ and $a=\sum
a_{\gamma}\mathbbm{1}_{\gamma}\in G$ let $b=\sum
b_{\gamma}\mathbbm{1}_{\gamma}:=\sigma(a)$. Then for all $\alpha\in\Gamma$
define $\sigma_{\alpha\beta}(a_{\alpha}):=b_{\beta}$. Since $b\in G$ then
$\operatorname*{supp}(b)$ is finite and therefore $\sigma_{\alpha\beta}$
satisfies condition (iii) above. Hence $(\sigma_{\alpha\beta})\in\Delta$ and
it is easy to verify that the correspondence
$\sigma\mapsto(\sigma_{\alpha\beta})$ is an isomorphism of
$\operatorname{End}G$ onto $\Delta$. Thus the group $\operatorname{Aut}G$
corresponds isomorphically to the group of units in $\Delta$. Here by
$\operatorname{Aut}G$ we mean _all_ automorphisms of $G$ as an abelian group,
order (reps. valuation) preserving or otherwise. ∎

Now we see how we can characterise the order preserving automorphisms. Let us
consider the multiplicative monoid $T$ of all matrices
$(\sigma_{\alpha\beta})\in\Delta$ such that

  1. (i’)

$(\sigma_{\alpha\beta})$ is lower triangular;

  2. (ii’)

for all $\alpha\in\Gamma$, $\sigma_{\alpha\alpha}\in
o\text{-}\operatorname{Aut}A_{\alpha}$.

The order on $G$ is the lexicographic one (cf. Definition 2.5).

######  Lemma 4.2.

A matrix $(\sigma_{\alpha\beta})\in T$ induces, via the correspondence
described in Remark 4.1, an order preserving endomorphism $\sigma$ on $G$. In
particular, an invertible matrix in $T$ induces an order preserving
automorphism of $G$.

###### Proof.

Let $a=\sum_{\gamma}a_{\gamma}\mathbbm{1}_{\gamma}>0$ with $v(a)=\gamma$. Then

$\displaystyle\sigma(a)=(a)_{\alpha}(\sigma_{\alpha\beta})$
$\displaystyle=\left(\sum_{\alpha\geq\gamma}\sigma_{\alpha\beta}(a_{\alpha})\right)_{\beta\in\Gamma}$
$\displaystyle=\left(\sum_{\alpha\geq\gamma}\sigma_{\alpha\beta}(a_{\alpha})\right)_{\beta\geq\gamma}=\sigma_{\gamma\gamma}(a_{\gamma})+\left(\sum_{\alpha\geq\gamma}\sigma_{\alpha\beta}(a_{\alpha})\right)_{\beta>\gamma}>0$

where the third equality holds because $\sigma_{\alpha\beta}$ is lower
triangular and the final inequality is due to the fact that, by (ii’),
$\sigma_{\gamma\gamma}$ is order preserving. ∎

######  Corollary 4.3.

Let $U$ denote the group of units (the invertible matrices) in $T$. Then $U$
embeds into $o\text{-}\operatorname{Aut}G$.∎

######  Remark 4.4.

From the proof of Lemma 4.2 it is clear that all $o$-automorphisms of $G$
induced by the elements $(\sigma_{\alpha\beta})\in U$ induce the identity on
$\Gamma$. If, moreover, such an automorphism also induces the identity on each
component then it will be an internal automorphism. Again, from the formula in
the proof of Lemma 4.2, we see that this happens if and only if
$\sigma_{\alpha\alpha}=1$ for all $\alpha\in\Gamma$. ∎

######  Theorem 4.5.

Let us denote by $o\text{-}\operatorname{Aut}_{\Gamma}G$ the automorphisms of
$G$ that induce the identity on the chain $\Gamma$. Similarly, let
$\operatorname{Aut}_{\Gamma}S(G)$ denote the group of automorphisms of the
skeleton whose component on $\Gamma$ is the identity. Then we have

$o\text{-}\operatorname{Aut}_{\Gamma}G\simeq\operatorname{Int}\operatorname{Aut}G\rtimes\operatorname{Aut}_{\Gamma}S(G).$

It follows, in particular, that $o\text{-}\operatorname{Aut}_{\Gamma}G$ forms
a group.

###### Proof.

Let $U^{1}$ be the subgroup of $U$ given by all the _unitriaugular_ matrices
of $T$ (all the diagonal entries are 1). Then
$\operatorname{Int}\operatorname{Aut}G\simeq U^{1}$. When we multiply two
diagonal matrices, the elements on the diagonal of the product are the
products of the corresponding diagonal entries. Hence $U^{1}$ is a normal
subgroup of $U$. Moreover, every element $(\sigma_{\alpha\beta})\in U$ is the
product of an element in $U^{1}$ and a diagonal matrix: namely
$(\sigma_{\alpha\beta})=(\sigma_{\alpha\beta})^{1}(\sigma_{\alpha\beta})^{d}$
where $(\sigma_{\alpha\beta})^{1}$ is obtained by $(\sigma_{\alpha\beta})$
replacing all diagonal elements by 1 and $(\sigma_{\alpha\beta})^{d}$ is the
diagonal matrix with the diagonal entries of $(\sigma_{\alpha\beta})$ on its
diagonal. And finally, if we denote by $U^{d}$ the subgroup of diagonal
matrices in $U$ we have $U^{d}\cap U^{1}=\\{1\\}$. Hence $U=U^{1}\rtimes
U^{d}$. ∎

The last proposition gives us a matrix description of the group of
automorphisms that induce the identity on $\Gamma$. But we know that every
automorphism of the skeleton (there might be some where the component on
$\Gamma$ is not trivial) lifts to an automorphism of $G$, because Hahn sums
have the (canonical) lifting property. So, in order to completely describe
$o\text{-}\operatorname{Aut}G$, we need to take into account the lifts of
automorphisms of the skeleton with non trivial component on $\Gamma$. Let
$\operatorname{Aut}_{\Gamma}^{\prime}S(G)$ be a complement of
$\operatorname{Aut}_{\Gamma}S(G)$ in $\operatorname{Aut}S(G)$. We have:

######  Theorem 4.6.

We have

$o\text{-}\operatorname{Aut}G=\left(U^{1}\rtimes
U^{d}\right)\rtimes\operatorname{Aut}_{\Gamma}^{\prime}S(G).$

###### Proof.

By Theorem 4.5 we have
$o\text{-}\operatorname{Aut}_{\Gamma}G\simeq\operatorname{Int}\operatorname{Aut}G\times\operatorname{Aut}_{\Gamma}S(G)\simeq
U^{1}\rtimes U^{d}$ hence it suffices to show that
$o\text{-}\operatorname{Aut}G\simeq
o\text{-}\operatorname{Aut}_{\Gamma}G\rtimes\operatorname{Aut}_{\Gamma}^{\prime}S(G)$.
We notice immediately that the two groups have trivial intersection and that
$o\text{-}\operatorname{Aut}_{\Gamma}G$ is normal inside
$o\text{-}\operatorname{Aut}G$. To show that they generate
$o\text{-}\operatorname{Aut}G$ we take an automorphism $\sigma\in
o\text{-}\operatorname{Aut}G$. This will induce an automorphism
$\tau\in\operatorname{Aut}S(G)$ which might or might not fix $\Gamma$. In any
case, if we lift $\tau$ to a $\tilde{\tau}\in o\text{-}\operatorname{Aut}G$ we
certainly have that $\sigma\tilde{\tau}^{-1}$ induces the identity on
$\Gamma$. So $\sigma=(\sigma\tilde{\tau}^{-1})\tilde{\tau}$. ∎

Theorem 4.6 improves [DG97, Corollary 3.2] in two ways: it provides a
generalisation from the balanced case to that of a Hahn sum over an arbitrary
skeleton and gives a more precise description of
$o\text{-}\operatorname{Aut}G$ by identifying the internal automorphisms
within the group of matrices $U$.

## References

  * [Bag+23] V. Bagayoko, S. Krapp, D. Panazzolo and M. Serra  “Automorphisms and derivations on algebras endowed with formal infinite sums” Work in progress, 2023
  * [Con] K. Conrad  “Splitting of short exact sequences for groups”  URL: https://kconrad.math.uconn.edu/blurbs/grouptheory/splittinggp.pdf
  * [Con58] P. Conrad  “The Group of Order Preserving Automorphisms of an Ordered Abelian Group”  In _Proc. Amer. Math. Soc._ 9.3 American Mathematical Society, 1958, pp. 382–389
  * [DG97] M. Droste and R. Göbel  “The automorphism groups of Hahn groups”  In _Ordered algebraic structures (Curaçao, 1995)_ Kluwer Acad. Publ., Dordrecht, 1997, pp. 183–215
  * [Hof91] H. Hofberger  “Automorphismen formal reeller Körper”, 1991
  * [KKS22] L.. Krapp, S. Kuhlmann and M. Serra  “On Rayner structures”  In _Communications in Algebra_ 50.3 Taylor & Francis, 2022, pp. 940–948  DOI: 10.1080/00927872.2021.1976789
  * [KS22] S. Kuhlmann and M. Serra  “The automorphism group of a valued field of generalised formal power series”  In _Journal of Algebra_ 605, 2022, pp. 339–376  DOI: https://doi.org/10.1016/j.jalgebra.2022.04.023
  * [KS23] S. Kuhlmann and M. Serra  “On the structure of the automorphism group of an ordered Hahn group” Work in progress, 2023
  * [Kuh00] S. Kuhlmann  “Ordered exponential fields” 12, Fields Institute Monographs  American Mathematical Society, Providence, RI, 2000, pp. xviii + 166
  * [Ray68] F.. Rayner  “An algebraically closed field”  In _Glasgow Math. J._ 9, 1968, pp. 146–151  DOI: 10.1017/S0017089500000422


# Self-Supervised Learning of Dynamic
Planar Manipulation of Free-End Cables

Jonathan Wang1,∗, Huang Huang1,∗, Vincent Lim1,
Harry Zhang1, Jeffrey Ichnowski1, Daniel Seita1, Yunliang Chen1, Ken Goldberg1
*Equal contribution.1AUTOLAB at the University of California, Berkeley,
USA.Correspondence to<EMAIL_ADDRESS>
###### Abstract

Dynamic manipulation of free-end cables has applications for cable management
in homes, warehouses and manufacturing plants. We present a supervised
learning approach for dynamic manipulation of free-end cables, and focus on
the problem of getting the cable endpoint to arrive at a designated target
position, which may lie outside the reachable workspace of the robot’s end
effector. We create a simulator and tune it to closely match experiments with
physical cables. We study how to efficiently collect training data for
learning dynamic cable manipulation. We extensively benchmark the simulation
environment on planar cable manipulation, and transfer the trained models to a
physical UR5 robot. Results over [Daniel: XXX trials] and three cables suggest
that a physical UR5 robot can attain accuracy of [Daniel: TODO describe
results] Supplementary material is available at https://tinyurl.com/dyncable.

##  I Introduction

Dynamic free-end cable manipulation is useful in a wide variety of cable
management settings in homes and warehouses. For example, in a decluttering
setting with cables, humans may take a cable and dynamically manipulate it to
a distant location to efficiently move the cable out of the way, cast it
towards a human partner for subsequent manipulation, or hang it over a hook or
beam. Dynamic cable manipulation may also be advantageous when a cable must be
passed through an obstacle to be retrieved on another side, requiring
precision in the location of the cable endpoint. We use the generic term
_cable_ to refer to a 1D deformable object with minimal stiffness, which
includes ropes, wires, and threads.

In this paper, we propose a supervised learning procedure that, given the
target location of a cable’s endpoint, generates a dynamic manipulation
trajectory. Targets $(r,\theta)$ are defined in polar coordinates with radius
$r$ and angle $\theta$ with respect to the robot’s base. We focus on planar
dynamic cable manipulation tasks in which the cable lies on a surface and
(after a reset action) a robot must dynamically adjust the cable via planar
motions. See Figure 1 for an overview. In prior work [harry_rope_2021], we
explored dynamic manipulation of fixed-end cables. In this work we relax this
assumption to consider free-end cables, where the far end is unconstrained.

We develop a simulation environment for learning dynamic manipulation of free-
end cables in PyBullet [coumans2019]. We efficiently tune the simulator by
tracking the cable’s location and comparing it with a physical location with
the help of differential tuning [de_tuning_2020]. We use Hindsight Experience
Replay [HER_2017] for training policies in simulation to take actions to get
cable endpoints to reach desired targets. We perform physical experiments for
evaluating cable dynamic manipulation policies on a UR5 robot.

Figure 1:  Overview of the proposed dynamic cable manipulation procedure with
a UR5 robot. Each manipulation consists of a reset procedure (Section IV)
followed by a dynamic planar action (Section VI-A) with two waypoints. The top
two rows show a frame-by-frame side view of the UR5 as it resets by letting
the cable settle (A,B), pulling back (C), casting forward (D,E), and dragging
back slightly (F). After the reset, the bottom two rows show a frame-by-frame
frontal view as the UR5 pulls towards one waypoint (G,H,I) and then towards
the second waypoint (J,K,L) in an effort to get the cable endpoint to reach a
desired target. [Daniel: Draft figure. Should highlight cable (I think we can
do in image editing software). Also wondering if side views and front views
are OK. We could also overlay some arrows.]

The contributions of this paper are:

  1. 1.

Formulation of the dynamic planar manipulation of free-end cables.

  2. 2.

A simulation platform that accurately models dynamic planar cable
manipulation.

  3. 3.

A dataset from 600 physical trials with 3 cables and the UR5 robot.

  4. 4.

Physical experiments with a UR5 robot and 3 cables [JI: suggesting that …].

Supplementary material is available at https://tinyurl.com/dyncable.

##  II Related Work

This paper builds on extensive prior research in data-driven robotics,
deformable object manipulation, and dynamic manipulation. While using learned,
data-driven techniques for robot manipulation has become a highly effective
technique for generalizing to a wide variety of objects, as judged from strong
empirical results [levine_finn_2016, mahler2019learning,
dense_object_nets_2018, QT-Opt, openai-dactyl, rubik_cube_2019], the majority
of approaches focus primarily on manipulation of rigid objects in largely
static settings such as a bin with objects, in contrast to the dynamic setting
we consider with a deformable free-end cable.

###  II-A 1D Deformable Object Manipulation

Manipulation of 1D deformable objects such as cables has a long history in
robotics research. Some representative applications include surgical suturing
[robot_heart_surgery_2006, schulman_case_study_2013, suturing_autolab_2016],
knot-tying [van_den_berg_2010, case_study_knots_1991, knot_planning_2003,
tying_precisely_2016], untangling ropes [rope_untangling_2013,
grannen2020untangling, tactile_cable_2020], deforming wires
[wire_insertion_1996, wire_insertion_1997], and inserting cables into
obstacles [tight_tolerance_insertion_2015]. We refer the reader to Sanchez et
al. [manip_deformable_survey_2018] for a survey.

There has been a recent surge in using learning-based techniques for
manipulating cables towards target configurations. A common setting is to
employ pick-and-place actions with quasistatic dynamics, meaning that the
robot slowly deforms the cable through a series of small adjustments, while
allowing the cable to settle between actions. Under this paradigm, prior work
has focused on cable manipulation strategies via techniques such as learning
from demonstrations [zeng_transporters_2020, seita_bags_2021], self-supervised
learning [nair_rope_2017, ZSVI_2018], generative modeling
[wang_visual_planning_2019], state estimation [state_estimation_LDO], and
dense object descriptors [priya_2020]. Prior work has also explored the use of
simulators [mujoco, corl2020softgym] to train cable manipulation policies
using reinforcement learning for eventual transfer to physical robots
[lerrel_2020, yan_fabrics_latent_2020]. In this work, we develop a simulator
for dynamic cable manipulation to accelerate data collection for training
policies, with transfer to a physical UR5 robot.

Other cable manipulation settings that do not necessarily assume quasistatic
dynamics include sliding cables [one_hand_knotting_tactile_2007,
zhu_sliding_cables_2019] or using tactile feedback [tactile_cable_2020] to
inform policies. This paper focuses on dynamic planar actions to manipulate
cables from one configuration to another.

###  II-B Dynamic Manipulation of Cables

In dynamic manipulation settings, a robot takes actions rapidly to quickly
move objects towards desired configurations, as exemplified by applications
such as swinging items [swingbot_2020, diabolo_2020] or tossing objects to
target bins [zeng_tossing_2019]. As with robot manipulation, these approaches
tend to focus on dynamic manipulation of rigid objects.

A number of analytic physics models have been developed to describe the
dynamics of moving cables, which can then be used for subsequent planning. For
example, Gatti-Bono and Perkins [fly_fishing_2004] present a two-dimensional
dynamic casting model of fly fishing. They model the fly line as a long
elastica and the fly rod as a flexible Euler-Bernoulli beam, and propose a
system of differential equations to predict the movement of the fly line in
space and time. In contrast to continuum models, Wang and Wereley
[wang2011analysis] propose using a finite-element model to represent the fly
line by a series of rigid cylinders that are connected by massless hinges.
These works focus on developing mathematical models for cables, and do not
test on physical robots.

In the context of robotic dynamic-cable manipulation, Yamakawa et al.
[dynamic_knotting_2010, yamakawa2012simple, high_speed_knotting_2013,
one_hand_knotting_tactile_2007] demonstrate that a high-speed manipulator can
effectively tie knots by initially whipping cables upwards. Since their robot
arms move at high speeds, they are able to simplify the modelling of cable
deformation by assuming each cable component follows the robot end-effector
motion with constant time-delay. Kim and Shell [kim2016using] study a novel
robot system with a flexible rope-like structure attached as a tail that can
be used to strike objects of interest at high speed. This high-speed setting
allows them to construct primitive motions that exploit the dynamics of the
tail. They adopt the Rapidly-exploring Random Tree (RRT) [RRT_2001] and a
particle-based representation to address the uncertainty of the object’s state
transition. In contrast to these works, we aim to control cables for tasks in
which we may be unable to rely on these simplifying assumptions.

Two closely related prior works are Zhang et al. [harry_rope_2021] and
Zimmermann et al. [zimmermanndynamic]. Zhang et al. [harry_rope_2021] propose
a self-supervised learning technique for dynamic manipulation of fixed-end
cables for vaulting, knocking, and weaving tasks. They parameterize actions by
computing a motion using a quadratic program and learning the apex point of
the trajectory. In contrast to Zhang et al., we use _free-end_ cables.
Zimmermann et al. [zimmermanndynamic] also study the dynamic manipulation of
deformable objects in the free-end setting. They model elastic objects using
the finite-element method [bathe2006finite] and use optimal control techniques
to solve trajectory optimization problems. They study the task of whipping,
which aims to find a trajectory such that the free end of the cable hits a
predefined target with maximum speed, and laying a cloth on the table. Their
simulated motions perform well for simple dynamical systems such as a
pendulum, but since their model depends on good parameter estimation, its
performance decreases for more complex systems like soft bodies due to
mismatch between the simulation and the real world. In contrast, we adopt a
learned, data-driven technique for robotic manipulation of free-end cables,
focus on planar manipulation, and are interested in finding the best action
such that the free end of the cable achieves a target position on the plane.

##  III Problem Statement

We consider a 6-DOF robot arm which controls a free-end cable and continually
grips one end of the cable throughout each dynamic motion. We assume a planar
setup where the cable endpoint lies on the plane and the robot end-effector
takes actions at a fixed height slightly above the plane. The full
configuration of the cable and the environment is infinite dimensional and
complex to model, and we are interested in the cable endpoint. Thus, we
concisely represent the state as ${\mathbf{s}=(r,\theta)}$, indicating the
endpoint position on the plane in polar coordinates, and where the robot base
is located at $(0,0)$. The objective is to train a parameterized inverse model
$f_{\rm inv}$ which, when given the desired target
${\mathbf{s}^{(d)}=(r_{d},\theta_{d})}$ for the cable endpoint, produces a
one-step arm motion ${\mathbf{a}=f_{\rm inv}(\mathbf{s}^{(d)})}$ that robustly
achieves $\mathbf{s}^{(d)}$. We assume access to a dataset of $N$ pairs of end
positions and actions, denoted as
${\mathcal{D}=\\{(\mathbf{s}^{(d)},\mathbf{a})^{(i)}\\}_{i=1}^{N}}$ to
supervise $f_{\rm inv}$.

[JI: I think we need to provide an overview of the method, including
resets/repeatability, sim-to-real, and the rest, before jumping into resets
and repeatability. I also recommend we top-level sections that are abstract,
e.g. ”Preliminaries”, ”Background”, ”Setup”, ”Method”, etc… So ”Resets and
Repeatability” and ”Simulation to Reality” would be either under their own
top-level section, or moved into the Methods section.] [Vincent: Perhaps
saying we train $f_{\rm inv}$ is a little confusing when later we train a
forward dynamics model and sample from that?]

##  IV Resets and Repeatability

The first step in dynamic cable manipulation is to determine a highly
repeatable _reset state_ , a position that the cable can be quickly and
reliably manipulated into. We propose the following reset procedure [JI: we
may consider defining a general reset procedure/method, and leave the specific
one below to the experiments section. The idea being to keep the method
description here high-level, and use the experiments as an “instantiation” of
the method.]:

  1. 1.

Lift the cable vertically up with the free endpoint touching the work surface
to prevent the cable from dangling.

  2. 2.

Continue to lift the cable up such that the free endpoint loses contact with
the surface.

  3. 3.

Let the cable hang for 3 seconds to stabilize. [Raven: I changed the reset
motion as you suggested. However, we still need to hang up for a few sec since
there could be some dangling after we lift it up. At the very beginning,
without step 1 and 2, we have to wait for 20 seconds. Now we just need to wait
3s to entirely stabilize the cable.]

  4. 4.

Perform a motion to swing the free endpoint forward to land on the surface,
then slowly pull the cable towards the robot until the reset position is
reached.

We execute this reset motion before each dynamic planar cable manipulation
action to ensure the same initial cable configuration. Figure 1 (top half)
shows a frame-by-frame overview of the reset procedure. After this reset
procedure, we assume that the robot end-effector performs dynamic manipulation
at a fixed height.

##  V Simulation to Reality

Figure 2:  Above: The extracted cable endpoint positions at each time step
spaced 100 milliseconds apart. The pixel coordinates of each waypoint are
converted to simulation coordinates using x reference points. Below: The same
trajectory is run in the simulator, and the distance between the cable
endpoint and the waypoint at each time step is used as a metric to for
optimizing the simulation parameters. In this example, the simulated endpoint
location matches the ground truth endpoint location, but the cable
configuration is different, demonstrating a limitation with the optimization
metric. The cables are highlighted for the purpose of visibility.

We create a simulation environment to efficiently collect data for policy
training. In PyBullet [coumans2019] simulation, cables consist of capsule-
shaped rigid bodies held together by 6-DOF spring constraints. We denote the
collective set of PyBullet simulation parameters as $\theta_{\rm sim}$. For
each trajectory $m_{j}$ in the training set of $M$ total _physical_
trajectories, we record the 2D location of the cable endpoint,
$p_{t}=(x_{t},y_{t})$ [Jonathan: Earlier we refer to endpoints in polar
coordinates, we need to keep it consistent], at each time step $t$ spaced
$100\text{\,}\mathrm{ms}$ apart [Jonathan: Consider moving this section V
after section VI-A, where we can define planar actions and corresponding
trajectories]. The tuning objective is to find parameters $\theta_{\rm sim}$
that minimize the average $L^{2}$ distance between the cable endpoint in
simulation, $\hat{p}_{t}=(\hat{x}_{t},\hat{y}_{t})$, to the real waypoint at
each $t$:

$\operatorname*{minimize}_{\theta_{\rm sim}}\;\;\epsilon_{\rm
trajs}={\sum_{j}^{M}{\sum_{i}^{N_{j}}{\lVert p_{i}-\hat{p}_{i}\rVert}_{2}}}.$
(1)

The number of waypoints compared for each trajectory is
$N_{j}=\lfloor{T_{j}}/{100}\rfloor$, where $T_{j}$ is the duration of $m_{j}$
in milliseconds. For each cable, we collect 60 physical trajectories and
evaluate on an additional 20 trajectories where
$(\theta_{1},\theta_{2},r_{2},\psi)$, for each trajectory are uniformly
sampled. Increasing the number of trajectories to 80 led to an increase in the
validation metric $D$ by $2\%-12\%$, suggesting that additional trajectories
would not improve tuning. [Daniel: Above text needs work, we’re using notation
before we define it, and we are talking about physical trajectories (not
resets!) before we define what those are.] [JI: Also… we should keep this at a
higher level, and move the 60 vs 80 discussion to experiments and discuss as
part of ablations or similar discussion.]

###  V-A PyBullet Simulation Parameter Tuning

To tune parameters more efficiently compared to grid search or random search
[random_search_2012], we search for parameters using Differential Evolution
(DE) [de_tuning_2020]. We do this because DE does not require a differentiable
optimization metric, and is more efficient than grid search and Bayesian
optimization, which both scale poorly with a large number of parameters. Grid
search runtime is exponential in the number of parameters, while evolutionary
algorithms allow for a variable stopping condition. [JI: Is the previous
statement backed up by Collins et al. [de_tuning_2020]? If so, we should state
that clearly. Otherwise this statement needs more justification.] At
generation $0$, the algorithm initializes a size $N$ population of candidate
parameter vectors via Latin Hypercube sampling. After each generation, each
parameter vector in the current generation produces a ‘child’ vector, which
each element in the vector is derived from a combination of the parent, the
best-performing individual in the current generation, and $2$ random
population members. This heuristic permits enough stochasticity to
sufficiently explore the parameter search space, while converging toward an
optimum.

Table IV [Jonathan: This table is now in the Appendix] lists the simulation
parameters we tuned, which we chose based on intuition of the parameters that
best capture cable physics. Comparing the selected parameters with Collins et
al. [de_tuning_2020], who explore a similar physics engine tuning problem, we
also optimize over mass, lateral friction, rolling friction, sliding friction,
linear damping, and angular damping. Two other parameters the authors claim
were most influential were time step and maximum joint velocity. We do not
vary PyBullet’s time step because decreasing it from 1/480s causes the cable,
modeled as discrete capsules, to exhibit unstable behavior such as breaking
apart, while increasing the time step greatly decreases the simulation speed.
We find that running DE with the maximum joint velocities as parameters as
opposed to the original set increased the deviation in cable trajectory by 11%
for one cable, most likely due to an increase in the search space. Tuning
maximum joint velocities also increases the duration of the algorithm by 60%
due to an additional 6 parameters to the population size.

Table I compares similarity between cable behavior in simulation and reality.
For each cable, DE was able to reduce the discrepancy in the cable endpoint
location throughout and after the trajectory, compared to PyBullet’s default
parameter settings. Despite the tuning, results suggest a reality gap in cable
behavior still remains, motivating the use of the fine-tuning procedure
discussed in Section VI-B.

Table I:  Results for the differential evolution algorithm on a set of 20 validation trajectories over all 3 cables (see Figure 4). The two metrics considered are final $L^{2}$-distance, which is the 2D Euclidean distance between endpoint locations between simulation and reality after a trajectory terminates, and $\epsilon_{\rm trajs}$, defined in Equation 1. We compare the performance of parameters found by DE with that of PyBullet’s default parameter settings. Values are averaged across the 20 trajectories, and expressed in percentages of cable length.  Metric | Cable 1 | Cable 2 | Cable 3
---|---|---|---
(Default) Final $L^{2}$-distance | 38.2% | 27.5% | 27.5%
(Default) $\epsilon_{\rm trajs}$ | 38.2% | 29.4% | 24.5%
(DE) Final $L^{2}$-distance |  20.5% |  19.2% |  16.6%
(DE) $\epsilon_{\rm trajs}$ |  28.4% |  23.4% |  18.7%

##  VI Method

In this section, we describe how we propose to address dynamic manipulation of
free-end cables, including the action parameterization and the supervised
learning procedure. We use this procedure to deploy 2D planar manipulation on
a physical UR5 robot.

###  VI-A Planar Action Parameters

Each action $\mathbf{a}$ defines a dynamic trajectory of the robot.

[Daniel: I think we need better justification for the action params – section
is major WIP, will work on it.] We define and test with the action
parameterizations that replicate recorded human-generated trajectories of free
driving the robot to dynamically move the cable towards a target. We notice
the trajectories can be represented by two splines that are parameterized in
polar coordinates. We choose to use two splines as opposed to one because the
second spline is required to produce an inertial planar motion that
accelerates the endpoint in the direction of the end effector’s velocity.
[Jonathan: Possibly support the reasoning with the coverage metric of using a
single spline] [Jonathan: Need to revise the reasoning and better integrate it
with below explanation] [Daniel: easy workaround: do with 1 spline, show it is
bad]

We represent the robot as a 3 joint mechanism in the global frame: a rotation
joint at the base $\phi_{1}$, a prismatic joint $d_{2}$ along the direction
from the base to the wrist, and a rotation angle $\phi_{3}$ with respect to
$d_{2}$. In the polar coordinate frame, we define $(r,\theta)$ to be the
position of the end effector, and $\psi$ as the angle of the wrist. Using a
standard IK solver, the 6-DOF robot can achieve any
$(\phi_{1},d_{2},\phi_{3})$, which corresponds to moving the end effector to
$(r,\theta,\psi)$ in polar space.

Each action is parameterized by a fixed set and a variable set of parameters,
which we refer to as _system parameters_ and _action parameters_ ,
respectively. The system parameters are $r_{0}$, the 2D distance from the
reset position of the end effector to the origin in the polar coordinate
frame, and $v_{\rm max}$, the linear velocity of the end effector. The $r_{0}$
can be modified to produce trajectories that vary in curvature, as seen in
Figure 3. Larger values for $r_{0}$ will produce “flatter” splines whereas
smaller values will produce splines with more curvature. In practice, we find
that the flatness of the action and the maximum velocity $v_{\rm max}$ in the
jerk-limited motion significantly impact the cable endpoint coverage over the
workspace. [Daniel: Need plots / results!] To find the best $(r_{0},v_{\rm
max})$ that generate actions with maximum coverage, we perform grid search in
the PyBullet simulator and choose the combination with maximum coverage as in
Section VII [Daniel: Section reference does not make sense here]. Once we
select the system parameters, they are fixed and not included in the actions
$\mathbf{a}$ for learning the inverse model $f_{\rm inv}$.

We consider two sets of action parameters. The first consists of three values:

$\mathbf{a}=(\theta_{1},\theta_{2},r_{2}).$ (2)

With a fixed reset location $(r_{0},\theta_{0})$ given by the reset motion in
Section IV, the trajectory of the end effector passes through two waypoints
$(r_{1},\theta_{1})$ and $(r_{2},\theta_{2})$ in polar space while maintaining
a fixed height, as shown in Figure 3, where $\theta$ changes with jerk-
limited, maximum velocity $v_{\rm max}$. A limited jerk velocity reduces the
stochasticity brought by a jerk-intensive motion. The $r$ changes from $r_{0}$
to $r_{2}$ using a cubic spline $f(t)$ as a function of time, where
$f^{\prime}(t_{r_{0}})=f^{\prime\prime}(t_{r_{0}})=f^{\prime}(t_{r_{2}})=f^{\prime\prime}(t_{r_{2}})=0$.
[Jonathan: TODO Verify this is accurate]. Here, $r_{1}$ is equal to
$f(T_{\theta_{1}})$, where $T_{\theta_{1}}$ is the time it takes to change
from $\theta_{0}$ to $\theta_{1}$. [Raven: Jeff would be the best person to
further explain this.]

[Raven: This is hard to explain and we need to discuss how to make this clear.
Basically, when we generate the spline action, we didn’t generate in the polar
coordinate whose origin is the robot base. Since we just want to generate the
spline action with the starting point ($r_{0}$, $\theta_{0}$) fixed, we can do
this in the action space. Since $r_{0}$ is directly related to the flatness of
the generated spline, if we want to generate actions with different flatness,
we need to change $r_{0}$. However, the reset position is fixed in real and we
can only move the origin further or near to change $r_{0}$. I hope this could
explain this better but again Jeff would be the best person to explain. ]

We notice the coverage space for actions with the above three parameters is
limited in practice (see Figure 5). Taking inspiration from human generated
action [Daniel: this is not scientific] and verifying our intuition with
physical motions (Section VII-C), we include a fourth parameter, the wrist
rotation angle $\psi$ around the z-axis, resulting in:

$\mathbf{a}=(\theta_{1},\theta_{2},r_{2},\psi).$ (3)

During the trajectory, the end effector follows the spline parameterized by
$(\theta_{1},\theta_{2},r_{2})$, and for the portion of the trajectory from
$(r_{1},\theta_{1})$ to $(r_{2},\theta_{2})$, the wrist rotates
$\psi-\theta_{2}$ radians with a maximum velocity, jerk-limited rotation. In
an ablation study in [Daniel: insert section], we examine the impact of the
wrist angle parameter.

Figure 3:  Example of two sets of splines that are traced by the end effector
during the trajectory (Equation 2) (not drawn to scale) of the UR5, for two
different $r_{0}$ values: $r_{0}\in\\{0.6,2.0\\}$. The reset procedure brings
the end effector to $(r_{0},\theta_{0})$ in the polar space before the start
of each dynamic motion. Decreasing $r_{0}$ increases the curvature of the
trajectory. [Daniel: draft figure, but much improved! We need to make wrist
rotation clear, and adjust as per Prof Goldberg’s feedback (e.g., showing
consistent coordinate systems) Also, we show $\phi_{1}$ and $\phi_{3}$ but not
$d_{2}$, and the $\psi$ could be visualized better maybe with an inset image?
Finally use $r_{0}=2.0$ instead of just 2.]

###  VI-B Supervised Learning Procedure

The inverse model $f_{\rm inv}$ consists of sampling multiple input actions
$\mathbf{a}$, passing each action through a forward dynamics model $f_{\rm
forw}$ that outputs the predicted cable endpoint location $(\hat{x},\hat{y})$,
and selecting the optimal $\mathbf{a}$. We use a feedforward neural network
for this forward dynamics model, with 4 fully connected layers with 256 hidden
units each. [Daniel: Now we’re talking Cartesian coordinates, whereas earlier
it seems like everything was in polar coordinates. We should find a way to
make all this clear.]

We first generate dataset $\mathcal{D}_{\rm real}$ for each cable using real
trajectories, by grid sampling $\theta_{1}$, $\theta_{2}$, $r_{2}$ and $\psi$.
Collecting data for 200 trajectories takes approximately one hour, motivating
the need for more efficient data generation in simulation.

[Raven: will revise according to Prof.G’s note 0301]To facilitate data-
efficient training in simulation, we take inspiration from Hindsight
Experience Replay (HER) [HER_2017] by executing an action and pretending a
target was at the final endpoint location. For generating training data
$\mathcal{D}_{\rm sim}$ in PyBullet simulation, we sample each parameter
$\theta_{1}$, $\theta_{2}$, $r_{2}$, and $\psi$ independently from a uniform
distribution. The uniform distribution bounds for $\theta_{1}$, $\theta_{2}$,
$r_{2}$ and $\psi$ are set to be as wide as possible such that the end
effector stays within a reachable, collision-free bounding area during the
trajectory. We then execute the action in a simulator tuned using real data as
discussed in Section V, record the corresponding cable endpoint location after
the action terminates and store it as a transition $(\mathbf{a},\mathbf{s})$.
We generate 36,000 such transitions and train $f_{\rm forw}$ to predict
$\mathbf{s}$ given $\mathbf{a}$ on the simulated transitions. [Daniel: please
formalize this network with math notation somewhere, also people who don’t
know HER will likely not understand.] We transfer the trained dynamics model
to real by executing 200 grid-sampled actions again in the physical setup,
recording the transitions and fine-tuning the dynamics model using those
transitions.

Taking advantage of the symmetrical aspect of the problem, actions in all
datasets are sampled such that $\theta_{1}<0$, $\theta_{2}>0$, and $\psi>0$,
to obtain targets on the right of the workspace axis of symmetry. If
$(\theta_{1},\theta_{2},r_{2},\psi)$ produces target $(r_{d},\theta_{d})$,
$(-\theta_{1},-\theta_{2},r_{2},-\psi)$ will produce $(r_{d},-\theta_{d})$.
Therefore, during evaluation, for targets $(r_{d},\theta_{d})$ on the left of
the axis, we take $f_{\rm
inv}(r_{d},-\theta_{d})=(\hat{\theta}_{1},\hat{\theta}_{2},\hat{r}_{2},\hat{\psi})$
and output $(-\hat{\theta}_{1},-\hat{\theta}_{2},\hat{r}_{2},-\hat{\psi})$.

##  VII Experiments

In this section, we describe the experiments we perform in PyBullet simulation
and on a physical UR5 robot.

###  VII-A Baseline Methods

We benchmark performance of the forward dynamics model trained on (1)
$\mathcal{D}_{\rm real}$ only, (2), $\mathcal{D}_{\rm sim}$ only, and (3)
$\mathcal{D}_{\rm sim}$ first, then fine-tuned on $\mathcal{D}_{\rm real}$.
Along with these three forward models, we test with two baseline methods, one
analytic and one learned:

Polar Casting. Given a target cable endpoint location $(r_{d},\theta_{d})$,
first perform a casting motion that causes the free endpoint to land at
$(r_{\rm max}+r_{c},\theta)$, where $r_{\rm max}$ is the maximal distance from
the base to the robot wrist, and $r_{c}$ is the cable length. To perform this
motion, the robot rotates $\theta_{d}$ radians, and executes a predefined
casting motion. After the casting motion, the robot slowly pulls the cable
toward the base for a distance of $r_{\rm max}+r_{c}-r_{d}$. Targets
$(r_{d},\theta_{d})$ achievable by this method are limited to the circular
segment seen in figure 5. In Cartesian coordinates, the arm is limited to a
minimum $y$ coordinate of $$0.466\text{\,}\mathrm{m}$+r_{c}$ to prevent the
end effector from hitting the robot’s supporting table, where
$0.466\text{\,}\mathrm{m}$ is the distance from the center of the robot base
to the edge of the table.

[Lawrence: I think this polar casting should be mentioned way earlier in the
paper as a motivation for our method, and here bring it up again to remind the
readers that our method indeed verifies our hypothesis that our method address
the limitation of naive casting. We need motivation for why we choose these as
baselines.] [Daniel: Yes, show coverage. Also agree with Lawrence’s comment
here]

Forward Dynamics Model learned with Gaussian Process. While polar casting
serves as an analytic baseline, we choose a Gaussian process model as a
learning baseline to compare with the proposed deep neural network model.
Gaussian process is a commonly used model in motion planning and machine
learning [Gaussian_process_ML]. We train a Gaussian process regressor to
predict the cable endpoint location $(\hat{x},\hat{y})$ given input action
$\mathbf{a}$ using the previously generated 36,000 transitions training data.

###  VII-B Physical Setup

We use a UR5 robot with three cables of different stiffnesses and diameters
(Figure 4). Each cable has a weighted attachment (a hook or a magnet
[Lawrence: see overleaf comment, are we using it as a magnet?]) to resemble a
home or industrial setting in which the passing of the cable sets up a
downstream task such as plugging into a mating part. [Daniel: Do we have a
citation? I think this justification needs improving.] [JI: I reworded to be
less specific, do you think it needs a citation now?]

Figure 4:  Three cables used in physical experiments, consisting of two
polyester ropes and one braided iron wire. A mass is attached to each cable
endpoint. From top to bottom, the cable lengths are $0.62\text{\,}\mathrm{m}$,
$0.65\text{\,}\mathrm{m}$, and $0.67\text{\,}\mathrm{m}$ without the
attachment, and $0.67\text{\,}\mathrm{m}$, $0.68\text{\,}\mathrm{m}$, and
$0.68\text{\,}\mathrm{m}$ with the attachment. The cable diameters are
$11\text{\,}\mathrm{m}\mathrm{m}$, $1.5\text{\,}\mathrm{m}\mathrm{m}$, and
$7\text{\,}\mathrm{m}\mathrm{m}$.

We use a flat table of length $2.75\text{\,}\mathrm{m}$ and width
$1.50\text{\,}\mathrm{m}$ in front of the UR5 robot. To avoid the hooks
damaging the table, we cover the entire table with foam boards. To collect
training data efficiently, we attach a Logitech Brio 4K webcam on the ceiling
directly above the table to record the cable motion videos and the target
images. The webcam operates at 60 frames per second with 1920$\times$1080
video resolution. We attach a green marker on the end of each cable to track
the motion of the cable. The experimental setup is shown in Figure 1.

###  VII-C Experiment Protocol

For each cable, we compare performance on 32 fixed target locations. [Daniel:
Will fix, Prof G asked this section to be modified in 3/1 feedback, we
probably don’t want to say targets are images since we really take a point in
the image…] Each target location is extracted from an overhead image of the
workspace with a green marker indicating the desired cable endpoint location.
We sample the target locations symmetrically and to cover most of the
workspace, as shown in Figure 7.

####  VII-C1 System Parameter Selection

Figure 5:  Plots showing the area on the workspace plane that is reachable by
the cable endpoint across sampled trajectories for each system parameter
combination in simulation, where parameters were tuned to match Cable 1. The
reachable area for the cable endpoint is defined by a semicircle with radius
$r_{\rm max}+r_{c}$. Each plot is generated by executing grid sampled actions
$(\theta_{1},\theta_{2},r_{2})$ with $\theta_{1}<0$ and $\theta_{2}>0$ in the
simulator, reflecting endpoints across the vertical axis due to symmetrical
actions, and applying a concave hull to the coordinates of the endpoints. All
planar actions can achieve coverage beyond the area reachable by polar
casting. [Vincent: We should justify why we reflect the points across the x
axis] [Lawrence: TODO: Modify the caption.]

To find the best $(r_{0},v_{\rm max})$ combination, we perform a grid search
in simulation and real, with the goals of maximizing endpoint coverage and
repeatability. We try $r_{0}\in\\{0.6,1,2\\}$ and $v_{\rm
max}\in\\{1.2,1.8\\}$ for $2\times 3=6$ combinations. For each combination of
$r_{0}$ and $v_{\rm max}$, we generate actions
$\mathbf{a}=(\theta_{1},\theta_{2},r_{2})$ in via grid sampling, choosing the
bounds for each parameter such that the trajectory does not violate any joint
limits. We generate $15\times 15\times 15$ actions with $15$ values for each
parameter and execute them in the simulator to evaluate the coverage of the
end points.

Aggressive actions tends to give higher coverage, however generally bring more
uncertainty resulting in less repeatability. We evaluate the _repeatability_
of the actions by repeating the same action given by the model five times in
physical environment and calculate the variance of the distance between the
end position of cable and the mean position.

We pick the combination of $r_{0}$ and $v_{\rm max}$ with maximizing coverage
while maintaining repeatability as our final setting.

With the best $r_{0}$ combination, we further validate the importance of the
additional wrist rotation parameter $\psi$ for $v_{\rm
max}\in\\{1.2,1.5,1.8\\}$. We generate $15\times 15\times 15$ actions for
$a=(\theta_{1},\theta_{2},r_{2})$ and $10\times 10\times 10\times 5$ actions
for $a=(\theta_{1},\theta_{2},r_{2},\psi)$ and execute them in real to
evaluate the repeatability.

####  VII-C2 Model Evaluation

After selecting the most ideal system parameters $(r_{0},v_{\rm max})$ for
coverage and repeatability, we use actions generated with these fixed values
to learn a forward dynamics model as in Section VI-B. To achieve the task of
manipulating the cable such that the endpoint goes to the target position on
the right side of the workspace, we sample 50,000 candidate actions using a
grid sample on the joint-limit-abiding action parameters, and select the
action that minimizes the $L^{2}$ distance between the predicted cable
endpoint given by the learned dynamics model and desired cable endpoint. For
target positions on the other side, we flip the target position to the right
side and flip the action.

Since some edge target positions require more stochastic actions to arrive, we
evaluate the _repeatability_ of the target locations by repeating the same
action output by the model three times and calculate the mean and variance of
the distance between the end position of cable and the target position. The
median and variance distance of each target position are reported.

##  VIII Results

In this section, we report results for dynamic planar cable manipulation in
simulation and real.

###  VIII-A System Comparison

We compare the performance of models across various $(r_{0},v_{\rm max})$
values and perform an ablation study for including $\psi$ in the action (see
Equation 3).

####  VIII-A1 Coverage

From simulation, we find that $r_{0}=0.6,v_{\rm max}=1.8$ provides the maximal
coverage of $74\%$ of the reachable workspace, defined to be a semicircle with
radius $r_{\rm max}+r_{c}$. Larger $r_{0}$ and smaller $v_{\rm max}$ result in
a decrease in coverage. This is visualized in the top of Figure 5. The bottom
of Figure 5 compares the coverage with and without $\psi$ using $r_{0}=0.6$.
From the graph, we can see that adding $\psi$ notably increases coverage,
confirming our hypothesis in Section VI-A. Moreover, both $v_{\rm max}=1.5$
and $v_{\rm max}=1.8$ provide sufficient coverage to reach targets on the
workspace. The coverage map in simulation is shown in Figure. 5.

####  VIII-A2 Repeatability

We observe that $v_{\rm max}=1.2$ produces the most repeatable actions with an
average standard deviation of $8.9\text{\times}{10}^{-3}\text{\,}\mathrm{m}$
compared to $3.4\text{\times}{10}^{-1}\text{\,}\mathrm{m}$ for $v_{\rm
max}=1.8$, but offers minimal coverage $22.3\%$. Adding $\psi$ increases the
standard deviation by 40% - 115%, but including $v_{\rm max}=1.5$ with $\psi$
is still more repeatable than $v_{\rm max}=1.8$ without $\psi$. We thus choose
$r_{0}=0.6$, $v_{\rm max}=1.5$ to maximize coverage while maintaining
repeatability, and include $\psi$ to increase coverage by $198\%$.

Figure 6:  Repeatability analysis for cable 1 where repeatability is represented by normalized Gaussian distributions. The position of each point represents the average endpoint location across 5 trials with the same action, and the color (i.e., intensity of the distribution) represents the standard deviation across the 5 trials, expressed in percentages of cable length (0.62 m).  Table II:  Physical evaluation results of the 2 baselines and $f_{\rm inv}$ on cable 1. All distances are in meters. [Jonathan: Also show distances in units where 1 unit = radius of the reachable semicircle] [Lawrence: For relative error, are we using the cable length or cable length + robot arm length?] [Jonathan: cable length + robot arm length] Model | Median | Std. | Min | Max | % err.
---|---|---|---|---|---
Polar Casting | 0.267 | 0.217 | 0.024 | 0.700 | 17.1 %
Model GP | 0.180 | XXX | 0.035 | 0.979 | 11.5 %
$f_{\rm inv}$ $\mathcal{D}_{\rm real}$ | 0.366 | 0.165 | 0.051 | 0.724 | 23.5 %
$f_{\rm inv}$ $\mathcal{D}_{\rm sim}$ | XX | XXX | XX | XX | XX %
$f_{\rm inv}$ $\mathcal{D}_{\rm sim}+\rm finetune$ | XX | XX | XX | XX | XX %

###  VIII-B Physical Evaluation Results

We evaluate 3 models on Cable 1, which are trained on only $\mathcal{D}_{\rm
real}$, only $\mathcal{D}_{\rm sim}$, and $\mathcal{D}_{\rm sim}$ with fine-
tuning on $\mathcal{D}_{\rm real}$. We find that training on $\mathcal{D}_{\rm
sim}$ and fine-tuning on $\mathcal{D}_{\rm real}$ yields the best performance.
We generate $\mathcal{D}_{\rm real}$ and $\mathcal{D}_{\rm sim}$ using system
parameters $r_{0}=0.6,v_{max}=1.5$. The physical evaluation results for 32
targets are summarized in Table III. We find that the best model produces
actions that have a median distance of $0.143\text{\,}\mathrm{m}$ from the
target. Compared to the baselines, the inverse model has a lower median
distance to the target. The GP model has a low median distance, but is highly
inaccurate for certain target points with a maximum distance of
$0.979\text{\,}\mathrm{m}$. Although polar casting is accurate for targets
within its reachability, the limited coverage greatly reduces its median
distance. $f_{\rm inv}$ $\mathcal{D}_{\rm real}$ performs particularly poorly,
most likely as a result of overfitting to the small size of $\mathcal{D}_{\rm
real}$, supporting the need for $f_{\rm inv}$ $\mathcal{D}_{\rm sim}$. Fine-
tuning on 200 real examples is also shown to improve accuracy. [Jonathan:
Include error heatmap] [Daniel: TODO adjust the numbers, we need FAR more
analysis here, also qualitative results, failure cases (next subsection),
etc.]

Table III:  Physical evaluation results of 3 cables (see Figure 4) with 32 target positions for each cable. The model used is $f_{\rm inv}$ $\mathcal{D}_{\rm sim}+\rm finetune$. Min and Max show the minimum and maximum distance between end point and targets among 32 targets. [Jonathan: Distances are normalized by cable length] [Lawrence: The standard deviation I filled in is across targets, not between trials for each target.] Cable | Median | Std. | Min | Max | % err.
---|---|---|---|---|---
Cable 1 | 0.143 | 0.115 | 0.024 | 0.466 | XX
Cable 2 | 0.159 | 0.111 | 0.041 | 0.478 | XX
Cable 3 | 0.226 | 0.106 | 0.068 | 0.458 | XX
Figure 7: The 32 target positions on the workspace plane, shown with small
tape. The target positions are chosen to cover most of the reachable space to
thoroughly evaluate the model performance. The grid lines shown are 0.5 m
apart. There are 5 trials of a successful case, marked by the star target, and
5 trials of a failure case, marked by the red cross. The successful case has
median endpoint deviation of 0.05 m, while the unsuccessful case has a median
endpoint deviation of 0.33 m.

####  VIII-B1 Repeatability

We evaluate the repeatability for all 32 target positions for all cables, by
repeating each output action 5 times per target. The results are shown in
Figure 6. We can see points with higher standard deviation also have higher
error implying those points are more difficult to reach. $f_{\rm inv}$
$\mathcal{D}_{\rm sim}$ has a lower standard deviation than the baselines and
$f_{\rm inv}$ $\mathcal{D}_{\rm real}$, suggesting the model is able to learn
and favor motions that are more reliably able to reach the target. $f_{\rm
inv}$ $\mathcal{D}_{\rm sim}+\rm finetune$ can achieves consistent
repeatability across the 3 cables.

Figure 8:  Repeatability analysis for cable 1 where repeatability is
represented by normalized Gaussian distributions. The position of each point
represents the average endpoint location across 5 trials with the same action,
and the color (i.e., intensity of the distribution) represents the standard
deviation across the 5 trials, expressed in percentages of cable length (0.62
m).

####  VIII-B2 Error Analysis

Certain targets can be consistently reached with lower distances to target
than others, as seen in Figure 8 [Lawrence: The figures are WIP]. We attribute
the error to two errors, learning error and sim-to-real error. Evaluating
$f_{\rm inv}$ $\mathcal{D}_{\rm sim}$ on the same 32 targets in simulation
achieves a median distance of $0.054\text{\,}\mathrm{m}$, which represents the
difficulty of learning the forward dynamics of a complex cable via a neural
network. The sim-to-real error is demonstrated by Table I, however, while the
sim-to-real gap was least prevalent for Cable 3, it is surprising that Cable 3
actually has the lowest performance. We conjecture that the cylindrical shape
of the endpoint mass contributes to increased stochasticity and is thus more
difficult to model, leading to a higher median distance than the other two
cables. [Jonathan: Describe qualitative examples of failure cases] While we
modeled the extra mass at the endpoint, we may consider modeling irregular
shapes to decrease the sim-to-real gap. [Lawrence: Not sure if we should put
in another error heatmap or just use words to describe.]

##  IX Conclusion and Future Work

[Daniel: Need to give better conclusion overview]

Experimental results suggest that a data-driven approach via simulator tuning
is a promising for learning dynamic planar cable manipulation. The self-
supervised data collection in a combination of simulated and real-world
environments provides a procedure for efficiently adapting to cables with
different properties.

When tuning the simulation parameters, we choose our optimization metric to be
the 2D Euclidean distance from the simulated free end to the ground truth
endpoint location at discrete time steps, assuming that endpoint location is
sufficient to find parameters that model the cable. In future work, we plan to
relax this assumption and explore additional ways to better tune the
simulator.

The fine-tuning process relies on executing hundreds of trajectories in the
physical setup, which may be infeasible in many other dynamic manipulation
domains. Recently, dynamics randomization [dynamics_randomization_2018] has
shown to aid in effectively transfer polices to real. Pre-training the learned
model with trajectories executed with random dynamics could provide better
initialization for transfer than a model trained on only a single set of
parameters. Previous dynamics randomization literature has found that
recurrent polices transfer more effectively with dynamics randomization than
memoryless feedforward neural networks. However, as the current action
parameterization is output in a single step, a recurrent network may not be
applicable.

We also note that this method relies on learning a separate set of simulation
parameters and forward dynamics model for each cable. Another approach we will
consider is a meta-learning approach such as MAML [maml], which may assist
with rapid adaptation to other cables to avoid learning parameters for each
cable.

## Acknowledgments

This research was performed at the AUTOLAB at UC Berkeley in affiliation with
the Berkeley AI Research (BAIR) Lab, the CITRIS “People and Robots” (CPAR)
Initiative, and the Real-Time Intelligent Secure Execution (RISE) Lab. The
authors were supported in part by donations from Toyota Research Institute and
by equipment grants from PhotoNeo, NVidia, and Intuitive Surgical. Daniel
Seita is supported by a Graduate Fellowship for STEM Diversity (GFSD). We
thank our colleagues who provided helpful feedback, in particular Ashwin
Balakrishna, Ellen Novoseller, and Brijen Thananjeyan.

This appendix is structured as follows:

  * •

In Appendix A we discuss the proposed method in further detail.

  * •

In Appendix B we present experiment details.

  * •

In Appendix C we describe additional results.

##  Appendix A Additional Method Details

Here we describe additional details of the method. [Daniel: Maybe describe
things like how we tuned things, etc. Could also describe how errors propagate
throughout time as in Figure 9 but adapted to our setting.]

###  A-A Simulator Tuning via Differential Evolution

Table IV:  PyBullet simulation parameters for dynamic cable manipulation tasks across all cables we test in physical experiments (see Figure 4). The stiffness, friction, and damping constants are unit-less, and the mass per segment is in kilograms. [JI: We should minimally move this to the experiments section. We may also consider moving the table to supplemental material. While its important for our experiments, it doesn’t tell the reader much, other than the cables have different parameters.] PyBullet |  |  |  | Range
---|---|---|---|---
Hyperparameter | Cable 1 | Cable 2 | Cable 3 | Explored
Twist stiffness | 1.90 | 2.83 | 3.20 | $[0,5]$
Bend stiffness | 3.32 | 2.69 | 1.75 | $[0,5]$
Mass per segment | 1.66 | 1.93 | 0.60 | $[0.3,2]$
Lateral friction | 0.41 | 0.83 | 0.70 | $[0.3,2]$
Spinning friction | 0.01 | 0.01 | 0.04 | $[0,0.1]$
Rolling friction | 0.88 | 1.04 | 1.15 | $[0,1.5]$
Scaled endpoint mass | 4.34 | 4.28 | 3.52 | $[1\textrm{x},5\textrm{x}]$
Linear damping | 0.62 | 0.83 | 0.45 | $[0,1.5]$
Angular damping | 12.2 | 8.74 | 1.10 | $[1,20]$
Table friction | 0.53 | 0.14 | 0.46 | $[0,1]$

[Daniel: Starting now, rest of paragraph, move to appendix] [JI: I agree, and
I would also like to see a summary of how DE works here, and an overview of
our implementation.] We use a “best1bin” strategy to optimize for the minimum
deviation in trajectory across 80 different trajectories. In this technique,
each child parameter array is created by either choosing a parameter for its
parent array, or a combination of the current best (lowest cost) array and two
random parameter arrays, where ${\rm Unif}(0,1)>CR$ selects the parameter from
the parent array. $CR$, the crossover probability, affects the frequency at
which a child inherits a parameter from its parent, or from a mutated version
of the most fit individual. An additional parameter, $F={\rm Unif}(F_{\rm
min},F_{\rm max})$, is the differential weight. For each generation, $f\sim F$
is sampled, and a large $f$ increases the search space of the algorithm, but
decreases convergence rate. We choose DE parameters that were consistent with
Collins et al., because their results showed that DE was able to find PyBullet
parameters that were more optimal than its default settings, in relatively low
convergence time. We use $CR=0.7$, $F=0.5-1.0$, and population size of $N=1D$,
where $D$ is the number of simulation dynamics parameters. Convergence is set
to be the standard deviation of the population reaching below $1\%$ of the
population mean.

Figure 9:  [Lawrence: Maybe we can compare real vs sim as in the fly fishing
paper? It would be nice to illustrate how the errors propagate through time]
[Daniel: Why is this figure here?] [Lawrence: I was thinking that it may be
good to show the readers how our parameter tuning did by illustrating the sim
to real gap visually. But we probably don’t have time to do this anyway.]
[Daniel: Moving to appendix, I think something like this could be useful for
further investigation] Figure 10:  [Lawrence: Maybe somewhere we can
illustrate our action parameters as in the fly fishing paper?] [Daniel: Why do
we have this figure here?][Lawrence: I am talking about the figure that
Jonathan mentioned TODO. Just a placeholder, ours can be different.] [Daniel:
OK sounds good, let’s iterate upon this figure]

##  Appendix B Additional Experiment Details

Since the overhead camera is not perfectly parallel to the plane of the table,
we put four fiducial markers on the corners of the foam boards. We process the
image by using the fiducial markers to detect the corners of the table, crop
out the residual area, and apply a perspective shift to mitigate perspective
effects.

Due to the irregular shapes of the hooks, we find that the end of the cable
would bounce out in random directions when colliding with the table during the
reset tossing motion. To mitigate this uncertainty, we pull the cable back
slightly after the tossing motion to straighten the cable. In addition, the
foam boards on the table can help in uncertainty reduction.

##  Appendix C Additional Results

Figure 11:  [Daniel: Add caption, remove excess whitespace (are we using tight
layout?)] [Lawrence: Are we going to put this image in the appendix? If so, I
will re-create the image with appropriate data. This is just half of the
result as we have symmetric targets as well.] [Jonathan: This is a temporary
placeholder figure. We will update this when we obtain results for our most
recent models]

See Figure 11? [Daniel: Please describe]