# Transformers for One-Shot Visual Imitation

Sudeep Dasari
Robotics Institute
Carnegie Mellon University, USA
<EMAIL_ADDRESS>
&Abhinav Gupta
Robotics Institute
Carnegie Mellon University, USA
<EMAIL_ADDRESS>

###### Abstract

Humans are able to seamlessly visually imitate others, by inferring their
intentions and using past experience to achieve the same end goal. In other
words, we can parse complex semantic knowledge from raw video and efficiently
translate that into concrete motor control. Is it possible to give a robot
this same capability? Prior research in robot imitation learning has created
agents which can acquire diverse skills from expert human operators. However,
expanding these techniques to work with a single positive example during test
time is still an open challenge. Apart from control, the difficulty stems from
mismatches between the demonstrator and robot domains. For example, objects
may be placed in different locations (e.g. kitchen layouts are different in
every house). Additionally, the demonstration may come from an agent with
different morphology and physical appearance (e.g. human), so one-to-one
action correspondences are not available. This paper investigates techniques
which allow robots to partially bridge these domain gaps, using their past
experience. A neural network is trained to mimic ground truth robot actions
given context video from another agent, and must generalize to unseen task
instances when prompted with new videos during test time. We hypothesize that
our policy representations must be both context driven and dynamics aware in
order to perform these tasks. These assumptions are baked into the neural
network using the Transformers attention mechanism and a self-supervised
inverse dynamics loss. Finally, we experimentally determine that our method
accomplishes a $\sim 2$x improvement in terms of task success rate over prior
baselines in a suite of one-shot manipulation tasks.111For code and project
video please check our website: https://oneshotfeatures.github.io/

> Keywords: Representation Learning, One-Shot Visual Imitation

##  1 Introduction

Imitation is one of the most important cornerstones of intelligence. Watching
other humans act, inferring their intentions, and attempting the same actions
in our own home environments allows us to expand our skill set and enhance our
representations of the world [1]. On the other hand, robots - while capable of
imitating skills like table tennis [2] and driving [3] – are much less
flexible when it comes to visual imitation. Most prior work in robotic
imitation assumes that the agent is trying to acquire a single skill from
demonstration(s) collected kinesthetically [4] (i.e. a human manually guides a
robot) or via tele-operation [5]. These approaches can work so long as the
target test-time task and environment are do not significantly differ from
those seen during training. Is it possible to develop a robotic agent which
can learn to imitate without these restrictions?

Visual imitation requires extracting a higher level goal from the visual
demonstration and using the inferred goal to predict actions from pixels. But
how does one represent goal/intention and how can this contextual information
be incorporated into the policy function itself? There are three primary
approaches in prior work: the first approach is to represent goals/intentions
as pixels by generating goal images, and then inferring actions given current
observations and inferred goals [6, 7]. While this approach is intuitive and
interpretable, it is difficult to generate pixels, in a way that respects
structural differences in the image. Figure 1 shows an example with well
defined task semantics, but where a change in object positions makes it
difficult to visually map the human state to the robot environment. The second
approach has been to model visual imitation as a one-shot learning problem
[8], which can be solved with meta-learning algorithms [9]. Here, a robot is
given a single example, in the form of a video or demonstration (e.g. video +
control telemetry), and must use that information to perform new instances of
the same task. The demonstration is used to update the parameters of a policy
function and the updated policy is executed on the robot. Domain gaps can be
addressed with a learned adaptive los function [10]. While the one-shot
formalism is very useful, estimating policy parameters from a single example
can be an extremely difficult problem and prone to over-fitting.

In this paper, we explore a third alternative: task-driven features for one-
shot learning. We process both observations from the target agent and
demonstrations frames from a ”teacher” agent in order to extract context-
conditioned state representations. What neural network architectures can
create task-driven features? While in the past, approaches such as LSTMs have
been used, in this work, we focus on self-attention architectures. In
particular, the Transformers architecture - while simple - has seen broad
success in NLP [11] and Vision [12] tasks. Furthermore, using attention for
control tasks has has basis in biology and psychology. Indeed, humans use
attention mechanisms to create context driven representations [13], and
directly supervising policies with human attention can dramatically increase
task performance [14].

In this paper, we propose using transformers [11] (or non-local self-attention
modules [12]) to extract relational features which act as input state vectors
for the policy function. Our transformers take as input both the spatial
ResNet Features from teacher demonstration and the target agent. This allows
the policy to automatically adapt its features to the task at hand, by using
context frames to focus only on important task-specific details. For example,
in Figure 1 the robot could use human context frames to focus only on relevant
details like the red block’s location, and entirely ignore distracting
elements like the table’s leg. However, transformer features could easily end
up improperly weighting important details during test time. We propose to
solve this issue by further supervising the state representation learning with
an unsupervised inverse dynamics loss. This loss constrains the learning
problem and ensures the final representations can model the underlying
dynamics, as well as task specific details. Ultimately, our method achieves
significant improvements over one-shot imitation learning baselines on a suite
of pick and place tasks: our final policies demonstrate a $2$x performance
gain and can match baseline performance with $3$x fewer data-points.

Figure 1: What should the robot do given video from another demonstration
agent? A human would immediately know to place the red triangle on the blue
square, and can use their past experience to execute the task. Is it possible
to teach a robot to do the same?

##  2 Related Work

Learning from Demonstration (LfD) is a rich and diverse field of study which
focuses on enabling robots to learn skills from human or other expert
demonstrations. A thorough review is out of scope for this paper, so we gladly
refer the reader to survey articles [15, 16, 17]. Of prior work, Behavior
Cloning (BC) [18, 19], a common formulation of LfD, is most related to our
project. BC involves imitating an expert agent given a set of trajectories
(a.k.a time series of observations and actions), by fitting a function which
approximates the expert’s action in a given state. This simple formulae has
proven successful in imitating a wide range of behaviors from visual inputs,
including robotic manipulation tasks [20] and driving [21]. These methods have
been extended to situations where expert observations are present without
action labels [22], including prior work which linked this problem to inverse
dynamics minimization [23]. However, both of these approaches require the
demonstration agent match the imitator.

BC algorithms often assume that they are approximating a single state
conditioned policy. In an environment with multiple tasks or multiple
variations of the same task, this constraint can be limiting. Work on goal
conditioned imitation learning seeks to relax these assumptions by allowing
for policies which condition on a goal variable alongside the current state,
and adjust their behavior accordingly. There are myriad ways to introduce goal
conditioning, including with the robot’s state [24], ”goal” images of the
final state [25, 26, 27], natural language [28], and video or images of humans
[29, 30]. In our project, we assume the robot has a single video of another
agent (be it another robot or a human) doing a task, and must complete that
same task itself using past experience. This is a specific instance of the
one-shot learning problem [8], and has been investigated before previously
using meta-learning with an adaptive loss [10]. Instead of using meta-
learning, we propose to attack this problem with an attention mechanism over
image frames.

A challenge in this line of work is learning visual representations which can
enable the robot to deduce the task from video of another agent _and_ perform
the task itself. Work in computer vision demonstrated that deep neural
networks are capable of learning such flexible representations for action
recognition [31] and state estimation [32], but often require large image
datasets to fully train. Unfortunately, collecting ImageNet [33] scale
datasets on robotics platforms is prohibitively expensive, due to the cost of
continuous robot operation and hardware fragility. Work in self-supervised
learning [34, 35, 36] offers a glimmer of hope, by showing how large and
(relatively) cheap sets of unlabelled images can be used to learn expressive
and useful representations for other downstream tasks. These representations
could be used directly as reward functions [37, 38], but it can be very
difficult to define rewards for a suite of tasks. Instead, unsupervised
learning techniques alongside simple data augmentation can be used to increase
data efficiency when directly acquiring policies with reinforcement learning
[39, 40, 41]. Even simpler self-supervised losses - like inverse modelling
(i.e. predicting action between two sequential states) - can be used to learn
robust policies which adapt to new environments [42]. Our goal in this project
is to apply these insights in representation learning to the one-shot
imitation learning problem.

##  3 Our Method

###  3.1 Problem Definition

Our method follows prior work [9, 10], and formalizes the one-shot imitation
learning problem as supervised behavior cloning on a data-set of tasks. For
each task $\mathcal{T}$ (e.g. place blue bottle in bin), we have several
demonstration videos and target trajectories. Note that the demonstration
videos and target trajectories are semantically similar tasks but could have
different starting/end states. We represent each demonstration video as
$v_{i}$ and each target trajectory, $t_{i}$, as a temporal sequence of
observations ($o$) and actions ($a$). Hence,
$t_{i}=\\{(o_{i}^{(1)},a_{i}^{(1)}),\dots,(o_{i}^{(k)},a_{i}^{(k)})\\}$.

Models are trained on a dataset of tasks
$\mathcal{D}=\\{\mathcal{T}_{1},\dots,\mathcal{T}_{n}\\}$. During test time,
new test tasks - $\mathcal{T}_{test}$ \- are sampled which the model must
successfully control the imitator agent to perform. Thus, all methods are
evaluated on task success rates in held out environments. Our setup is
challenging because: (a) morphological differences between demonstration and
target agent (e.g. one is human and other is robot arm); (b) missing
correspondence between demonstration videos and target trajectories.

Figure 2: Our method uses a Transformer neural network to create task-specific
representations, given context and observation features computed with
ResNet-18 (w/ added positional encoding). The attention network is trained
end-to-end with a behavior cloning loss, an inverse modelling loss, and an
optional point loss supervising the robot’s future pixel location in the
image.

###  3.2 Feature Learning with Transformers

Given video context from a demonstrator agent and image frames from the test
environment, our representation module must deduce relevant features and
efficiently pass them on to later stages of the pipeline for action
prediction. For example, when given a video of a green bottle being dropped in
a bin, the vision module should detect and represent the green bottle in its
own environment while ignoring other distracting objects. We propose to learn
this mechanism end-to-end using self-attention Transformer modules [11], in
the hope that this powerful inductive bias helps the policy perform tasks
successfully.

Before the attention module, individual images from both the context video and
current state are passed through a ResNet-18 architecture [43], and spatial
features (size $[512,T,H,W]$) are collected before the average pooling step.
At this stage, the features are flattened (size $[512,T*H*W]$) and sinusoidal
positional encodings [11] are added to the tensor (i.e. time and space treated
as single dimension). These embeddings can allow neural networks to represent
higher frequency functions [44], and we empirically found that they were
crucial to preserving spatial and temporal information in the attention
module. After adding positional encodings, the features are reshaped to their
original size.

Next, the non-local multi-head attention operator is applied to the input
tensor. We adopt a specific implementation of the Transformers self-attention
module presented in Wang et al. [12], which we augment with multi-headed self-
attention [11]. First, the module generates Key, Query, and Value tensors by
applying three separate 3D spatio-temporal convolutions (we use kernel size
$k=1$) with ReLU activation to the input tensor. To be clear, each convolution
layer’s input and output are $[d,T,H,W]$ tensors, where $d$ is the
Transformer’s embedding size. These generated key, query, and value tensors
are then flattened and projected down $n$ separate times - once for each
attention “head” - before attention is applied (final shape per head
$[d,T*H*W]$). The self-attention operator is applied to each head
individually. Considering attention head $j$, temperature parameter $\tau$,
and projected tensors $K_{j},Q_{j},V_{j}$, this amounts to:

$A_{j}=\textbf{softmax}(K_{j}^{T}Q_{j}/\tau)\hskip
22.76219ptV_{j}^{(out)}=V_{j}A_{j}$

The individual attention heads are then concatenated together channel-wise,
and then projected back to the original $512$ dimension size with another 3D
convolution
($O=\textbf{Conv3D}(\textbf{concat}[V_{1}^{(out)},\dots,V_{n}^{(out)}])$).
Note that this multi-head attention operator can be implemented with little
overhead using batched matrix multiplication. Dropout [45], then a residual
connection, and finally batch normalization [46] are applied to get the final
output $f(x)=\textbf{batchnorm}(x+\textbf{dropout}(O))$, with final size
$[512,T,H,W]$. In order to appropriately apply this to behavior cloning (where
$o_{t+1}$ is not known during test time), we make this operation causal by
appropriately padding the 3D convolution operators and masking the attention.

###  3.3 Goal Conditioned Behavior Cloning

As discussed previously, our objective is to learn a policy
$\pi(a_{t}|o_{1:t},v)$ which ingests the current (or optionally all previous)
state observations alongside a context video, and predicts a distribution over
possible actions the expert policy would select. We process the input video
stream with stacked attention modules to yield fixed size spatial features,
with one feature map per time-step. The features are projected down to a fixed
size representation vector using a spatial softmax operator [47], followed by
a multi-layer perceptron with ReLU activations, and finally L2 normalization
to unit length. This representation $\phi_{t}=F(o_{1:T},v)$ is used for action
prediction.

Multi-Modal Action Prediction: One of the most naive ways to predict
$\pi(a_{t}|o_{1:t},v)$ from $\phi_{t}$ is to simply parameterize the policy as
a normal distribution
$\pi(a_{t}|o_{1:t},v)=\mathcal{N}(\mu(\phi_{t}),\sigma(\phi_{t}))$, and to
sample actions from that. However, this approach can run into severe
limitations when the real expert distribution is multi-modal. Consider a robot
attempting to top-down lift a cup by its handle. Rotating the gripper by 90°or
-90°, but not rotating at all (i.e. the mean action) would result in task
failure since the gripper would close on top of the handle. Prior work [20,
48, 26] showed this limitation matters in practice, and rectifies the
situation by predicting a mixture of uni-modal distributions. We adopt the
same solution used by Lynch et al. [26]. First, we discretize the action space
(discussed in detail in Section 4.1) and then parameterize the policy as a
discretized logistic mixture distribution [49]. For each timestep, we predict
$k$ logistic distributions with separate mean and scale, and form a mixture by
convexly weighting them with vector $\alpha$. The behavior cloning training
loss is simply negative log-likelihood for this distribution:

$\mathcal{L}_{BC}(\mathcal{D},\theta)=-\ln(\Sigma_{i=0}^{k}\hskip
2.84526pt\alpha_{k}(\phi_{t})\hskip
2.84526ptP(a_{t},\mu_{i}(\phi_{t}),\sigma_{i}(\phi_{t}))$

Where,
$P(a_{t},\mu_{i}(\phi_{t}),\sigma_{i}(\phi_{t}))=F(\frac{a_{t}+0.5-\mu_{i}(\phi_{t})}{\sigma_{i}(\phi_{t})})-F(\frac{a_{t}-0.5-\mu_{i}(\phi_{t})}{\sigma_{i}(\phi_{t})})$
and $F(\cdot)$ is the logistic CDF. During test time, actions are simply
sampled from the distribution and executed on the robot without rounding. For
most of our experiments, the model performed best when using two mixture
components and learned constant variance parameters per action dimension.

###  3.4 Inverse Model Regularizer

Our method also adds a self-supervised inverse modeling objective to act as a
regularizer to the behavior cloning loss during training. Context and
trajectory snippets are sampled from the dataset, and images in them are
randomized with sampled translations, color shifts, and crops. This
randomization is applied consistently to frames from the context video,
whereas images from the agent’s observation stream (a.k.a trajectory images)
are randomized individually. This randomized image stream is passed through
the attention and representation modules to generate $\tilde{\phi_{t}}$. The
representations $\tilde{\phi}_{t}$ and $\tilde{\phi}_{t+1}$ are used to
predict a discretized logistic mixture distribution over intermediate actions.
Thus, the inverse loss is:

$\mathcal{L}_{INV}(\mathcal{D},\theta)=-\ln(\Sigma_{i=0}^{k}\hskip
2.84526pt\alpha_{k}(\tilde{\phi}_{t},\tilde{\phi}_{t+1})\hskip
2.84526pt\text{logistic}(\mu_{i}(\tilde{\phi}_{t},\tilde{\phi}_{t+1}),\sigma_{i}(\tilde{\phi}_{t},\tilde{\phi}_{t+1})))$

We share parameters between the behavior cloning and inverse modeling
objectives for the attention module, representation module, and distribution
prediction heads (i.e. after first layer). In practice, we use the randomized
image stream for both tasks as well, in order to minimize memory consumption.

###  3.5 Point Prediction Auxiliary Loss

Finally, our model uses $\phi_{t}$ to predict a 2D keypoint location
corresponding to the location of the gripper in the image $H$ timesteps in the
future. Ground truth for this auxiliary loss is easy to acquire given either a
calibrated camera matrix or object detector trained on the robot gripper. One
could instead predict the 3D gripper position in world coordinates if neither
is available. While not strictly needed for control, this loss is very
valuable during debugging, since it lets us visually check during training if
the model understand where the robot ought to be $H$ timesteps in the future.
The point prediction is parameterized with a simple multi-variate 2D normal
distribution $\hat{p}_{t+H}\sim\mathcal{N}(\mu(\phi_{t}),\Sigma(\phi_{t}))$
with loss
$\mathcal{L}_{pnt}(\mathcal{D},\theta)=-\ln(\text{likelihood}(p_{t+H},\hat{p}_{t+H}))$.
Thus, the overall loss for our method is:

$\mathcal{L}(\mathcal{D},\theta)=\lambda_{BC}\hskip
2.84526pt\mathcal{L}_{BC}(\mathcal{D},\theta)+\lambda_{INV}\hskip
2.84526pt\mathcal{L}_{INV}(\mathcal{D},\theta)+\lambda_{pnt}\hskip
2.84526pt\mathcal{L}_{pnt}(\mathcal{D},\theta)$

##  4 Experimental Results

Our model is evaluated on robotic manipulation tasks - namely pick and place
tasks - in simulation using multi-agent MuJoCo [50] environments. Our
evaluations investigate the following questions: (1) can our model perform new
task instances (defined in 4.1) previously unseen during training? And (2)
what components (e.g. inverse loss, etc.) are most crucial for successful
control?

Figure 3: Our base environment is adopted from RoboTurk [51]. The $16$ tasks
consist of taking an object (a-b) to a bin (1-4). Top robot is agent and
bottom is demonstrator.

###  4.1 Simulation Environment and Tasks

Environment Description: The environments we use are modified variants of
those originally presented in RoboTurk [51]. Visually, the base environment \-
shown in Figure 3 \- is the exact same as the original from RoboTurk, except
the object meshes are replaced with primitive geometric types (e.g. boxes and
cylinders) in order to improve simulation contact stability and run-time. This
modification results in only minor visual differences. In order to investigate
visual imitation across agent morphology, we use duplicate versions of the
environment with two visually distinct robots. The Sawyer robot (red robot in
Figure 3) provides demonstration videos and the Panda robot (white robot in
Figure 3) acts as the agent which our model must control. Both environment’s
action spaces are modified to support end-effector control. Given a target
$x,y,z$ position, rotation in axis-angle form, and gripper joint angle the
environment solves for desired robot joint angles with inverse kinematics and
sends joint velocities to the robot using a simple PD controller. Thus, the
final action space consists of a target pose discretized into 256 independent
bins per dimension in order to support our behavior cloning loss. It’s
important to note that the demonstrations we train on do not cover the whole
state space, so the robot is mostly constrained to 3-DOF movement.

Task Definition: A “task instance” consists of picking an object from a
specific start location - uniformly distributed on the table in Fig. 3 \- and
placing the object in one of the four bins on the right. Task instances are
grouped into “tasks” based on shared properties. For example, picking a milk
carton (from Fig. 3) and placing it into bin 1 is a task, and different task
instances are constructed by changing the carton’s start position. This
precise definition allows us to collect a suite of train task instances, train
models on that data, and test generalization to new task instances.

Data Collection Methodology: Training data is collected using an expert pick-
place policy (built using privileged information from the simulator) in the
target environment(s). For each task ($\mathcal{T}$) we repeatedly, sample a
demonstration video ($v_{i}$) by executing the expert policy on the Sawyer
robot, then shuffle the objects, and sample an expert trajectory ($t_{i}$) by
executing the expert policy on the Panda robot. This way a dataset of tasks is
formed from individual task instances.

###  4.2 Baseline Comparisons

Model |  Reaching Success |  Picking Success |  Placing/Overall Success
---|---|---|---
Our Method | $\textbf{99.4\%}\pm 1.2\%$ | $\textbf{92.5\%}\pm 4.1\%$ | $\textbf{88.8\%}\pm 5.0\%$
Contextual-LSTM | $38.8\%\pm 7.6\%$ | $26.3\%\pm 6.9\%$ | $23.8\%\pm 6.7\%$
DAML [10] | $36.9\%\pm 7.6\%$ | $10.6\%\pm 4.8\%$ | $6.9\%\pm 4.0\%$
DAML Auxiliary | $47.8\%\pm 7.4\%$ | $17.8\%\pm 5.6\%$ | $13.3\%\pm 5.0\%$

Table 1: Comparison between our method and baselines in $16$ pick and place
tasks. Values indicate success rates and $95\%$ confidence intervals for
“stages”footnote 2 in the overall pick and place task.

Our investigation begins by evaluating our method’s performance in $16$ tasks
in the base environment (Figure 3). We seek to determine the robot’s physical
competency at manipulating all four objects, as well as its ability to deduce
which task it should perform from context video. A natural way to quantify
this is by breaking down the $16$ pick and place tasks into “reach,” “pick,”
and “place” stages222Reaching is defined as placing the gripper within $<0.03$
units from the target object, picking requires stably lifting the object
$>0.05$ units off ground, and placing requires putting the object in the
correct bin (a.k.a task completion), and reporting success rates on each stage
individually. Failure modes can be successfully deduced from these rates. For
example, since reaching is a physically easy task, if the robot does not reach
the object then it is likely unable to deduce the target object from the
context frames. Furthermore, if the robot reaches the object but is unable to
pick it up, its physical dexterity (or lack thereof) is likely to blame.

We collect $100$ train task instances using the methodology described
previously for each of the $16$ tasks. That amounts to $1600$ total
demonstration videos alongside $1600$ expert robot trajectories. We train our
method on the dataset and compare against the following baselines:

  * •

Contextual-LSTM: This baseline utilizes a standard Encoder-Decoder LSTM [52,
53] (augmented with self-attention [54, 55]), to first consume the context
video, and then predict actions from encoded observations. It uses the same
mixture distribution our model uses. Before LSTM processing, images frames are
embedded using a pre-trained ResNet-18 [43] neural net combined with spatial-
softmax [47] and fully-connected layers. The whole network is trained end-to-
end with a behavior cloning loss.

  * •

Domain Adaptive Meta-Learning: DAML [10] uses a learned loss function to adapt
a neural network’s parameters to perform the desired task. We used a wider
version of the network used in the original paper, since we found that using
deeper models (like ResNet-18) resulted in overfitting on this task. To
increase performance, the same discrete logistic action distribution is used.
DAML is trained end-to-end with the MAML meta-learning algorithm [56] using a
behavior cloning loss, along with explicit supervision of the pick and drop
locations.

  * •

DAML-Auxiliary: This method uses the same meta-learning model described above,
except only the predicted pick and place locations are used during test time.
Given this prediction, a grasp motion is executed in the environment using a
hard coded grasp policy.

For each of the $16$ tasks, the models are prompted to perform new task
instances (unseen during training) using freshly generated context videos.
Success rates for our method and baselines (averaged across tasks) are shown
in Table 1. As you can see, our method is the only one which can reliably
perform new task instances. Its overall success rate is double the competing
models’ reaching success rate, including the DAML-auxiliary model which makes
strong task assumptions, and the LSTM model which uses embedding level
attention. The LSTM baseline’s (which uses standard attention) relative
failure supports our hypothesis that the Transformer architecture uniquely
enables difficult visual processing. For additional experiments testing
generalization to new objects (i.e. new tasks instead of new task instances)
refer to Appendix A.1.

###  4.3 Architecture Ablation

While the our model clearly outperforms the other baselines, it is unclear if
the Transformers architecture or additional losses deserve more credit. To
test this thoroughly, the Transformers model is tested against an ablated
version of itself without the attention mechanism (i.e. just temporal-spatial
convolutions) using the same base environment comparison described before.
Furthermore, models are trained with various versions of the baseline neural
network architectures, alongside the additional loss terms. Specifically, $4$
baseline architectures are considered: $2$ of them adopt the small
convolutional network used in prior work [10, 57] either with or without an
additional LSTM [52] on top, and the other $2$ use ResNet features [43] (again
with or without LSTM). Note all architectures were tuned to maximize their own
test performance rather than to match some other metric (e.g. number of
parameters), since doing so often led to worse results for the baseline (e.g.
larger LSTMs overfit more than Transformers). Results are presented in Figure
4. The key takeaways are encouraging. First, the Transformers architecture (w/
attention) outperforms a library of other architectures for this task by large
margins, even using the same losses. Furthermore, the baselines perform better
when trained with the additional losses compared to being trained purely with
a behavior cloning loss as done before (contextual-LSTM’s success rate
improves $20\%\rightarrow 40\%$).

Figure 4: Our Transformer model is compared against other neural networks (all
trained w/ our losses and code) to determine how useful the attention
mechanism really is. The Transformer architecture outperforms all others,
including a version of itself w/out attention.

Figure 5: We compute success rate v.s number of train samples for our method
and versions with one loss excluded (all w/ Transformer). Note the model
without inverse loss is usually outperformed when compared to its peers
trained on the same data.

###  4.4 Loss Function and Method Ablations

Given that our training losses/code boosted baseline architecture performance
compared to using just behavior cloning, we now seek to test exactly which
component was most useful. It’s entirely possible that some of the additional
parts offer more utility in the ”low-data” regime where over-fitting is more
likely, and thus are less useful when more data is present. Thus, we collect
two more versions of the base environment dataset with fewer samples ($480$
and $800$ samples pairs), and train three ablations - one model without the
inverse loss, one without the point loss, and one without data augmentation -
alongside our base model on all three datasets (two new sets + original). That
results in a total of 12 models, all of which we evaluate in the same manner
as before. Overall success rates for all models are in Figure 5. Note that the
model without the inverse loss is outperformed by its counterparts in two out
of three datasets, whereas the point loss only makes a significant difference
in the smallest dataset. Indeed as the number of datapoints increases, so does
the importance of the inverse loss: the model without inverse loss is more
than $~{}25\%$ worse than its counterparts in the $N=1600$ case! While the
inverse loss clearly makes a difference, this cannot be observed as “positive
transfer” in the behavior cloning train/test loss (see Appendix A.2). This
suggests inverse loss regularization helps test time performance in ways not
captured in the training objective. Finally, conditioning our policy on
context video proved to be more effective than just feeding it the last frame,
which indicates the demonstration helps our model determine which task to
perform compared to using a “goal image” frame. For more check Appendix A.3.

##  5 Discussion

In this project we explore the one-shot visual imitation learning problem. Our
experiments highlight two technical contributions - applying the Transformers
architecture to one-shot imitation tasks and a self-supervised inverse
modelling objective - which both result in large performance gains over
baseline one-shot imitation learning approaches. More specifically, our
ablations show that our model trained without the self-supervised inverse loss
performs significantly worse when compared to other versions with the inverse
loss, and all of our Tansformers models (even without inverse loss) outperform
a Seq2Seq LSTM trained with traditional “embedding level” attention mechanisms
by roughly $2$x.

The main takeaway here is that injecting the right biases - both in terms of
network design and the loss function - can help policies perform better during
test-time. We believe that the Transformer’s attention mechanism provides such
a bias by allowing for task conditioned representations, whereas the inverse
model forces the policy to preserve information which is needed for robust
control during test time. We hope that these findings prove useful to others
working on one-shot imitation learning and goal conditioned reinforcement
learning in general.

#### Acknowledgments

We’d like to begin by acknowledging the students and collaborators at CMU who
gave valuable feedback which made the final paper much stronger. In
particular, we’d like to recognize Sam Powers, Devendra Chaplot, Kenneth
Marino, Adithya Murali, and Shubham Tulsiani. Finally, this research was
funded by ONR MURI, the ONR Young Investigator Award to Abhinav Gupta and the
DAPRA MCS award.

## References

  * Vogt and Thomaschke [2007] S. Vogt and R. Thomaschke.  From visuo-motor interactions to imitation learning: behavioural and brain imaging studies.  _Journal of Sports Sciences_ , 25(5):497–517, 2007.
  * Mülling et al. [2013] K. Mülling, J. Kober, O. Kroemer, and J. Peters.  Learning to select and generalize striking movements in robot table tennis.  _The International Journal of Robotics Research_ , 32(3):263–279, 2013.
  * Pomerleau [1989] D. A. Pomerleau.  Alvinn: An autonomous land vehicle in a neural network.  In _Advances in neural information processing systems_ , pages 305–313, 1989.
  * Pastor et al. [2011] P. Pastor, L. Righetti, M. Kalakrishnan, and S. Schaal.  Online movement adaptation based on previous sensor experiences.  In _2011 IEEE/RSJ International Conference on Intelligent Robots and Systems_ , pages 365–371. IEEE, 2011.
  * Zhang et al. [2018] T. Zhang, Z. McCarthy, O. Jow, D. Lee, X. Chen, K. Goldberg, and P. Abbeel.  Deep imitation learning for complex manipulation tasks from virtual reality teleoperation.  In _2018 IEEE International Conference on Robotics and Automation (ICRA)_ , pages 1–8. IEEE, 2018.
  * Sharma et al. [2019] P. Sharma, D. Pathak, and A. Gupta.  Third-person visual imitation learning via decoupled hierarchical controller.  In _Advances in Neural Information Processing Systems_ , pages 2597–2607, 2019.
  * Smith et al. [2019] L. Smith, N. Dhawan, M. Zhang, P. Abbeel, and S. Levine.  Avid: Learning multi-stage tasks via pixel-level translation of human videos.  _arXiv preprint arXiv:1912.04443_ , 2019.
  * Duan et al. [2017] Y. Duan, M. Andrychowicz, B. Stadie, O. J. Ho, J. Schneider, I. Sutskever, P. Abbeel, and W. Zaremba.  One-shot imitation learning.  In _Advances in neural information processing systems_ , pages 1087–1098, 2017.
  * Finn et al. [2017] C. Finn, T. Yu, T. Zhang, P. Abbeel, and S. Levine.  One-shot visual imitation learning via meta-learning.  In _Conference on Robot Learning_ , pages 357–368, 2017.
  * Yu et al. [2018] T. Yu, C. Finn, A. Xie, S. Dasari, T. Zhang, P. Abbeel, and S. Levine.  One-shot imitation from observing humans via domain-adaptive meta-learning.  _arXiv preprint arXiv:1802.01557_ , 2018.
  * Vaswani et al. [2017] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin.  Attention is all you need.  In _Advances in neural information processing systems_ , pages 5998–6008, 2017.
  * Wang et al. [2018] X. Wang, R. Girshick, A. Gupta, and K. He.  Non-local neural networks.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 7794–7803, 2018.
  * Rothkopf et al. [2007] C. A. Rothkopf, D. H. Ballard, and M. M. Hayhoe.  Task and context determine where you look.  _Journal of vision_ , 7(14):16–16, 2007.
  * Zhang et al. [2018] R. Zhang, Z. Liu, L. Zhang, J. A. Whritner, K. S. Muller, M. M. Hayhoe, and D. H. Ballard.  Agil: Learning attention from human for visuomotor tasks.  In _Proceedings of the european conference on computer vision (eccv)_ , pages 663–679, 2018.
  * Argall et al. [2009] B. D. Argall, S. Chernova, M. Veloso, and B. Browning.  A survey of robot learning from demonstration.  _Robotics and autonomous systems_ , 57(5):469–483, 2009.
  * Billard et al. [2008] A. Billard, S. Calinon, R. Dillmann, and S. Schaal.  Survey: Robot programming by demonstration.  _Handbook of robotics_ , 59(BOOK_CHAP), 2008.
  * Schaal [1999] S. Schaal.  Is imitation learning the route to humanoid robots?  _Trends in cognitive sciences_ , 3(6):233–242, 1999.
  * Ross et al. [2011] S. Ross, G. Gordon, and D. Bagnell.  A reduction of imitation learning and structured prediction to no-regret online learning.  In _Proceedings of the fourteenth international conference on artificial intelligence and statistics_ , pages 627–635, 2011.
  * Bain [1995] M. Bain.  A framework for behavioural cloning.  In _Machine Intelligence 15_ , pages 103–129, 1995.
  * Rahmatizadeh et al. [2018] R. Rahmatizadeh, P. Abolghasemi, L. Bölöni, and S. Levine.  Vision-based multi-task manipulation for inexpensive robots using end-to-end learning from demonstration.  In _2018 IEEE International Conference on Robotics and Automation (ICRA)_ , pages 3758–3765. IEEE, 2018.
  * Bojarski et al. [2016] M. Bojarski, D. Del Testa, D. Dworakowski, B. Firner, B. Flepp, P. Goyal, L. D. Jackel, M. Monfort, U. Muller, J. Zhang, et al.  End to end learning for self-driving cars.  _arXiv preprint arXiv:1604.07316_ , 2016.
  * Torabi et al. [2018] F. Torabi, G. Warnell, and P. Stone.  Behavioral cloning from observation.  _arXiv preprint arXiv:1805.01954_ , 2018.
  * Yang et al. [2019] C. Yang, X. Ma, W. Huang, F. Sun, H. Liu, J. Huang, and C. Gan.  Imitation learning from observations by minimizing inverse dynamics disagreement.  In _Advances in Neural Information Processing Systems_ , pages 239–249, 2019.
  * Ding et al. [2019] Y. Ding, C. Florensa, P. Abbeel, and M. Phielipp.  Goal-conditioned imitation learning.  In _Advances in Neural Information Processing Systems_ , pages 15324–15335, 2019.
  * Mandlekar et al. [2019] A. Mandlekar, F. Ramos, B. Boots, L. Fei-Fei, A. Garg, and D. Fox.  Iris: Implicit reinforcement without interaction at scale for learning control from offline robot manipulation data.  _arXiv preprint arXiv:1911.05321_ , 2019.
  * Lynch et al. [2020] C. Lynch, M. Khansari, T. Xiao, V. Kumar, J. Tompson, S. Levine, and P. Sermanet.  Learning latent plans from play.  In _Conference on Robot Learning_ , pages 1113–1132, 2020.
  * Fu et al. [2018] J. Fu, A. Singh, D. Ghosh, L. Yang, and S. Levine.  Variational inverse control with events: A general framework for data-driven reward definition.  In _Advances in Neural Information Processing Systems_ , pages 8538–8547, 2018.
  * Lynch and Sermanet [2020] C. Lynch and P. Sermanet.  Grounding language in play.  _arXiv preprint arXiv:2005.07648_ , 2020.
  * Shao et al. [2020] L. Shao, T. Migimatsu, Q. Zhang, K. Yang, and J. Bohg.  Concept2robot: Learning manipulation concepts from instructions and human demonstrations.  In _Proceedings of Robotics: Science and Systems (RSS)_ , July 2020\.
  * Xie et al. [2019] A. Xie, F. Ebert, S. Levine, and C. Finn.  Improvisation through physical understanding: Using novel objects as tools with visual foresight.  _arXiv preprint arXiv:1904.05538_ , 2019.
  * Simonyan and Zisserman [2014] K. Simonyan and A. Zisserman.  Two-stream convolutional networks for action recognition in videos.  In _Advances in neural information processing systems_ , pages 568–576, 2014.
  * Lee et al. [2019] T. E. Lee, J. Tremblay, T. To, J. Cheng, T. Mosier, O. Kroemer, D. Fox, and S. Birchfield.  Camera-to-robot pose estimation from a single image.  _arXiv preprint arXiv:1911.09231_ , 2019.
  * Deng et al. [2009] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei.  Imagenet: A large-scale hierarchical image database.  In _2009 IEEE conference on computer vision and pattern recognition_ , pages 248–255. Ieee, 2009.
  * He et al. [2020] K. He, H. Fan, Y. Wu, S. Xie, and R. Girshick.  Momentum contrast for unsupervised visual representation learning.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 9729–9738, 2020.
  * Chen et al. [2020] T. Chen, S. Kornblith, M. Norouzi, and G. Hinton.  A simple framework for contrastive learning of visual representations.  _arXiv preprint arXiv:2002.05709_ , 2020.
  * Grill et al. [2020] J.-B. Grill, F. Strub, F. Altché, C. Tallec, P. H. Richemond, E. Buchatskaya, C. Doersch, B. A. Pires, Z. D. Guo, M. G. Azar, et al.  Bootstrap your own latent: A new approach to self-supervised learning.  _arXiv preprint arXiv:2006.07733_ , 2020.
  * Sermanet et al. [2016] P. Sermanet, K. Xu, and S. Levine.  Unsupervised perceptual rewards for imitation learning.  _arXiv preprint arXiv:1612.06699_ , 2016.
  * Sermanet et al. [2018] P. Sermanet, C. Lynch, Y. Chebotar, J. Hsu, E. Jang, S. Schaal, S. Levine, and G. Brain.  Time-contrastive networks: Self-supervised learning from video.  In _2018 IEEE International Conference on Robotics and Automation (ICRA)_ , pages 1134–1141. IEEE, 2018.
  * Srinivas et al. [2020] A. Srinivas, M. Laskin, and P. Abbeel.  Curl: Contrastive unsupervised representations for reinforcement learning.  _arXiv preprint arXiv:2004.04136_ , 2020.
  * Laskin et al. [2020] M. Laskin, K. Lee, A. Stooke, L. Pinto, P. Abbeel, and A. Srinivas.  Reinforcement learning with augmented data.  _arXiv preprint arXiv:2004.14990_ , 2020.
  * Kostrikov et al. [2020] I. Kostrikov, D. Yarats, and R. Fergus.  Image augmentation is all you need: Regularizing deep reinforcement learning from pixels.  _arXiv preprint arXiv:2004.13649_ , 2020.
  * Hansen et al. [2020] N. Hansen, Y. Sun, P. Abbeel, A. A. Efros, L. Pinto, and X. Wang.  Self-supervised policy adaptation during deployment.  _arXiv preprint arXiv:2007.04309_ , 2020.
  * He et al. [2016] K. He, X. Zhang, S. Ren, and J. Sun.  Deep residual learning for image recognition.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 770–778, 2016.
  * Tancik et al. [2020] M. Tancik, P. P. Srinivasan, B. Mildenhall, S. Fridovich-Keil, N. Raghavan, U. Singhal, R. Ramamoorthi, J. T. Barron, and R. Ng.  Fourier features let networks learn high frequency functions in low dimensional domains.  _arXiv preprint arXiv:2006.10739_ , 2020.
  * Srivastava et al. [2014] N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov.  Dropout: a simple way to prevent neural networks from overfitting.  _The journal of machine learning research_ , 15(1):1929–1958, 2014.
  * Ioffe and Szegedy [2015] S. Ioffe and C. Szegedy.  Batch normalization: Accelerating deep network training by reducing internal covariate shift.  _arXiv preprint arXiv:1502.03167_ , 2015.
  * Levine et al. [2016] S. Levine, C. Finn, T. Darrell, and P. Abbeel.  End-to-end training of deep visuomotor policies.  _The Journal of Machine Learning Research_ , 17(1):1334–1373, 2016.
  * Rahmatizadeh et al. [2018] R. Rahmatizadeh, P. Abolghasemi, A. Behal, and L. Bölöni.  From virtual demonstration to real-world manipulation using lstm and mdn.  In _Thirty-Second AAAI Conference on Artificial Intelligence_ , 2018\.
  * Salimans et al. [2017] T. Salimans, A. Karpathy, X. Chen, and D. P. Kingma.  Pixelcnn++: Improving the pixelcnn with discretized logistic mixture likelihood and other modifications.  _arXiv preprint arXiv:1701.05517_ , 2017.
  * Todorov et al. [2012] E. Todorov, T. Erez, and Y. Tassa.  Mujoco: A physics engine for model-based control.  In _2012 IEEE/RSJ International Conference on Intelligent Robots and Systems_ , pages 5026–5033. IEEE, 2012.
  * Mandlekar et al. [2018] A. Mandlekar, Y. Zhu, A. Garg, J. Booher, M. Spero, A. Tung, J. Gao, J. Emmons, A. Gupta, E. Orbay, et al.  Roboturk: A crowdsourcing platform for robotic skill learning through imitation.  _arXiv preprint arXiv:1811.02790_ , 2018.
  * Hochreiter and Schmidhuber [1997] S. Hochreiter and J. Schmidhuber.  Long short-term memory.  _Neural computation_ , 9(8):1735–1780, 1997.
  * Sutskever et al. [2014] I. Sutskever, O. Vinyals, and Q. V. Le.  Sequence to sequence learning with neural networks.  In _Advances in neural information processing systems_ , pages 3104–3112, 2014.
  * Bahdanau et al. [2014] D. Bahdanau, K. Cho, and Y. Bengio.  Neural machine translation by jointly learning to align and translate.  _arXiv preprint arXiv:1409.0473_ , 2014.
  * Cheng et al. [2016] J. Cheng, L. Dong, and M. Lapata.  Long short-term memory-networks for machine reading.  _arXiv preprint arXiv:1601.06733_ , 2016.
  * Finn et al. [2017] C. Finn, P. Abbeel, and S. Levine.  Model-agnostic meta-learning for fast adaptation of deep networks.  _arXiv preprint arXiv:1703.03400_ , 2017.
  * Zhou et al. [2019] A. Zhou, E. Jang, D. Kappler, A. Herzog, M. Khansari, P. Wohlhart, Y. Bai, M. Kalakrishnan, S. Levine, and C. Finn.  Watch, try, learn: Meta-learning from demonstrations and reward.  _arXiv preprint arXiv:1906.03352_ , 2019.

##  Appendix A Appendix

###  A.1 Baseline Comparisons: Multi-Object Environments

While the prior experiments showed our model could successfully generalize to
new task instances, can it also generalize to new tasks including unseen
objects? To answer this question the baseline comparisons (described in
Section 4.2) are repeated in environments with multiple objects. Importantly,
the objects used during test time are unseen during training.

Environment Description: The multi-object environment is cloned from the base
environment (presented in Section 4.1) and modified to include more objects
with different shapes and textures. Note that while object appearance and
shape is randomized, dynamical properties - like friction - are kept constant
since they cannot be visually judged. The simulator has $30$ unique objects,
$26$ of which are seen during training and $4$ are only used during test time.

Data Collection Process: To collect train tasks, $4$ objects are sampled from
the $26$ train objects, which results in an environment with $16$ tasks. For
each task, multiple task instances composed of expert demonstration videos
($v_{i}$) and imitator trajectories ($t_{i}$) are collected using the same
methodology as before (refer to Section 4.2 and Section 4.1). In total, the
train dataset is composed of $1200$ tasks ($2400$ task instances total). Test
tasks are also sampled in the same fashion as before, except using the $4$ new
objects. Our method is able to succeed at the object picking stage of the
tasks $50\pm 9.9\%$ of the time which is $\sim 2$x better than the best
baseline (contextual-LSTM) which only picks $23\pm 8.4\%$ of the time.
Unfortunately, all methods (including ours) often place objects in the wrong
bin resulting in final success rates of $23\pm 8.4\%$ for our method and
$22\pm 8.3\%$ for the best baseline. In practice, this failure mode is easy to
rectify since a hard coded policy will always place the object in the right
bin. Encouragingly, our policy is best at grasping and picking unseen objects
which is the hardest part of this task. Nonetheless, this failure mode shows
more improvements are needed for this method to work in broader settings.

###  A.2 Regularization Effect on Behavior Cloning Loss

While the inverse model regularization term clearly changed test time
performance for the better (shown in Section 4.4), can this be explained by
positive transfer to the behavior cloning task? In other words, it is possible
the inverse modelling loss merely prevents over-fitting in the behavior
cloning loss, and thus some other regularization term could achieve the same
effect.

Figure 6: One hypothesis is that the ablated models fail at test time because
they cannot optimize the behavior cloning loss. Comparing train and val loss
for models trained on the same data (N=1600) eliminates this possibility.

To test this theory, we plot behavior cloning loss (both training and
validation) vs train iteration for both the base model, and ablation models
from Section 4.4. Note that behavior cloning train performance is nearly
identical, whereas final success rates are dramatically different. We believe
these facts in tandem confirm that self-supervised inverse modeling forces our
representation to capture information which is useful for robust test
performance, but not necessary to minimize the cloning loss.

###  A.3 Time-Step Ablation

Instead of using a context video from the demonstrator agent to infer the
task, our model could just use the last frame from the demonstration video.
After all, the last frame should uniquely specify which object should go in
which bin, and prior work [27] has successfully used goal image conditioning.
To test this, we train a version of our model which conditions just on the
final frame from the context video, and compare its performance on the
benchmarks from Section 4.2. This modified model achieves a final success rate
of $61\pm 9.7\%$ which is significantly less than the $88\pm 5.0\%$ our model
(which ingests more frames from context) can achieve. This effect holds even
if the base model uses just one extra context frames (i.e. both beginning and
end frame). We hypothesize that these frames, while not strictly necessary,
help the infer which task it needs to perform, thus resulting in a performance
boost.


The category $\operatorname{\underline{K}^{or}}(X)$ of virtual vector bundles
over $X$ modulo orientation is the groupoid whose objects are virtual locally
free sheaves $\mathcal{V}$ on $X$ and a morphism from $\mathcal{V}$ to
$\mathcal{V}^{\prime}$ is a morphism from
$(\operatorname{rk}\mathcal{V},\det\mathcal{V})$ to
$(\operatorname{rk}\mathcal{V}^{\prime},\det\mathcal{V}^{\prime})$ in
$\underline{\mathbb{Z}}_{X}\times\operatorname{\underline{Pic}^{or}}(X)$,
i.e., a quadratic isomorphism
$\det\mathcal{V}\rightarrowtail\det\mathcal{V}^{\prime}$ assuming
$\operatorname{rk}\mathcal{V}=\operatorname{rk}\mathcal{V}^{\prime}$.

We refer to morphisms in $\operatorname{\underline{K}^{or}}(X)$ as quadratic
isomorphisms and use the notation
$\mathcal{V}\rightarrowtail\mathcal{V}^{\prime}$. By definition,
$\operatorname{\underline{K}^{or}}(X)$ is the essential image of the composite
functor

$\operatorname{\underline{K}}(X)\xrightarrow{(\operatorname{rk},\det)}\underline{\mathbb{Z}}_{X}\times\operatorname{\underline{Pic}}(X)\xrightarrow{Id\times\rho}\underline{\mathbb{Z}}_{X}\times\operatorname{\underline{Pic}^{or}}(X)$

In particular, one gets a canonical monoidal structure on
$\operatorname{\underline{K}^{or}}(X)$, so that it becomes a Picard groupoid
with a canonical monoidal functor

$\operatorname{\underline{K}}(X)\xrightarrow{\rho}\operatorname{\underline{K}^{or}}(X)$

######  Remark 6.1.14.

We can again follow the lines of 6.1.5 and define an orientation of a virtual
vector bundle $v=\mathbb{V}(\mathcal{V})$ as either an orientation of its
determinant or a quadratic isomorphism $\mathcal{V}\rightarrowtail\langle
r\rangle$ where $r=\operatorname{rk}(\mathcal{V})$ (as a locally constant
function on $X$). The same applies to vector bundles and recovers the
classical definition of Barge and Morel.

######  Example 6.1.15.

Let $\mathbb{E}$ be a ring spectrum over a scheme $S$ which is
$\mathbf{SL}$-oriented in the sense of Panin-Walter (see [1]). Let $X$ be a
separated $S$-scheme and $v$ a virtual bundle over $X$. Using Thom
isomorphisms attached to the $\mathbf{SL}$-orientation of $\mathbb{E}$, one
obtains that the $v$-twisted $\mathbb{E}$-cohomology of $X$

$\mathbb{E}^{n}(X,v):=[\Pi_{S}(X,-v),\mathbb{E}[n]]$

depends only on the pair $(\operatorname{rk}(v),\det(v))$. Moreover, it
follows from 6.1.10 that $\mathbb{E}^{n}(X,v)$ is functorial in $v$ with
respect to isomorphisms modulo orientation.

Chow-Witt groups provide the most fundamental example for us (the unramified
Milnor-Witt sheaf $\underline{\mathbf{K}}^{MW}_{*}$ represents these groups
over fields).

######  Proposition 6.1.16.

Let $X$ be a scheme such that $K_{0}(X)$ is infinite cyclic, e.g., a semi-
local scheme or a unique factorization domain. Then there exists a canonical
monoidal extension $\tilde{\operatorname{Th}}_{X}$ of the Thom space functor
and a natural isomorphism

---
$\textstyle{\operatorname{\underline{K}}(X)\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{\operatorname{Th}_{X}}$$\scriptstyle{\rho}$$\scriptstyle{\sim}$$\textstyle{\operatorname{SH}(X)}$$\textstyle{\operatorname{\underline{K}^{or}}(X)\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{\tilde{\operatorname{Th}}_{X}}$

which is the identity on objects.

###### Proof.

By assumption, the morphism
$\operatorname{\underline{K}}(X)\xrightarrow{(\operatorname{rk},\det)}\underline{\mathbb{Z}}_{X}\times\operatorname{\underline{Pic}}(X)$
is an equivalence of categories. Thus, by definition of
$\operatorname{\underline{K}^{or}}(X)$, 6.1.10 concludes the proof. ∎

######  Remark 6.1.17.

Note that, for $X$ as above, we get an explicit quasi-inverse to
$(\operatorname{rk},\det)$ by associating to the class of $(r,\mathcal{L})$
the class of the locally free sheaf $\mathcal{O}_{X}^{r-1}\oplus\mathcal{L}$.
In particular, one can describe the functor $\tilde{\operatorname{Th}}_{X}$ as
the composite

$\operatorname{\underline{K}^{or}}(X)\xrightarrow{\simeq}\underline{\mathbb{Z}}_{X}\times\operatorname{\underline{Pic}^{or}}(X)\rightarrow\operatorname{SH}(X),\mathcal{V}\mapsto(r=\operatorname{rk}\mathcal{V},\det\mathcal{V})\mapsto\operatorname{Tw}_{X}(\mathcal{V})(r)[2r]$

The latter description clarifies the functoriality. In fact, the proposition
posits the existence of a canonical isomorphism (functorial for isomorphisms
in $\mathcal{V}$)

$\operatorname{Th}_{X}(\mathcal{V})\simeq\operatorname{Tw}_{X}(\det\mathcal{V})(r)[2r],r=\operatorname{rk}\mathcal{V}$

###  6.2. Quadratic 0-cycles and quadratic degrees

######  6.2.1.

Next, we recall a few definitions of Chow-Witt groups suitable for our
needs.181818We focus on zero cycles and emphasize (quadratic) cycles rather
than cycle classes. We fix a base field $k$, not necessarily perfect but
finitely generated over a perfect field $k_{0}$.191919This is to be able to
use the written account on Chow-Witt groups. This assumption will be removed
in [34].

Given a finitely generated extension field $K/k$, we let $K_{*}^{MW}(K)$ be
the Milnor-Witt ring of $K$ (see [70, Def. 3.1]). Given an invertible
$K$-vector space $\mathcal{L}$, we define the twisted Milnor-Witt ring of $K$
by the formula in [70, Rem. 3.21]

(6.2.1.a)
$K_{*}^{MW}(K,\mathcal{L}):=K_{*}^{MW}(K)\otimes_{\mathbb{Z}[K^{\times}]}\mathbb{Z}[\mathcal{L}^{\times}]$

where $\mathcal{L}^{\times}=\mathcal{L}-\\{0\\}$, using the action of
$K^{\times}$ on $K_{*}^{MW}(K)$ via the canonical map
$K^{\times}\rightarrow\operatorname{GW}(K)=K_{0}^{MW}(K)$.

Let now $X$ be an essentially smooth $k$-scheme of dimension $d$ and
$\mathcal{L}$ an invertible sheaf on $X$. One defines the group of _quadratic
$0$-cycles_ on $X$ twisted by $\mathcal{L}$ as

$\tilde{Z}{}^{d}(X,\mathcal{L}):=\bigoplus_{x\in
X^{(d)}}\operatorname{GW}(\kappa(x),\omega_{x/X}^{\vee}\otimes_{\kappa(x)}\mathcal{L}|_{x})$

Here $X^{(d)}$ is the set of closed points $x$ of $X$ and $\omega_{x/X}$ is
the determinant of the $\kappa(x)$-vector space
$\mathcal{C}_{x/X}=\mathfrak{m}_{x}/\mathfrak{m}_{x}^{2}$. The _support_ of a
quadratic $0$-cycle $\alpha$ is the set of points $x\in X^{(d)}$ whose
coefficient in $\alpha$ is non-zero. We will consider it as a finite reduced
closed subscheme of $X$.

Owing to [70, Rem. 5.13], [45], or [46, Def. 7.2]202020In Morel’s notation,
$\tilde{Z}{}^{d}(X,\mathcal{L})$ is the $d$-th term of the Rost-Schmid complex
$C^{*}_{\mathrm{RS}}(X,\mathrm{K}^{MW}_{d}\\{\mathcal{L}\\})$ while in Feld’s
notation it is the end of the complex
$C_{*}(X,\mathrm{K}^{MW}_{*},\omega^{\vee}_{X/k}\otimes\mathcal{L})$, where
$\omega_{X/k}=\det(\Omega_{X/k})$ is the canonical sheaf of $X/k$.  there is a
map

$\mathrm{div}:\bigoplus_{y\in
X^{(d-1)}}\mathrm{K}^{MW}_{1}(\kappa(y),\omega_{y/X}^{\vee}\otimes\mathcal{L}|_{y})\longrightarrow\operatorname{\tilde{Z}}^{d}(X,\mathcal{L})$

Two quadratic zero-cycles are said to be _rationally equivalent_ if their
difference is in the image of $\mathrm{div}$. The above defines an additive
equivalence relation $\sim_{rat}$ on quadratic $0$-dimensional cycles, and the
_$d$ -th Chow-Witt group_ of $X$ twisted by $\mathcal{L}$ is the quotient

$\operatorname{\widetilde{CH}}^{d}(X,\mathcal{L})=\operatorname{\tilde{Z}}^{d}(X,\mathcal{L})/\\!\sim_{rat}=\mathrm{coKer}(\mathrm{div})$

This group depends functorially on $\mathcal{L}$ for quadratic isomorphisms.

######  Proposition 6.2.2.

Let $X$ be an essentially smooth $k$-scheme of dimension $d$ and let
$v=\mathbb{V}(\mathcal{V})$ be a virtual vector bundle of rank $d$ on $X$.
Then there is a canonical isomorphism

$H^{0}_{\operatorname{SH}}(X,v):=[\mathbf{1}_{X},\operatorname{Th}(v)]\simeq\operatorname{\widetilde{CH}}^{d}\big{(}X,\det\mathcal{V}\big{)}$

###### Proof.

With $k$ being finitely generated over a perfect field $k_{0}$, one can work
over $k_{0}$ or assume that $k$ is perfect. The coniveau spectral sequence
(see [31], §1.1.1 and Def. 1.4) associated with the cohomology theory
$H^{*}_{\operatorname{SH}}(X,v)$ takes the form

$E_{1}^{p,q}=\oplus_{x\in
X^{(p)}}H^{p+q}_{\operatorname{SH}}(\operatorname{Th}(N_{x}X_{(x)}),v)\Rightarrow
H_{\operatorname{SH}}^{p+q}(X,v)$

Here $X_{(x)}=\operatorname{Spec}(\mathcal{O}_{X,x})$ and $N_{x}X_{(x)}$ is
the normal bundle of $x$ (this relies on Morel-Voevodsky’s homotopy purity
theorem). The $E_{1}$-term is concentrated in the range $p\in[0,d]$ and by the
$\mathbf{A}^{1}$-connectivity theorem, in the range $q\leq 0$. According to
Morel’s computation of the $0$-stable stem and Feld’s theory [47], there is an
isomorphism between complexes

$E_{1}^{*,0}\simeq
C^{*}(X,\mathrm{K}^{MW}_{*},\omega_{X/k}^{\vee}\otimes\det\mathcal{V})$

We conclude by looking at the line $p+q=d$. ∎

######  6.2.3.

Quadratic Degree. Let $X/k$ be a smooth proper scheme of dimension $d$ with
canonical sheaf $\omega_{X}=\det(\Omega_{X/k})$. In the following we define a
notion of quadratic degree of $\omega_{X}$-twisted quadratic $0$-cycles. To
begin, note that there is a canonical isomorphism

$\tilde{Z}{}^{d}(X,\omega_{X})=\oplus_{x\in
X^{(d)}}\operatorname{GW}(\kappa(x),\omega_{x/X}^{\vee}\otimes\omega_{X}|_{x})\simeq\oplus_{x\in
X^{(d)}}\operatorname{GW}\big{(}\kappa(x)\big{)}$

This holds because for any closed point $x\in X$, the conormal exact sequence

$0\to\mathcal{C}_{x/X}\to\Omega_{X}|_{x}\to\Omega_{x/k}\to 0$

together with the fact that $\Omega_{x/k}=0$ (as $\kappa(x)/k$ is étale)
provides a canonical isomorphism of $\kappa(x)$-vector spaces
$\omega_{x/X}^{\vee}\otimes\omega_{X}|_{x}=\det(\mathcal{C}_{x/X})^{\vee}\otimes\omega_{X}|_{x}\simeq\kappa(x)$.
Thus, an $\omega_{X}$-twisted quadratic $0$-cycle can be identified with a
formal sum $\alpha=\sum_{i\in I}\langle\sigma_{i}\rangle.x_{i}$, where
$x_{i}\in X$ is a closed point and $\sigma_{i}$ is a symmetric bilinear form
over $\kappa(x_{i})$.

One defines the _quadratic degree_ $\operatorname{\widetilde{deg}}$ of a
quadratic cycle $\alpha$ as the proper pushforward associated with the
projection of $X/k$ (see [15, Chap. 2, §3]). It is defined at the level of
cycles and factorizes through rational equivalence. If one assumes that the
support of $\alpha$ is étale over $k$, then it can be computed explicitly as
the element

$\operatorname{\widetilde{deg}}(\alpha)=\sum_{i\in
I}\langle\operatorname{Tr}_{\kappa(x_{i})/k}\circ\sigma_{i}\rangle\in\operatorname{GW}(k)$

where $\operatorname{Tr}_{\kappa(x_{i})/k}$ is the trace form for the finite
separable (by assumption) extension $\kappa(x_{i})/k$ (apply _loc. cit._ Lemma
2.3).

More generally, let $\mathcal{L}$ be an invertible sheaf over $X$ with a
relative orientation (see 6.1.8) given by a quadratic isomorphism
$\epsilon:\mathcal{L}\rightarrowtail\omega_{X}$. We define the _quadratic
$\epsilon$-degree_ as the composite

(6.2.3.a)
$\operatorname{\widetilde{deg}}_{\epsilon}:\tilde{Z}{}^{d}(X,\mathcal{L})\xrightarrow{\epsilon_{*}}\tilde{Z}{}^{d}(X,\omega_{X})\xrightarrow{\operatorname{\widetilde{deg}}}\operatorname{GW}(k)$

To compute this degree concretely, assuming that the support of $\alpha$ is
étale over $k$, one first choose any representative
$\tilde{\epsilon}:\mathcal{L}\xrightarrow{\epsilon}\omega_{X}\otimes\mathcal{M}^{\otimes
2}$ of the quadratic isomorphism $\epsilon$. Then, by linearity, one is
reduced to quadratic cycles of the form

$\alpha=\langle\sigma\rangle\otimes(t^{*}\otimes l).x\in
GW(\kappa(x),\omega_{x/X}^{\vee}\otimes\mathcal{L}|_{x})=GW(\kappa(x))\otimes_{\mathbb{Z}[\kappa(x)^{\times}]}\mathbb{Z}[(\omega_{x/X}^{\vee}\otimes\mathcal{L}|_{x})^{\times}]$

where $t=t_{1}\wedge...\wedge t_{d}$ belongs to $\omega_{x/X}$ and correspond
to a local parametrisation of $x\in X$, and $l\in\mathcal{L}|_{x}^{\times}$.
Then $\epsilon_{x}(l)=w_{l}\otimes(u\otimes u)$ where
$w_{l}\in\omega_{X}|_{x}$ and $u\in\mathcal{M}|_{x}^{\times}$. Finally, one
gets a canonical isomorphism $\omega_{X}|_{x}\simeq\omega_{x/X}$ so that $t$
defines a non-zero linear form

$t^{*}:\omega_{X}|_{x}\simeq\omega_{x/X}\rightarrow\kappa(x)$

Putting everything together, we get the formula

(6.2.3.b)
$\operatorname{\widetilde{deg}}_{\epsilon}\big{(}\langle\sigma\rangle\otimes(t^{*}\otimes
l).x\big{)}=\langle\operatorname{Tr}_{\kappa(x)/k}(t^{*}(w_{l}).\sigma)\rangle\in\operatorname{GW}(k)$

######  6.2.4.

Oriented degree of oriented cycles We keep the notation and hypotheses of the
previous paragraph and assume that $\omega_{X}$ is orientable, with chosen
orientation class $\tau\in\mathscr{O}r_{X}(\omega_{X})$. Suppose that $Y$ is a
reduced $d$-codimensional closed subscheme of $X$, such that $Y/k$ is étale.
Let $\omega_{Y/X}=\det\mathcal{C}_{Y/X}$ be the determinant of the conormal
sheaf of $Y$ in $X$ and assume given an orientation classe
$\epsilon\in\mathscr{O}r_{Z}(\omega_{Y/X})$. This allows us to define a
canonical quadratic $0$-cycle $[Y]_{X}^{\epsilon}\in\tilde{Z}^{d}(X)$
associated with $(Y,\epsilon)$, as the image of $\sum_{x\in Y_{(0)}}\langle
1\rangle.x\in\tilde{Z}^{0}(Y)$ under the composite map

$\tilde{Z}^{0}(Y)\xrightarrow{\epsilon^{-1}_{*}}\tilde{Z}^{0}(Y,\omega_{Y/X})\xrightarrow{i_{*}}\tilde{Z}^{d}(X)$

Note that given our conventions, the map $i_{*}$ is just the identity. As
$\tau$ corresponds to a quadratic isomorphism
$\omega_{X}\rightarrowtail\mathcal{O}_{X}$, we get the $\tau$-degree map

$\deg_{\tau^{-1}}:\tilde{Z}^{d}(X)\stackrel{{\scriptstyle\tau_{*}^{-1}}}{{\rightarrow}}\tilde{Z}^{d}(X,\omega_{X})\xrightarrow{\operatorname{\widetilde{deg}}}\operatorname{GW}(k)$

Then the $\tau$-oriented degree of the $\epsilon$-oriented cycle
$[Y]_{X}^{\epsilon}$ is given by the class

$\deg_{\tau^{-1}}([Y]_{X}^{\epsilon})=\sum_{x\in
Y_{(0)}}\operatorname{Tr}_{\kappa(x)/k}\circ\langle
u_{x}\rangle\in\operatorname{GW}(k)$

for quadratic classes $u_{x}\in Q(\kappa(x))$ computed as in the above
paragraph.

The quadratic class $u_{x}$ can be computed in terms of the chosen
orientations using again the canonical isomorphism
$\phi:\omega_{X}|_{x}\simeq\omega_{x/X}$ (as $\kappa(x)/k$ is separable).
Namely, we get two induced orientation classes $\epsilon|_{x}$ and
$\phi_{*}(\tau|_{x})$ of the $1$-dimensional $\kappa(x)$-vector space
$\omega_{x/X}=\omega_{X}|_{x}$, which, according to Remark 6.1.7, are uniquely
linked by a relation of the form

$\epsilon|_{x}=u_{x}.\phi_{*}(\tau|_{x})$

Equivalently, $u_{x}$ is the element
$\epsilon|_{x}\otimes(\tau|_{x}^{\vee})^{-1}$ in

$\mathscr{O}r_{\kappa(x)}(\omega_{x/X}\otimes\omega^{\vee}_{X}|_{x})\simeq\mathscr{O}r_{\kappa(x)}(\mathcal{O}_{\kappa(x)})\simeq
Q(\kappa(x))$

## References

  * [1] A. Ananyevskiy.  SL-oriented cohomology theories.  In Motivic homotopy theory and refined enumerative geometry, volume 745 of Contemp. Math., pages 1–19. Amer. Math. Soc., [Providence], RI, [2020] ©2020.
  * [2] M. Artin.  On isolated rational singularities of surfaces.  Am. J. Math., 88:129–136, 1966.
  * [3] M. Artin.  Coverings of the rational double points in characteristic p.  Complex Analysis and Algebraic Geometry, Collected Papers dedicated to K. Kodaira, 11-22 (1977), 1977.
  * [4] A. Asok and J. Fasel.  Algebraic vector bundles on spheres.  J. Topol., 7(3):894–926, 2014.
  * [5] A. Asok, S. Kebekus, and M. Wendt.  Comparing $\mathbb{A}^{1}$-$h$-cobordism and $\mathbb{A}^{1}$-weak equivalence.  Ann. Sc. Norm. Super. Pisa, Cl. Sci. (5), 17(2):531–572, 2017.
  * [6] A. Asok and F. Morel.  Smooth varieties up to $\mathbb{A}^{1}$-homotopy and algebraic $h$-cobordisms.  Adv. Math., 227(5):1990–2058, 2011.
  * [7] A. Asok and P. A. Østvær.  $\mathbb{A}^{1}$-homotopy theory and contractible varieties: a survey.  In Homotopy theory and arithmetic geometry – motivic and Diophantine aspects. LMS-CMI research school, London, UK, July 9–13, 2018. Lecture notes, pages 145–212. Cham: Springer, 2021.
  * [8] M. F. Atiyah.  Thom complexes.  Proc. London Math. Soc., 11(3):291–310, 1961.
  * [9] J. Ayoub.  Les six opérations de Grothendieck et le formalisme des cycles évanescents dans le monde motivique (I), volume 314 of Astérisque.  Soc. Math. France, 2007.
  * [10] J. Ayoub.  Les six opérations de Grothendieck et le formalisme des cycles évanescents dans le monde motivique (II), volume 315 of Astérisque.  Soc. Math. France, 2007.
  * [11] J. Ayoub.  The motivic vanishing cycles and the conservation conjecture.  In Algebraic cycles and motives. Volume 1. Selected papers of the EAGER conference, Leiden, Netherlands, August 30–September 3, 2004 on the occasion of the 75th birthday of Professor J. P. Murre, pages 3–54. Cambridge: Cambridge University Press, 2007.
  * [12] J. Ayoub.  Note sur les opérations de Grothendieck et la réalisation de Betti.  J. Inst. Math. Jussieu, 9(2):225–263, 2010.
  * [13] J. Ayoub.  La réalisation étale et les opérations de Grothendieck.  Ann. Sci. Éc. Norm. Supér. (4), 47(1):1–145, 2014.
  * [14] T. Bachmann.  $\eta$-periodic motivic stable homotopy theory over Dedekind domains.  arXiv e-prints, page arXiv:2006.02086, June 2020.
  * [15] T. Bachmann, B. Calmès, F. Déglise, J. Fasel, and P. A. Østvær.  Milnor-Witt motives.  to appear in Memoirs of the AMS, 2022.
  * [16] T. Bachmann and P. A. Østvær.  Topological models for stable motivic invariants of regular number rings.  Forum Math. Sigma, 10:27, 2022.  Id/No e1.
  * [17] T. Bachmann and K. Wickelgren.  $\mathbb{A}^{1}$-euler classes: six functors formalisms, dualities, integrality and linear subspaces of complete intersections.  arXiv: 2002.01848v2, 2021.
  * [18] J. Barge and F. Morel.  Groupe de Chow des cycles orientés et classe d’Euler des fibrés vectoriels.  C. R. Acad. Sci. Paris Sér. I Math., 330(4):287–290, 2000.
  * [19] A. A. Beĭlinson, J. Bernstein, and P. Deligne.  Faisceaux pervers.  In Analysis and topology on singular spaces, I (Luminy, 1981), volume 100 of Astérisque, pages 5–171. Soc. Math. France, Paris, 1982.
  * [20] S. Bloch and A. Ogus.  Gersten’s conjecture and the homology of schemes.  Ann. Sci. École Norm. Sup. (4), 7:181–201 (1975), 1974.
  * [21] M. V. Bondarko.  Weights for relative motives: relation with mixed complexes of sheaves.  Int. Math. Res. Not., 2014(17):4715–4767, 2014.
  * [22] M. V. Bondarko and F. Déglise.  Dimensional homotopy t-structures in motivic homotopy theory.  Adv. Math., 311:91–189, 2017.
  * [23] D.-C. Cisinski.  Cohomological methods in intersection theory.  In Homotopy theory and arithmetic geometry – motivic and Diophantine aspects. LMS-CMI research school, London, UK, July 9–13, 2018. Lecture notes, pages 49–105. Cham: Springer, 2021.
  * [24] D.-C. Cisinski and F. Déglise.  Mixed Weil cohomologies.  Adv. in Math., 230(1):55–130, 2012.
  * [25] D.-C. Cisinski and F. Déglise.  Integral mixed motives in equal characteristic.  Doc. Math., Extra Vol.:145–194, 2015.
  * [26] D.-C. Cisinski and F. Déglise.  Étale motives.  Compos. Math., 152(3):556–666, 2016.
  * [27] D.-C. Cisinski and F. Déglise.  Triangulated categories of mixed motives.  Cham: Springer, 2019.
  * [28] D. Coulette, F. Déglise, J. Fasel, and J. Hornbostel.  Formal ternary laws and Buchstaber’s 2-groups.  arXiv:2112.03646v1, 2021.
  * [29] W. Danielewski.  On a Cancellation Problem and Automorphism Group of Affine Algebraic Varieties.  Preprint, 1988.
  * [30] F. Déglise.  Around the Gysin triangle II.  Doc. Math., 13:613–675, 2008.
  * [31] F. Déglise.  Coniveau filtration and mixed motives.  In Regulators. Regulators III conference, Barcelona, Spain, July 12–22, 2010. Proceedings, pages 51–76. Providence, RI: American Mathematical Society (AMS), 2012.
  * [32] F. Déglise.  Orientation theory in arithmetic geometry.  In $K$-Theory—Proceedings of the International Colloquium, Mumbai, 2016, pages 239–347. Hindustan Book Agency, New Delhi, 2018.
  * [33] F. Déglise, J. Fasel, F. Jin, and A. A. Khan.  On the rational motivic homotopy category.  J. Éc. Polytech., Math., 8:533–583, 2021.
  * [34] F. Déglise, N. Feld, and F. Jin.  Perverse homotopy heart and MW-modules.  in preparation, 2022.
  * [35] F. Déglise, F. Jin, and A. A. Khan.  Fundamental classes in motivic homotopy theory.  J. Eur. Math. Soc. (JEMS), 23(12):3935–3993, 2021.
  * [36] P. Deligne.  Le déterminant de la cohomologie.  In Current trends in arithmetical algebraic geometry (Arcata, Calif., 1985), volume 67 of Contemp. Math., pages 93–177. Amer. Math. Soc., Providence, RI, 1987.
  * [37] P. Deligne and N. Katz.  Séminaire de Géométrie Algébrique du Bois Marie \- 1967-69 - Groupes de monodromie en géométrie algébrique II (SGA 7 II), volume 340 of Lecture Notes in Math. Springer-Verlag, 1973.
  * [38] B. Drew.  Motivic Hodge modules.  arXiv: 1801.10129v1, 2018.
  * [39] B. Drew and M. Gallauer.  The universal six-functor formalism.  arXiv: 2009.13610, Feb. 2021.
  * [40] A. Dubouloz and J. Fasel.  Families of $\mathbb{A}^{1}$-contractible affine threefolds.  Algebr. Geom., 5(1):1–14, 2018.
  * [41] A. Dubouloz, S. Pauli, and P. A. Østvær.  $\mathbb{A}^{1}$-contractibility of affine modifications.  Int. J. Math., 30(14):34 pp, 2019.  Id/No 1950069.
  * [42] B. I. Dundas, O. Röndigs, and P. A. Østvær.  Motivic functors.  Doc. Math., 8:489–525, 2003.
  * [43] E. Elmanto and H. Kolderup.  On modules over motivic ring spectra.  Ann. $K$-Theory, 5(2):327–355, 2020.
  * [44] E. Elmanto, M. Levine, M. Spitzweck, and P. A. Østvær.  Algebraic Cobordism and Étale Cohomology.  to appear in Geometry and Topology, page arXiv:1711.06258, Nov. 2022\.
  * [45] J. Fasel.  Groupes de Chow-Witt.  Mém. Soc. Math. Fr. (N.S.), 113:vi+197, 2008.
  * [46] N. Feld.  Milnor-Witt cycle modules.  J. Pure Appl. Algebra, 224(7):44, 2020.  Id/No 106298.
  * [47] N. Feld.  Morel homotopy modules and Milnor-Witt cycle modules.  Doc. Math., 26:617–659, 2021.
  * [48] K.-H. Fieseler.  On complex affine surfaces with ${\bf C}^{+}$-action.  Comment. Math. Helv., 69:5–27, 1994.
  * [49] P. Griffiths and J. Harris.  Principles of algebraic geometry.  New York, NY: John Wiley & Sons Ltd., 2nd ed. edition, 1994.
  * [50] N. Gupta.  A survey on Zariski cancellation problem.  Indian J. Pure Appl. Math., 46(6):865–877, 2015.
  * [51] F. Hirzebruch.  Some problems on differentiable and complex manifolds.  Ann. Math. (2), 60:213–236, 1954.
  * [52] M. Hoyois.  A quadratic refinement of the Grothendieck-Lefschetz-Verdier trace formula.  Algebr. Geom. Topol., 14(6):3608–3658, 2014.
  * [53] M. Hoyois, S. Kelly, and P. A. Østvær.  The motivic Steenrod algebra in positive characteristic.  J. Eur. Math. Soc., 19(12):3813–3849, 2017.
  * [54] M. Hoyois, A. Krishna, and P. A. Østvær.  $\mathbb{A}^{1}$-contractibility of Koras-Russell threefolds.  Algebr. Geom., 3(4):407–423, 2016.
  * [55] A. Huber and S. Müller-Stach.  Periods and Nori motives, volume 65 of Ergebnisse der Mathematik und ihrer Grenzgebiete. 3. Folge. A Series of Modern Surveys in Mathematics [Results in Mathematics and Related Areas. 3rd Series. A Series of Modern Surveys in Mathematics].  Springer, Cham, 2017.  With contributions by Benjamin Friedrich and Jonas von Wangenheim.
  * [56] B. Hughes and A. Ranicki.  Ends of complexes, volume 123 of Cambridge Tracts in Mathematics.  Cambridge University Press, Cambridge, 1996.
  * [57] L. Illusie, Y. Laszlo, and F. Orgogozo, editors.  Travaux de Gabber sur l’uniformisation locale et la cohomologie étale des schémas quasi-excellents.  Société Mathématique de France, Paris, 2014.  Séminaire à l’École Polytechnique 2006–2008. With the collaboration of F. Déglise, A. Moreau, V. Pilloni, M. Raynaud, J. Riou, B. Stroh, M. Temkin and W. Zheng, Astérisque No. 363-364 (2014) (2014).
  * [58] F. Ivorra and S. Morel.  The four operations on perverse motives.  arXiv: 1901.02096, 2019.
  * [59] J. F. Jardine.  Motivic symmetric spectra.  Doc. Math., 5:445–553, 2000.
  * [60] F. Jin.  Borel-Moore motivic homology and weight structure on mixed motives.  Math. Z., 283(3-4):1149–1183, 2016.
  * [61] F. Jin and E. Yang.  Künneth formulas for motives and additivity of traces.  Adv. Math., 376:107446, 83, 2021.
  * [62] A. A. Khan.  Motivic homotopy theory in derived algebraic geometry.  https://www.preschema.com/papers/thesis.pdf, 2016.
  * [63] A. A. Khan.  Voevodsky’s criterion for constructible categories of coefficients.  https://www.preschema.com/papers/six.pdf, 2021.
  * [64] J. Kollár.  Real algebraic threefolds. III. Conic bundles.  J. Math. Sci., New York, 94(1):996–1020, 1999.
  * [65] M. Levine.  Tate motives and the vanishing conjectures for algebraic $K$-theory.  In Algebraic $K$-theory and algebraic topology (Lake Louise, AB, 1991), volume 407 of NATO Adv. Sci. Inst. Ser. C Math. Phys. Sci., pages 167–188. Kluwer Acad. Publ., Dordrecht, 1993.
  * [66] M. Levine.  Motivic tubular neighborhoods.  Doc. Math., 12:71–146, 2007.
  * [67] J. Lurie.  Higher algebra.  https://www.math.ias.edu/ lurie/papers/HA.pdf, Sept. 2017.
  * [68] J. Milnor.  Singular points of complex hypersurfaces.  Annals of Mathematics Studies, No. 61. Princeton University Press, Princeton, N.J.; University of Tokyo Press, Tokyo, 1968.
  * [69] J. Milnor and E. Spanier.  Two remarks on fiber homotopy type.  Pacific J. Math., 10:585–590, 1960.
  * [70] F. Morel.  $\mathbb{A}^{1}$-algebraic topology over a field, volume 2052.  Berlin: Springer-Verlag, 2012.
  * [71] D. Mumford.  The topology of normal singularities of an algebraic surface and a criterion for simplicity.  Inst. Hautes Études Sci. Publ. Math., 9:5–22, 1961.
  * [72] N. Naumann, M. Spitzweck, and P. A. Østvær.  Motivic Landweber exactness.  Doc. Math., 14:551–593, 2009.
  * [73] H. Poincaré.  Complément à l’Analysis situs.  Rend. Circ. Mat. Palermo, 13:285–343, 1899.
  * [74] C. Ramanujam.  A topological characterisation of the affine plane as an algebraic variety.  Ann. of Math. (2), 94(1):69–88, 1971.
  * [75] M. Rapoport and T. Zink.  Über die lokale Zetafunktion von Shimuravarietäten. Monodromiefiltration und verschwindende Zyklen in ungleicher Charakteristik.  Invent. Math., 68(1):21–101, 1982.
  * [76] J. Riou.  Dualité de Spanier-Whitehead en géométrie algébrique.  C. R. Math. Acad. Sci. Paris, 340(6):431–436, 2005.
  * [77] M. Robalo.  K-theory and the bridge from motives to noncommutative motives.  Adv. Math.,, 269(10):399–550, 2015.
  * [78] O. Röndigs and P. A. Østvær.  Motives and modules over motivic cohomology.  C. R. Math. Acad. Sci Paris, 342(10):751–754, 2006.
  * [79] O. Röndigs and P. A. Østvær.  Modules over motivic cohomology.  Adv. Math., 219(2):689–727, 2008.
  * [80] J. Scholbach.  Mixed Artin-Tate motives over number rings.  J. Pure Appl. Algebra, 215(9):2106–2118, 2011.
  * [81] I. R. Shafarevich.  Basic algebraic geometry 1. Varieties in projective space. Berlin: Springer, 3rd ed. edition, 2013.
  * [82] M. Spitzweck.  A commutative ${\mathbb{P}}^{1}$-spectrum representing motivic cohomology over Dedekind domains.  Mém. Soc. Math. Fr., Nouv. Sér., 157:1–110, 2018.
  * [83] J. Steenbrink.  Limits of Hodge structures.  Invent. Math., 31(3):229–257, 1975/76.
  * [84] R. W. Thomason and T. Trobaugh.  Higher algebraic $K$-theory of schemes and of derived categories.  In The Grothendieck Festschrift, Vol. III, volume 88 of Progr. Math., pages 247–435. Birkhäuser, Boston, MA, 1990.
  * [85] V. Voevodsky.  Unstable motivic homotopy categories in Nisnevich and cdh-topologies.  J. Pure Appl. Algebra, 214(8):1399–1406, 2010.
  * [86] V. Voevodsky, A. Suslin, and E. M. Friedlander.  Cycles, Transfers, And Motivic Homology Theories, volume 143 of Annals Of Mathematics Studies.  Princeton University Press, Princeton, NJ, 2000.
  * [87] J. Wildeshaus.  The boundary motive: definition and basic properties.  Compos. Math., 142(3):631–656, 2006.
  * [88] J. Wildeshaus.  Boundary motive, relative motives and extensions of motives.  In Autour des motifs—École d’été Franco-Asiatique de Géométrie Algébrique et de Théorie des Nombres. Vol. II, volume 41 of Panor. Synthèses, pages 143–185. Soc. Math. France, Paris, 2013.