x[1]¿p#1pt y[1]¿p#1pt z[1]¿p#1pt

# Predicting masked tokens in stochastic locations improves
masked image modeling

Amir Bar1,2,3 Florian Bordes3,5 Assaf Shocher2 Mahmoud Assran3 Pascal
Vincent3,5
Nicolas Ballas3 Trevor Darrell2 Amir Globerson1 Yann LeCun3,4
1Tel Aviv University 2UC Berkeley 3Meta AI (FAIR) 4New York University 5Mila,
Université de Montréal

###### Abstract

Self-supervised learning is a promising paradigm in deep learning that enables
learning from unlabeled data by constructing pretext tasks that require
learning useful representations. In natural language processing, the dominant
pretext task has been masked language modeling (MLM), while in computer vision
there exists an equivalent called Masked Image Modeling (MIM). However, MIM is
challenging because it requires predicting semantic content in accurate
locations. E.g, given an incomplete picture of a dog, we can guess that there
is a tail, but we cannot determine its exact location. In this work, we
propose FlexPredict, a stochastic model that addresses this challenge by
incorporating location uncertainty into the model. Specifically, we condition
the model on stochastic masked token positions to guide the model toward
learning features that are more robust to location uncertainties. Our approach
improves downstream performance on a range of tasks, e.g, compared to MIM
baselines, FlexPredict boosts ImageNet linear probing by 1.6% with ViT-B and
by $2.5\%$ for semi-supervised video segmentation using ViT-L.

Figure 1: Given a partial image of a dog, can you precisely determine the
location of its tail? Existing Masked Image Modeling (MIM) models like [24, 1]
are deterministic and predict masked tokens conditioned on fixed positions
(a), while FlexPredict predicts masked tokens conditioned on stochastic
positions (b). This guides our model to learning features that are more robust
to location uncertainties and leads to improved performance when compared to
similar MIM baselines. E.g, FlexPredict improves linear probing on ImageNet
(c).

##  1 Introduction

Self-supervised learning (SSL) has emerged as a promising paradigm in deep
learning. By constructing pretext training tasks, it’s possible to leverage
unlabeled data to learn representations that can be transferred across a wide
range of downstream tasks. This approach has shown remarkable progress in
various domains, including natural language processing [16, 8, 15], speech
recognition [4, 2, 44], and computer vision [50, 35, 10, 24].

In NLP, masked language modeling (MLM) has emerged as a prominent pre-training
task. MLM’s primary goal is to predict masked parts in a text based on rest of
the text. This task is an essential component of the training process for
popular models such as BERT [16], GPT [8], and similar models. Likewise, in
computer vision, there exists a natural counterpart to MLM, known as Masked
Image Modeling (MIM). In MIM, part of an image is masked, and the pretext task
is to complete it. While this approach has been considered for quite some time
[35] and is a form of denoising auto-encoders [42], the dominant approach to
semi-supervised learning (SSL) in computer vision relies on learning
representations that are invariant to handcrafted image augmentations [39, 22,
6]. Although these approaches produce highly semantic representations, they
necessitate prior knowledge of task-specific invariances [46].

More recently, new MIM methods have emerged. Masked Auto-Encoders (MAE) [24],
which are trained to minimize a reconstruction error in pixel space, have
demonstrated competitive performances in fine-tuning with respect to SSL
methods relying on handcrafted image augmentations. Some follow up works have
removed the pixel space decoder to allow reconstruction directly in the latent
space [3, 53, 1]. The most recent is I-JEPA [1], which stressed the importance
of masking large blocks, and of predicting latent representations rather than
pixel values. These works have narrowed the gap between MIM methods and
invariance-based methods. However, the latter still outperforms the former on
tasks such as ImageNet linear probing.

Here we argue that MIM suffers from an inherent difficulty that makes it
challenging to learn representations. For instance, let’s take a partial image
of a dog, as depicted in Figure 1. We know that the image contains the tail of
the dog, but we cannot predict its precise location. Yet, current MIM methods
do not model this uncertainty and attempt to provide an accurately localized
prediction.

In this work, we propose a solution to address this challenge by introducing a
stochastic MIM model. There are various approaches to achieve this, and we
suggest a simple yet effective one. Instead of training the model to make
predictions in exact locations, we introduce noise to masked tokens positions,
thereby forcing the model to make stochastic predictions. This approach guides
the model towards features that are more resilient to location uncertainties,
such as the fact that a tail exists somewhere in a broad region of the image.
However, it is crucial to design the noise injection method carefully, so that
the model does not merely scale down weights to “overcome” the noise. We
demonstrate how to tackle this issue in our proposed method.

Our contributions are twofold. First, we propose a novel approach for MIM that
addresses the uncertainty in the MIM pretext task (e.g, the location of
semantic features in the image is stochastic). Second, we demonstrate that our
approach outperforms existing methods across a variety of downstream tasks,
highlighting its effectiveness.

##  2 Related Work

Invariance-based methods. Invariance-based methods involve training an encoder
to ensure similar augmentations of the same image have similar representations
while avoiding a trivial solution. For example, contrastive learning is used
to prevent collapse to trivial solution by introducing negative examples [23,
18, 10, 25, 12, 19]. This can be achieved using a memory bank of previous
instances [45, 34, 39, 33]. However, there are also non-contrastive solutions
that have been proposed. Of particular interest, a momentum encoder has been
shown to prevent collapse even without the use of negative pairs [22, 9, 38].
Other methods include stopping the gradient to one branch [13] or applying
regularization using batch statistics [48, 6, 7, 20, 26]. Our approach is
based on MIM, which doesn’t require assumptions on batch statistics or
handcrafted invariances.

Masked image modeling (MIM). There is a significant body of research exploring
visual representation learning by predicting corrupted sensory inputs.
Denoising autoencoders [43], for example, use random noise as input
corruption, while context encoders [35] regress an entire image region based
on its surrounding. The idea behind masked image modeling [24, 47, 5] has
emerged as a way to address image denoising. In this approach, a Vision
Transformer [17] is used to reconstruct missing input patches. The Masked
Autoencoders (MAE) architecture [24], for example, efficiently reconstructs
missing patches in pixel space and achieves strong performance on large
labeled datasets. Other approaches, such as BEiT [5], predict a latent code
obtained using a pretrained tokenizer. However, pixel-level pre-training has
been shown to outperform BEiT in fine-tuning. SimMiM [47] explores simple
reconstruction targets like color clusters but shows no significant advantages
over pixel space reconstruction.

Joint embedding predictive architecture (JEPA). The recently proposed JEPA
[32] framework generalizes both the invariance-based and MIM approaches under
the same umbrella. iBOT [53] is a state-of-the-art representation learning
method that combines both global invariance loss and a MIM based loss, using
an online tokenizer. Recently, Image-JEPA (I-JEPA) [1] was proposed as a non-
generative approach for self-supervised learning of semantic image
representations. I-JEPA predicts the representations of various target blocks
in an image from a single context block to guide it toward producing semantic
representations. We propose FlexPredict, a model that focuses on the
prediction of coarse and more semantic features.

##  3 Preliminaries

Figure 2: FlexPredict architecture. The model predictor $g_{\psi}$ predicts a
target block given masked tokens with stochastic positions and the context
representation (obtain via $f_{\theta}$). The objective is to minimize the
error between the predicted features and the target features obtained via
target encoder $f_{\bar{\theta}}$

Our work leverages the I-JEPA framework [1], which we introduce by outlining
its key concept. Specifically, I-JEPA is designed to predict the features of
target blocks, based on contextual blocks from the same image. We proceed to
elaborate on this in more detail.

#### Patchification.

Given an image, the standard tokenization process presented at [17] is
applied. Specifically, given an input image $I_{x}\in\mathbb{R}^{H\times
W\times 3}$, it is first patchified into a sequence of non-overlapping image
patches $\hat{p}=(\hat{p}_{1},...,\hat{p}_{k})$ where
$\hat{p}_{i}\in\mathbb{R}^{H^{\prime}\times W^{\prime}\times 3}$ and
$K=\frac{HW}{H^{\prime}W^{\prime}}$ is the number of patches. Then, each patch
is projected to $\mathbb{R}^{d_{e}}$ through a linear fully connected layer.
Next, for every patch $\hat{p}_{i}$ the positional embedding features of the
$i^{th}$ token are added to it, resulting in the patchfied set
$p=\\{p_{1},...p_{K}\\}$.

#### Context encoding.

Let $x=\\{p_{i}|i\in B_{x}\\}$ be the set of context patches where $B_{x}$
denotes the set of context indices. The set of context tokens is randomly
chosen as in [1]. First, the context tokens are processed via an encoder model
$f_{\theta}$ to obtain deep representations:

${s}_{x}=f_{\phi}(x)$

Where $s_{x_{i}}\in\mathbb{R}^{d_{e}}$ is the $i^{th}$ context token
representation.

#### Prediction and loss.

First, a target block of patches is randomly chosen (e.g, tokens annotated in
yellow in Figure 2). We denote its corresponding patch indices by $B_{y}$.
Next, we define $m=\\{\psi_{j}+\tilde{m}\\}_{j\in B_{y}}$ to be the set of
masked tokens, where for each $j\in B_{y}$, token $m_{j}$ is a summation of a
learned masked token $\tilde{m}$, shared across all tokens, and a positional
embedding $\psi_{j}$. The predictor $g$ is then used to map from the context
tokens and masked tokens to the predicted tokens: $\hat{s}_{y}=g(s_{x},m)$.

To supervise the prediction, $s_{y}=\\{s_{y_{i}}\\}_{i\in B_{y}}$ is obtained
by feeding the patchified image tokens $p$ into a target encoder
$f_{\bar{\theta}}$, then selecting the tokens corresponding to $B_{y}$.
Finally, the loss is the mean squared error between $s_{y}$ and the predicted
tokens $\hat{s}_{y}$:

$\frac{1}{\lvert B_{y}\rvert}\sum_{i\in B_{y}}\|s_{y_{i}}-\hat{s}_{y_{i}}\|$
(1)

Here $s_{y}$ is taken as constant, and the parameters of the target encoder
$f_{\bar{\theta}}$ are updated via an exponential moving average of the
context encoder $f_{{\theta}}$ which has shown to prevent collapse [9, 22].

##  4 FlexPredict

The I-JEPA method and other MIM-like approaches condition the predictor model
on the locations of the target patches, given by the masked tokens positional
embeddings, and train the model to predict their content (either in pixel or
latent space). This approach does not take into account that the exact
location of objects is highly stochastic.

Instead, we force our model to be more flexible in representing locations by
conditioning our model on stochastic positions, such that it is impossible to
provide a location-accurate prediction. Hence, we refer to our approach as
FlexPredict. A high-level schematic view of the model is included in Figure 2.

In what follows, we will explore the process of replacing the positional
embeddings of the masked tokens with a stochastic alternative. This involves a
few crucial steps, including defining the distribution of the stochastic
positions, parameterizing it appropriately, and implementing measures to
prevent the model from reducing the impact of the noise to the point where it
becomes negligible.

#### Stochastic Position Embeddings.

In most Visual Transformer implementations, the position of a patch $i$ is
encoded via an embedding vector $\psi_{i}$. A common choice is to map the
position to sine and cosine features in different frequencies [41, 17]. Here
we wish to replace this fixed, deterministic mapping with a stochastic map.
This is contrary to past works that use a deterministic mapping to determine
the positional embedding of a token [1, 24].

Given a position $i$, we denote by $\hat{\psi}_{i}$ the random variable
providing the position embedding. We assume:

$\hat{\psi}_{i}\sim N(\psi_{i},\Sigma)$ (2)

Namely, $\hat{\psi}$ is distributed as Gaussian whose mean is the fixed
embedding $\psi_{i}$, and covariance matrix $\Sigma\in\mathbb{R}^{d_{p}\times
d_{p}}$.

Naturally, we want to learn an optimal $\Sigma$. However, this is challenging
for two reasons. First, learning might result in the optimization process
setting the values of $\Sigma$ to zero, leading to no randomness. We refer to
this case as a “shortcut solution”. Second, the sampling process of
$\hat{\psi}$ is non-differential, and therefore we cannot derive gradients to
directly optimize it with SGD.

To solve these issues, we start by paramertizing $\Sigma$, then describe how
to avoid the “shortcut solution”, and the reparametrization trick to derive a
differential algorithm. We start by parameterizing $\Sigma$, and use a general
formulation of a low-rank covariance matrix:

$\Sigma=\sigma AA^{T}$

Where $A\in\mathbb{R}^{d_{p}\times d_{e}}$ is a learned matrix and
$\sigma\in\mathbb{R^{+}}$ is a positive predefined scalar (hyperparameter). By
learning matrix $A$, this formulation is flexibile enough, e.g, it is possible
learning to assign small variance to low-res location features, while
assigning higher variance to higher-frequency features, and also capturing
correlations between location features.

#### Avoiding “shortcuts”.

Without posing any constraints on $A$, it is easy for the model to scale down
the noise by setting $A=0$ and making the prediction problem deterministic
again, and thereby easier. This would collapse back to the standard I-JEPA
model, and lose the advantage of noisy spatial predictions. To avoid this
shortcut, we use the following simple trick. We use the matrix $A$ to linearly
project every context token $s_{x_{i}}$ as follows:
$\hat{c}_{i}=As_{x_{i}}+b$, where $b$ is a learned bias. With this simple
trick, it is easy to see that setting $A$ to zero would set the context tokens
to zero as well, making the prediction task too difficult for the network and
successfully avoiding the above shortcut. This can also be viewed as a
regularization of $A$, and we discuss this further in Section 7.

#### Reparametrization Trick.

Since $\hat{\psi}$ is sampled from a parameterized distribution, it isn’t
immediately clear how to optimize over the learned parameters of the
distribution $A$, because the sampling operation is non-differentiable in $A$.
However, a standard trick in these cases is to reparameterize the distribution
so that only sampling is from a fixed distribution that does not depend on $A$
(e.g., see [29]). Specifically, we generate samples from $\hat{\psi}$ by first
sampling a vector $n_{i}\in\mathbb{R}^{d_{e}}$ from a standard Gaussian
distribution: $n_{i}\sim N(0,\sigma I)$. Then we set $\hat{\psi}$ to the
following function:

$\displaystyle\hat{\psi}_{i}=An_{i}+\psi_{i}$

The resulting distribution of $\hat{\psi}$ is equal to that in Equation 2,
however, we can now differentiate directly through $A$.

#### Prediction and loss.

Finally, for every $i\in B_{x}$ and $j\in B_{y}$, we define the set of context
and masked tokens to be:

$\displaystyle c_{i}=\psi_{i}+\hat{c}_{i}$ $\displaystyle
m_{i}=\hat{\psi}_{i}+\tilde{m}$

Note that here the masked token $m_{i}$ has a stochastic position, and
$\tilde{m}$ is a learned bias shared across all positions. We can then apply
$g$ to predict the target features $\hat{s}_{y}=g(c,m)$ and use the same loss
as in Equation 1.

Algorithm 1 FlexPredict pseudo-code

1:Input: num iterations $K$, image dist $S$, hyperparam $\sigma$,

2:params ${A,b,\tilde{m}}$, encoder $f_{\theta}$, target-encoder
$f_{\bar{\theta}}$, predictor $g_{\phi}$, scalar $q$

3:Initialize: $\bar{\theta}=\theta$

4:for $i=1,2,...,K$ do

5: # sample image mini-batch, apply mask, and encode

6: $I_{x}\sim S$

7: $x,y\leftarrow\text{context\\_target\\_mask}(I_{x})$

8: $s_{x},s_{y}\leftarrow f_{\theta}(x),f_{\bar{\theta}}(y)$

9: # apply FlexPredict

10: $n\sim\mathcal{N}(0,\sigma I)$

11: $m=An+\psi_{masked}+\tilde{m}$

12: $c=As_{x}+b+\psi_{context}$

13: # predict targets and compute loss

14: $\hat{s}_{y}\leftarrow g_{\phi}(c,m)$

15:
$\text{loss}\leftarrow||\hat{s}_{y}-{s_{y}}\text{\scriptsize.detach()}||_{2}^{2}$

16: # perform sgd step and update ${\bar{\theta}}$ via ema

17: $\text{sgd\\_step}(\text{loss};\\{\theta,\phi,A,b,\tilde{m}\\})$

18: $\bar{\theta}=q{\bar{\theta}}+(1-q)\theta.\text{\scriptsize detach()}$

19:end for

#### Optimal Predictor.

Our approach relies on using stochastic positional embeddings. Here we provide
further analysis of this prediction setting and show that the optimal
prediction is indeed to perform spatial smoothing.

Consider a random variable $X$ (corresponding to the context in our case. For
simplicity assume $X$ is just the positional embedding of the context) that is
used to predict a variable $Y$ (corresponding to the target in our case). But
now instead of predicting from $X$, we use a noise variable $Z$ that is
independent of both $X,Y$, and provide the predictor with only the noisy
result $R=g(X,Z)$. Here $g$ is some mixing function (in our case
$g(x,z)=x+z$). We next derive the optimal predictor $f(R)$ in this case.
Formally we want to minimize:

$E_{R,Y}[(f(R)-Y)^{2}]$ (3)

A classic result in estimation is that this is optimized by the conditional
expectation $f(r)=E[Y|R=r]$. We simplify this as follows:

$\displaystyle E[Y|R=r]$ $\displaystyle=$
$\displaystyle\sum_{x,y}yp(Y=y,X=x|R=r)$ $\displaystyle=$
$\displaystyle\sum_{x,y}yp(y|X=x)p(X=x|R=r)$ $\displaystyle=$
$\displaystyle\sum_{x}E[Y|X=x]p(X=x|R=r)$

where in the second line we used the fact that:

$p(y,x|r)=p(y|x,r)p(x|r)=p(y|x)p(x|r)$ (4)

To further illustrate, consider the case where $z$ is Gaussian with zero mean
and unit variance. Then $p(x|r)$ is also Gaussian with expectation $r$, and
the expression above amounts to convolution of the clean expected values with
a Gaussian:

$\int_{x}E[Y|X=x]\frac{1}{\sqrt{2\pi}}e^{-0.5(x-r)^{2}}dx$ (5)

##  5 Experiments

Next, we turn to discuss the main experiments presented in the paper. We start
by discussing the ablation study and design choices in Section 5.1. Then in
Section 5.2, we describe the application of FlexPredict to various downstream
tasks including image recognition, dense prediction, and low-level vision
tasks.

###  5.1 Ablation study

Our primary focus was to evaluate the effectiveness of adding noise. For this
purpose, we experimented with learning $A$ given different hyper-parameter
$\sigma$. We also investigated the impact of adding noise from fixed Gaussian
distributions, namely $\Sigma=\sigma I$, without learning. Lastly, we evaluate
the effect of applying FlexPredict to context and/or masked tokens positions.

We evaluated various design options for the FlexPredict model. For each
setting, we implemented the encoder and predictor using ViT-B architecture and
pre-trained them for $300$ epochs on IN-1k. We then assessed the linear
probing performance on IN-1k using only 1% of the available labels.

###  5.2 Downstream Tasks

We conducted pre-training of the FlexPredict model on IN-1k for a period of
$600$ epochs, utilizing either ViT-B or ViT-L architectures for the encoder
and predictor. Subsequently, we proceeded to evaluate the model’s performance
on a variety of downstream tasks. We include the full implementation details
in the Supplementary Material.

Following past works, we focus on evaluating the (target) encoder
representations [24, 1], and use the standard VISSL [21] evaluation protocol
like in [1].

#### Image recognition.

For image classification, we evaluated the FlexPredict model linear probing
performance on multiple datasets, including ImageNet (IN-1k) [37], Places 205
[51], iNaturalist 2018 [40], and CIFAR 100 [30]. These datasets vary in their
size, their purpose, and the geographical environments from which the images
were captured. For example, IN-1k contains over $1.2$ million images compared
to CIFAR-100 which contains only $60,000$ images, and while IN-1k is focused
on object recognition, Places is focused on scene recognition.

#### Dense prediction.

To evaluate how well FlexPredict performs on dense prediction tasks, e.g,
tasks that require fine-grained spatial representations, we utilized the
learned model for semi-supervised video object segmentation on the DAVIS 2017
[36] dataset. We follow previous works (e.g [27, 9]) and use the pretrained
FlexPredict to extract frames features and use patch-level affinities between
frames to propagate the first segmentation mask.

#### Low-level vision.

We assessed the linear probing performance of our model on downstream tasks
related to low-level vision. These tasks included object counting and object
ordering by depth, which were evaluated using the CLEVR [28] dataset. In order
to accurately perform these tasks, the model needed to not only recognize
objects but also capture their location features.

##  6 Results

We report the ablation study results in Section 6.1, then discuss results on
various downstream tasks in Section 6.2.

###  6.1 Ablations

Figure 3: Using stochastic positional embeddings. sampling from distribution with learned covariance matrix as in FlexPredict, e.g, $\Sigma=\sigma AA^{T}$ leads to $+3.5\%$ improvement, while using a fixed covariance matrix $\Sigma=\sigma I$ leads to smaller $1.9\%$ improvement. Accuracy is based on probing using 1% of the data from IN-1k.  Method | Top-1
---|---
No Noise (I-JEPA [1]) | 54.3
Context tokens only | 55.1
Masked tokens only |  fbApp57.8
Masked + context tokens | 56.8
Table 1: Applying noise to different tokens. Applying learned noise to context
and/or masked tokens positional embeddings. Accuracy is based on linear
probing using 1% of the data from IN-1k.

We present the results comparing different noise, and the impact when changing
the hyperparam $\sigma$. Figure 3 indicates that it is optimal to learn the
parameters of the distribution as in FlexPredict, rather than use fixed
parameters. Our findings demonstrate that setting $\sigma=0.25$ leads to an
improvement of $3.5\%$ points compared to I-JEPA. Additionally, Table 1
reveals that FlexPredict is most beneficial when applied solely to masked
tokens positional embeddings, not to context.

###  6.2 Downstream Tasks

Next, we report the downstream performance of FlexPredict on image
recognition, dense prediction, and low-level vision tasks.

#### Image recognition.

In Table 2, we present the linear probing image classification results
conducted on IN-1k. Our approach, FlexPredict, achieves a performance
improvement of $1.6\%$ and $0.9\%$ when using ViT-B and ViT-L, respectively,
compared to previous MIM methods. Additionally, FlexPredict narrows the
relative performance gap from iBOT [53] by 25%. Furthermore, our approach
outperforms existing methods in downstream linear probing tasks. For example,
FlexPredict leads to over 10% improvement on CIFAR-100 using ViT-B and 1%
using ViT-L. This confirms that the learned representations lead to
improvements in a large variety of image recognition tasks.

Method | Arch. | Epochs | Top-1
---|---|---|---
MIM methods, without view data augmentations
data2vec [3] | ViT-L/16 | 1600 | 53.5
MAE [24] | ViT-B/16 | 1600 | 68.0
ViT-L/16 | 1600 | 76.0
I-JEPA [1] | ViT-B/16 | 600 | 72.9
ViT-L/16 | 600 | 77.5
FlexPredict |  fbAppViT-B/16 |  fbApp600 |  fbApp74.5
fbAppViT-L/16 |  fbApp600 |  fbApp78.4
Invariance-based methods, using extra view data augmentations
SimCLR v2 [11] | RN152 ($2\times$) | 800 | 79.1
DINO [9] | ViT-B/16 | 400 | 78.1
MoCo v3 [14] | ViT-B/16 | 300 | 76.7
iBOT [53] | ViT-B/16 | 250 | 79.8
ViT-L/16 | 250 | 81.0
Table 2: Linear-evaluation on IN-1k. FlexPredict improves linear probing performance compared to other methods that do not rely on hand-crafted view data-augmentations during pretraining. Method | Arch. | CIFAR100 | Places205 | iNat18
---|---|---|---|---
MIM methods, without view data augmentations
data2vec [3] | ViT-L/16 | 59.6 | 36.9 | 10.9
MAE [24] | ViT-B/16 | 68.1 | 49.2 | 26.8
ViT-L/16 | 77.4 | 54.4 | 33.0
I-JEPA [1] | ViT-B/16 | 69.2 | 53.4 | 43.4
ViT-L/16 | 83.6 | 56.5 | 48.4
FlexPredict |  fbAppViT-B/16 |  fbApp81.2 |  fbApp54.3 |  fbApp44.7
fbAppViT-L/16 |  fbApp84.7 |  fbApp57.2 |  fbApp49.2
Invariance-based methods, using extra view data augmentations
DINO [9] | ViT-B/16 | 84.8 | 55.2 | 50.1
iBOT [53] | ViT-B/16 | 85.5 | 56.7 | 50.0
ViT-L/16 | 88.3 | 60.4 | 57.3
Table 3: Linear-probe transfer for image classification. Linear-evaluation on downstream image classification tasks. FlexPredict significantly outperforms previous methods that also do not use augmentations (MAE and data2vec), and decreases the gap with the best view-invariance-based methods that leverage hand-crafted data augmentations during pretraining. Method | Arch. | J-Mean | F-Mean | J&F Mean
---|---|---|---|---
MIM methods, without view data augmentations
MAE [24] | ViT-B/16 | 49.4 | 52.6 | 50.9
ViT-L/16 | 52.5 | 54.3 | 53.4
I-JEPA [1] | ViT-B/16 | 56.1 | 56.2 | 56.1
ViT-L/16 | 56.1 | 55.7 | 55.9
FlexPredict |  fbAppViT-B/16 |  fbApp56.6 |  fbApp57.3 |  fbApp57.0
fbAppViT-L/16 |  fbApp58.1 |  fbApp58.7 |  fbApp58.4
Invariance-based methods, using extra view data augmentations
DINO [9] | ViT-B/16 | 60.7 | 63.9 | 62.3
iBOT [53] | ViT-B/16 | 60.9 | 63.3 | 62.1
ViT-L/16 | 61.7 | 63.9 | 62.8
Table 4: Video objects semi-supervised segmentation. The results demonstrate that compared to MIM baselines, FlexPredict learns features in a finer level of granularity. Results reported on the DAVIS 2017 [36] dataset.  Method | Arch. | Clevr/Count | Clevr/Dist
---|---|---|---
MIM methods, without view data augmentations
data2vec [3] | ViT-L/16 | 72.7 | 53.0
MAE [24] | ViT-B/16 | 86.6 | 70.8
ViT-L/16 | 92.1 | 73.0
I-JEPA [1] | ViT-B/16 | 82.2 | 70.7
ViT-L/16 | 85.6 | 71.2
FlexPredict |  fbAppViT-B/16 |  fbApp83.7 |  fbApp71.3
fbAppViT-L/16 |  fbApp85.7 |  fbApp70.2
Invariance-based methods, using extra view data augmentations
DINO [9] | ViT-B/16 | 83.2 | 62.5
iBOT [53] | ViT-B/16 | 85.1 | 64.4
ViT-L/16 | 85.7 | 62.8
Table 5: Linear-probing on low-level vision downstream tasks like object
counting (CLEVR/Count) and depth prediction (Clevr/Dist). FlexPredict
effectively captures low-level location features and it is on par or better
than I-JEPA.

#### Dense prediction.

We include semi-supervised video-object segmentation results in Table 4. We
find that FlexPredict significantly improves over I-JEPA [1], e.g, an
improvement of $2.5\%$ on $J\&F$ using ViT-L. Notably, we find that while
using I-JEPA does not lead to improvements here by scaling the model, scaling
the model to ViT-L leads to a $1.4\%$ improvement compared to ViT-B using
FlexPredict.

#### Low-level vision.

Table 5 provides evidence that the learned representations of FlexPredict
performs at least on-par with I-JEPA models in low-level tasks such as
counting and depth ordering on the CLEVR dataset.

##  7 Analysis

Figure 4: Similarity matrices of positional embeddings matrix between
deterministic ($\psi\psi_{i}$) and stochastic FlexPredict
($\psi\hat{\psi}_{i}$) positions. Each row represents a different target
position $i$. Position embeddings are based on sine and cosine features.

We perform a thorough analysis of FlexPredict. Specifically, we examine the
stochastic effect of FlexPredict and attempt to interpret the properties of
the learned model.

#### Regularization.

We train FlexPredict models, changing only the hyperparam $\sigma$. We find
that increasing the value of $\sigma$ leads to a decrease in the norm of $A$,
which can be viewed as regularization. On the other hand, increasing $\sigma$
leads to an increase in the norm of the masked token $m$. The mask token scale
increases to prevent losing its information relative to the noise. We show
this dynamic in Figure 5.

#### Regularized I-JEPA.

Based on the observations above, we train additional models to check whether
FlexPredict can be explained by regularization. Specifically, we train I-JEPA
models while applying $l_{1}$ regularization on the predictor’s linear
projection layer weights. We evaluate linear probing performance using $1\%$
of the labels and find this leads to $1.5\%$ improvement over I-JEPA, compared
to $3.5\%$ improvement using FlexPredict.

#### Stochastic positional embeddings visualization.

In order to visualize stochastic positional embeddings, we sampled stochastic
positions and generated a similarity matrix of each sample with the predefined
deterministic positions. Figure 4 provides examples of this. Our findings show
that when noise is added to a positional embedding, the resulting similarity
matrix changes, which makes it similar to a wider range of neighboring
locations.

#### Low-res prediction.

We build on the observations above and train additional I-JEPA models to
investigate if FlexPredict performance could be achieved through predicting
lower-scale features. We trained models to predict features in both the
original scale and a downscaled version by a factor of 2, using bilinear
resizing and max pooling for downscaling. However, we found that these methods
did not significantly improve performance, as reported in Table 6.

#### Predictions visualization.

We include heatmap visualization to visualize the similarity of a predicted
token to all other tokens within the same image (see Figure 6). For a given
image, mask, and a masked patch of interest, we apply cosine similarity
between the predicted patch and all other token representations within the
same image (given by the target encoder), followed by a softmax. For I-JEPA
the visualization indicates that adjacent tokens tend to share similar
features, implying a correlation between the features and spatial location. In
contrast, FlexPredict produces predictions correlated with non-neighboring
small areas. We speculate that training with stochastic positions prevents
spatial adjacency bias.

Figure 5: Increasing $\sigma$ induces regularization. The effect of changing the hyperparameter $\sigma$ on the norm of learned parameter $A$ and the masked token $m$. As we increase $\sigma$, the norm of $A$ decreases, indicating regularization. However, the norm of the masked token increases, likely to preserve its information relative to the added noise. Figure 6: Predicted features visualization. We show a similarity heatmap between the predicted features of a given patch (marked in white within the masked area) and the other tokens encoded by the target encoder in the same image. For I-JEPA, adjacent tokens tend to share similar features, implying a correlation between the features and spatial location. In contrast, FlexPredict produces predictions correlated with non-neighboring small areas. We speculate that FlexPredict reduces spatial adjacency bias.  Method | Top-1
---|---
I-JEPA [1]) | 54.3
Low res pred (bilinear resize) | 52.1
Low res (max pooling) | 54.1
FlexPredict |  fbApp57.8
Table 6: Low resolution prediction. We evaluated the performance of
FlexPredict against models that predict features on the original scale and an
$x2$ downscaled version using either max pooling or bilinear resizing.
Reporting linear evaluation results on IN-1K using only 1% of the labels

##  8 Conclusion

In this work, we proposed FlexPredict, a stochastic model that tackles
location uncertainties in the MIM task. By conditioning on stochastic masked
tokens positions, our model learns features that are more robust to location
uncertainties. The effectiveness of this approach is demonstrated on various
datasets and downstream tasks, outperforming existing MIM methods and
highlighting its potential for self-supervised learning. We speculate, based
on our experiments and visualizations, that by modeling location
uncertainties, FlexPredict suffers less from spatial adjacency bias. Other
sources of uncertainty, like uncertainty in appearance, require further
investigation in future work.

#### Acknowledgments:

AG’s group has received funding from the European Research Council (ERC) under
the European Unions Horizon 2020 research and innovation programme (grant ERC
HOLI 819080). TD’s group was funded by DoD including DARPA LwLL and the
Berkeley AI Research (BAIR) Commons. This work was completed in partial
fulfillment for the Ph.D degree of the first author.

## References

  * [1] Mahmoud Assran, Quentin Duval, Ishan Misra, Piotr Bojanowski, Pascal Vincent, Michael Rabbat, Yann LeCun, and Nicolas Ballas.  Self-supervised learning from images with a joint-embedding predictive architecture.  arXiv preprint arXiv:2301.08243, 2023.
  * [2] Alexei Baevski, Wei-Ning Hsu, Alexis Conneau, and Michael Auli.  Unsupervised speech recognition.  Advances in Neural Information Processing Systems, 34:27826–27839, 2021.
  * [3] Alexei Baevski, Wei-Ning Hsu, Qiantong Xu, Arun Babu, Jiatao Gu, and Michael Auli.  Data2vec: A general framework for self-supervised learning in speech, vision and language.  arXiv preprint arXiv:2202.03555, 2022.
  * [4] Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli.  wav2vec 2.0: A framework for self-supervised learning of speech representations.  Advances in neural information processing systems, 33:12449–12460, 2020.
  * [5] Hangbo Bao, Li Dong, and Furu Wei.  Beit: Bert pre-training of image transformers.  arXiv preprint arXiv:2106.08254, 2021.
  * [6] Adrien Bardes, Jean Ponce, and Yann LeCun.  Vicreg: Variance-invariance-covariance regularization for self-supervised learning.  arXiv preprint arXiv:2105.04906, 2021.
  * [7] Adrien Bardes, Jean Ponce, and Yann LeCun.  Vicregl: Self-supervised learning of local visual features.  arXiv preprint arXiv:2210.01571, 2022.
  * [8] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al.  Language models are few-shot learners.  Advances in neural information processing systems, 33:1877–1901, 2020.
  * [9] Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin.  Emerging properties in self-supervised vision transformers.  arXiv preprint arXiv:2104.14294, 2021.
  * [10] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton.  A simple framework for contrastive learning of visual representations.  preprint arXiv:2002.05709, 2020.
  * [11] Ting Chen, Simon Kornblith, Kevin Swersky, Mohammad Norouzi, and Geoffrey Hinton.  Big self-supervised models are strong semi-supervised learners.  arXiv preprint arXiv:2006.10029, 2020.
  * [12] Xinlei Chen, Haoqi Fan, Ross Girshick, and Kaiming He.  Improved baselines with momentum contrastive learning.  arXiv preprint arXiv:2003.04297, 2020.
  * [13] Xinlei Chen and Kaiming He.  Exploring simple siamese representation learning.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 15750–15758, 2021.
  * [14] Xinlei Chen, Saining Xie, and Kaiming He.  An empirical study of training self-supervised vision transformers.  arXiv preprint arXiv:2104.02057, 2021.
  * [15] Kevin Clark, Minh-Thang Luong, Quoc V Le, and Christopher D Manning.  Electra: Pre-training text encoders as discriminators rather than generators.  arXiv preprint arXiv:2003.10555, 2020.
  * [16] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.  Bert: Pre-training of deep bidirectional transformers for language understanding.  arXiv preprint arXiv:1810.04805, 2018.
  * [17] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al.  An image is worth 16x16 words: Transformers for image recognition at scale.  arXiv preprint arXiv:2010.11929, 2020.
  * [18] Alexey Dosovitskiy, Jost Tobias Springenberg, Martin A. Riedmiller, and Thomas Brox.  Discriminative unsupervised feature learning with convolutional neural networks.  In NIPS, 2014.
  * [19] Debidatta Dwibedi, Yusuf Aytar, Jonathan Tompson, Pierre Sermanet, and Andrew Zisserman.  With a little help from my friends: Nearest-neighbor contrastive learning of visual representations.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9588–9597, 2021.
  * [20] Aleksandr Ermolov, Aliaksandr Siarohin, E. Sangineto, and N. Sebe.  Whitening for self-supervised representation learning.  In International Conference on Machine Learning, 2020.
  * [21] Priya Goyal, Quentin Duval, Jeremy Reizenstein, Matthew Leavitt, Min Xu, Benjamin Lefaudeux, Mannat Singh, Vinicius Reis, Mathilde Caron, Piotr Bojanowski, Armand Joulin, and Ishan Misra.  Vissl.  https://github.com/facebookresearch/vissl, 2021.
  * [22] Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre H Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Daniel Guo, Mohammad Gheshlaghi Azar, et al.  Bootstrap your own latent: A new approach to self-supervised learning.  arXiv preprint arXiv:2006.07733, 2020.
  * [23] Raia Hadsell, Sumit Chopra, and Yann LeCun.  Dimensionality reduction by learning an invariant mapping.  2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), 2:1735–1742, 2006.
  * [24] Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick.  Masked autoencoders are scalable vision learners.  arXiv preprint arXiv:2111.06377, 2021.
  * [25] Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick.  Momentum contrast for unsupervised visual representation learning.  arXiv preprint arXiv:1911.05722, 2019.
  * [26] Tianyu Hua, Wenxiao Wang, Zihui Xue, Sucheng Ren, Yue Wang, and Hang Zhao.  On feature decorrelation in self-supervised learning.  In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 9598–9608, October 2021.
  * [27] Allan Jabri, Andrew Owens, and Alexei Efros.  Space-time correspondence as a contrastive random walk.  Advances in neural information processing systems, 33:19545–19560, 2020.
  * [28] Justin Johnson, Bharath Hariharan, Laurens Van Der Maaten, Li Fei-Fei, C Lawrence Zitnick, and Ross Girshick.  Clevr: A diagnostic dataset for compositional language and elementary visual reasoning.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2901–2910, 2017.
  * [29] Diederik P Kingma and Max Welling.  Auto-encoding variational bayes.  arXiv preprint arXiv:1312.6114, 2013.
  * [30] Alex Krizhevsky.  Learning multiple layers of features from tiny images.  Technical report, 2009.
  * [31] Alex Krizhevsky, Geoffrey Hinton, et al.  Learning multiple layers of features from tiny images.  2009\.
  * [32] Yann LeCun.  A path towards autonomous machine intelligence version 0.9. 2, 2022-06-27.  2022\.
  * [33] Ishan Misra and Laurens van der Maaten.  Self-supervised learning of pretext-invariant representations.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 6707–6717, 2020.
  * [34] Aaron van den Oord, Yazhe Li, and Oriol Vinyals.  Representation learning with contrastive predictive coding.  arXiv preprint arXiv:1807.03748, 2018.
  * [35] Deepak Pathak, Philipp Krahenbuhl, Jeff Donahue, Trevor Darrell, and Alexei A Efros.  Context encoders: Feature learning by inpainting.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2536–2544, 2016.
  * [36] Jordi Pont-Tuset, Federico Perazzi, Sergi Caelles, Pablo Arbeláez, Alex Sorkine-Hornung, and Luc Van Gool.  The 2017 davis challenge on video object segmentation.  arXiv preprint arXiv:1704.00675, 2017.
  * [37] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei.  Imagenet large scale visual recognition challenge.  International Journal of Computer Vision, 115(3):211–252, 2015\.
  * [38] Ruslan Salakhutdinov and Geoff Hinton.  Learning a nonlinear embedding by preserving class neighbourhood structure.  In Artificial Intelligence and Statistics, pages 412–419. PMLR, 2007.
  * [39] Yonglong Tian, Dilip Krishnan, and Phillip Isola.  Contrastive multiview coding.  In European Conference on Computer Vision, 2019.
  * [40] Grant Van Horn, Oisin Mac Aodha, Yang Song, Yin Cui, Chen Sun, Alex Shepard, Hartwig Adam, Pietro Perona, and Serge Belongie.  The inaturalist species classification and detection dataset.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 8769–8778, 2018.
  * [41] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin.  Attention is all you need.  In Advances in neural information processing systems, pages 5998–6008, 2017.
  * [42] Pascal Vincent, Hugo Larochelle, Yoshua Bengio, and Pierre-Antoine Manzagol.  Extracting and composing robust features with denoising autoencoders.  In Proceedings of the 25th international conference on Machine learning, pages 1096–1103, 2008.
  * [43] Pascal Vincent, Hugo Larochelle, Isabelle Lajoie, Yoshua Bengio, Pierre-Antoine Manzagol, and Léon Bottou.  Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion.  Journal of machine learning research, 11(12), 2010.
  * [44] Weiran Wang, Qingming Tang, and Karen Livescu.  Unsupervised pre-training of bidirectional speech encoders via masked reconstruction.  In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6889–6893. IEEE, 2020.
  * [45] Zhirong Wu, Yuanjun Xiong, Stella X Yu, and Dahua Lin.  Unsupervised feature learning via non-parametric instance discrimination.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3733–3742, 2018.
  * [46] Tete Xiao, Xiaolong Wang, Alexei A Efros, and Trevor Darrell.  What should not be contrastive in contrastive learning.  In International Conference on Learning Representations.
  * [47] Zhenda Xie, Zheng Zhang, Yue Cao, Yutong Lin, Jianmin Bao, Zhuliang Yao, Qi Dai, and Han Hu.  Simmim: A simple framework for masked image modeling.  arXiv preprint arXiv:2111.09886, 2021.
  * [48] Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, and Stéphane Deny.  Barlow twins: Self-supervised learning via redundancy reduction.  arXiv preprint arXiv:2103.03230, 2021.
  * [49] Xiaohua Zhai, Joan Puigcerver, Alexander Kolesnikov, Pierre Ruyssen, Carlos Riquelme, Mario Lucic, Josip Djolonga, Andre Susano Pinto, Maxim Neumann, Alexey Dosovitskiy, Lucas Beyer, Olivier Bachem, Michael Tschannen, Marcin Michalski, Olivier Bousquet, Sylvain Gelly, and Neil Houlsby.  A large-scale study of representation learning with the visual task adaptation benchmark, 2019.
  * [50] Richard Zhang, Phillip Isola, and Alexei A Efros.  Colorful image colorization.  2016\.
  * [51] Bolei Zhou, Agata Lapedriza, Jianxiong Xiao, Antonio Torralba, and Aude Oliva.  Learning deep features for scene recognition using places database.  In Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K.Q. Weinberger, editors, Advances in Neural Information Processing Systems, volume 27. Curran Associates, Inc., 2014.
  * [52] Bolei Zhou, Agata Lapedriza, Jianxiong Xiao, Antonio Torralba, and Aude Oliva.  Learning deep features for scene recognition using places database.  Advances in neural information processing systems, 27, 2014.
  * [53] Jinghao Zhou, Chen Wei, Huiyu Wang, Wei Shen, Cihang Xie, Alan Yuille, and Tao Kong.  Ibot: Image bert pre-training with online tokenizer.  arXiv preprint arXiv:2111.07832, 2021.

## Appendix

##  Appendix A Implementation Details

We include the full implementation details, pretraining configs and evaluation
protocols for the Ablations (see Appendix A.1) and Downstream Tasks (Appendix
A.2).

###  A.1 Ablations

Here we pretrain all models for $300$ epochs using $4$ V100 nodes, on a total
batch size of $2048$. In all the ablation study experiments, we follow the
recipe of [1]. We include the full config in Table 7.

To evaluate the pretrained models, we use linear probing evaluation using 1%
of IN-1k [37]. To obtain the features of an image, we apply the target encoder
over the image to obtain a sequence of tokens corresponding to the image. We
then average the tokens to obtain a single representative vector. The linear
classifier is trained over this representation, maintaining the rest of the
target encoder layers fixed.

###  A.2 Downstream Tasks

Here we pretrain FlexPredict for $600$ epochs using $4$ V100 nodes, on a total
batch size of $2048$ using ViT-B (see config in Table 8) and ViT-L (see config
in Table 9). We follow a similar config compared to [1] except we use a lower
learning rate. Intuitively, since FlexPredict is stochastic it is more
sensitive to high learning rates.

For evaluation on downstream tasks, we use the features learned by the target-
encoder and follow the protocol of VISSL [21] that was utilized by I-JEPA [1].
Specifically, we report the best linear evaluation number among the average-
pooled patch representation of the last layer and the concatenation of the
last $4$ layers of the average-pooled patch representations.

For baselines that use Vision Transformers [17] with a [cls] token (e.g, iBOT
[53], DINO [9] or MAE [24]), we use the default configurations of VISSL [21]
to evaluate the publicly available checkpoints on iNaturalist18 [40], CIFAR100
[31], Clevr/Count [28, 49], Clevr/Dist [28, 49], and Places205 [52]. Following
the evaluation protocol of VISSL [21], we freeze the encoder and return the
best number among the [cls] token representation of the last layer and the
concatenation of the last $4$ layers of the [cls] token.

For semi-supervised video object segmentation, we propagate the first labeled
frame in a video using the similarity between adjacent frames features. To
label the video using the frozen features, we follow the code and hyperparams
of [9]. To evaluate the segmented videos, we use the evaluation code of DAVIS
2017 [36].

config | value
---|---
optimizer | AdamW
epochs | 300
learning rate | $1e^{-3}$
weight decay | $(0.04,0.4)$
batch size | 2048
learning rate schedule | cosine decay
warmup epochs | 15
encoder arch. | ViT-B
predicted targets | 4
predictor depth | 6
predictor attention heads | 12
predictor embedding dim. | 384
$\sigma$ (noise hyperparam) | $0.25$
Table 7: Pretraining setting for ablations. Using ViT-B encoder, trained for $300$ epochs, config strictly follows [1]. config | value
---|---
optimizer | AdamW
epochs | $600$
learning rate | $8e^{-4}$
weight decay | $(0.04,0.4)$
batch size | $2048$
learning rate schedule | cosine decay
warmup epochs | 15
encoder arch. | ViT-B
predicted targets | 4
predictor depth | 6
predictor attention heads | 12
predictor embedding dim. | 384
$\sigma$ (noise hyperparam) | $0.25$
Table 8: Pretraining setting for downstream tasks (ViT-B). All models trained for $600$ epochs. config | value
---|---
optimizer | AdamW
epochs | $600$
learning rate | $8e^{-4}$
weight decay | $(0.04,0.4)$
batch size | $2048$
learning rate schedule | cosine decay
warmup epochs | 15
encoder arch. | ViT-L
predicted targets | 4
predictor depth | 12
predictor attention heads | 16
predictor embedding dim. | 384
$\sigma$ (noise hyperparam) | $0.25$
Table 9: Pretraining setting for downstream tasks (ViT-L). All models trained
for $600$ epochs.


# A ’MAP’ to find high-performing soft robot designs: Traversing complex
design spaces using MAP-elites and Topology Optimization

Yue Xie1† and Josh Pinskier2† and Lois Liow2 and David Howard2 and Fumiya
Iida1 1Yue Xie and Fumiya Iida are with Department of Engineering, University
of Cambridge, Cambridge, CB2 1PZ, UK.2Josh Pinskier, Lois Liow and David
Howard with the Data 61, CSIRO, 1 Technology Ct, Pullenvale QLD 4069,
Australia.† Equal contributions as first authors

###### Abstract

Soft robotics has emerged as the standard solution for grasping deformable
objects, and has proven invaluable for mobile robotic exploration in extreme
environments. However, despite this growth, there are no widely adopted
computational design tools that produce quality, manufacturable designs. To
advance beyond the diminishing returns of heuristic bio-inspiration, the field
needs efficient tools to explore the complex, non-linear design spaces present
in soft robotics, and find novel high-performing designs. In this work, we
investigate a hierarchical design optimization methodology which combines the
strengths of topology optimization and quality diversity optimization to
generate diverse and high-performance soft robots by evolving the design
domain. The method embeds variably sized void regions within the design domain
and evolves their size and position, to facilitating a richer exploration of
the design space and find a diverse set of high-performing soft robots. We
demonstrate its efficacy on both benchmark topology optimization problems and
soft robotic design problems, and show the method enhances grasp performance
when applied to soft grippers. Our method provides a new framework to design
parts in complex design domains, both soft and rigid.

###### Index Terms:

Soft Robotics, Manipulation, Topology optimization, Quality diversity

##  I INTRODUCTION

The rapid rise of soft robotics, an emerging research field which uses soft
and flexible materials to design robots which can deform around objects and
reconfigure to match their environment, has created a paradox: evermore work
is being produced by the field, (both academic and industrial), but few major
design innovations are being generated. Most new designs are based on standard
designs such as pneunets and tendon-driven continuum manipulators [1, 2, 3],
which have now existed for over a decade. This situation can be attributed in
part to the technological advancement within the field; however, the primary
reason lies in the persistent difficulties that designers encounter when
developing innovative soft robots [4]. Soft robots are complex devices which
exploit their material, structure and the environment to function effectively
[5]. They operate with rather than on the environment, a critical difference
which vastly increases the modelling complexity compared to traditional
precisely configurable, rigid-linked robots. The complex interplay between
materials, structure and behavior has proven difficult to resolve. As such no
general purpose design tools exist to generate novel, physically-realisable,
high-performing soft robots, with most new designs being created heuristically
based on standard actuators [6].

Figure 1: Example of high-performing soft robotic grippers generated through
OIDD method.

To address this problem, researchers have explored strategies including
evolutionary computation [7], reinforcement learning [8, 9], topology
optimization (TO) [10, 11, 12, 13, 14], and other learning-based methods [15,
16]. However, methods for learning complex designs (complete robot designs and
actuation/gaits) in simulation are typically based on fast computer graphics
engines. Their high-speeds necessitate abstracting away the real-world physics
required to manufacture a functioning robot. In contrast, detailed
optimization methods like topology optimization capture this physics in their
accurate finite element method solvers, but use gradient-based solves which
are unable to explore a complex design domain and find only a single design
candidate. The selection of an appropriate design domain (bounding shape of
design, design discretization/parameterization, initial guess) is an essential
element in TO, particularly in highly non-linear and poorly understood
problems. The challenge in these contexts is that the optimal choice for a
design domain is often neither straightforward nor intuitive, and initial
guesses must be close to the optimal solution for the solver to converge. We
propose a detailed design optimization method which combines quality diversity
methods with topology optimization (TO) to explore the design domain of TO
problems, enabling diverse and high-performing design candidates to emerge,
whilst retaining the detailed physics solvers, which enable designs both to be
manufactured and to function in reality as they do in simulation (cross the
reality gap). Our methodology enables systematic and generative exploration of
the design domain, thereby mitigating human bias and enhancing design
outcomes.

Topology Optimization [17, 18, 19] is a mature design optimization technique,
commonly used in structural engineering and architectural design. It is a
powerful technology which has been used across multiple physical domains
including thermal, structural, and fluidic problems [20, 21, 22] as well as
physics-informed point-cloud reconstruction [23], but is most commonly used to
design stiff-lightweight structures for automotive and aerospace applications
[24]. Despite TO’s remarkable achievements, it encounters challenges that stem
from its focus on identifying single local optima, limiting exploration across
varied design domains. This narrow focus restricts the generation of diverse
design solutions critical for achieving optimal performance in complex and
dynamic scenarios [25, 26]. These limitations highlight the need for
innovative research methodologies that enable the design domain to evolve
dynamically and intelligently throughout the optimization process, replacing
the fixed design domain of traditional TO methods with a flexible one that
adapts and reshapes during optimization.

We present the Optimizing Initial Design Domain (OIDD) technique, a novel
optimization methodology for soft robots which dynamically adjust the initial
conditions and design domain of the TO solver to explore the space, using the
placement of cavities to force the TO solver to find multiple, unconventional
designs. It leverages MAP-Elites [27], a well-known quality-diversity
algorithm, to navigate and exploit the high-dimensional design spaces. Through
the dynamic adjustment of the initial design domain-by incorporating void
regions of varying sizes and locations—OIDD facilitates the exploration of the
design domain, fostering the emergence of diverse and high-performance
structural configurations. An example soft gripper designed using the OIDD
method is presented in Figure 1, demonstrating the ability to design novel,
functional soft-robots.The method is universal in nature and can be applied to
topology optimization problems across physics and domains, however, here we
focus on soft robotics, and demonstrate its efficacy in this challenging
domain.

Our main contributions are:

  1. 1.

A novel evolutionary methodology to adapt the design domain during topology
optimization

  2. 2.

A topology optimization formulation for multi-jointed inflatable grippers

  3. 3.

A set of diverse soft grippers, which outperform comparable benchmarks in
grasp testing

##  II Enhancing Structural Design through Optimized Initial Domains

1 for  _iteration $\in[1,I]$_ do

2 if  _First iteration_ then

3 $\mathcal{B}\leftarrow$ random initial solutions

4 else

5 $\mathcal{B}\leftarrow$ select solutions from archive $\mathcal{A}$

6 $\tilde{\mathcal{B}}=(\tilde{x_{j}})_{j\in[1,N]}\leftarrow$ new
solutions(undergo variations) ;

7 for  _$j\in[1,N]$_ do

8 run SIMP, get return $F(x_{j})$ and $M(x_{j})$ ;

9 cell $\leftarrow$ get grid cell of descriptor $\tilde{d}(M(x_{j}))$;

10 $x_{cell}\leftarrow$ get content of cell;

11 if  _$x_{cell}$ is None_ then

12 Add $x_{j}$ to cell

13 else if  _$F(x_{j}) <F(x_{cell})$_ then

14 Replace $x_{cell}$ with $x_{j}$ in cell

15 else

16 Discard $x_{j}$

return archive $\mathcal{A}$

Algorithm 1 OIDD method

In this section, we present the development of the Optimized Initial Design
Domain (OIDD) method and its integration with the Solid Isotropic Material
with Penalization (SIMP) framework. At its core, the SIMP method discretizes a
design domain into a set of elements, representing the material. The elements
are allowed to occupy continuous distribution from $0$ to $1$, with $0$
representing a void region and $1$ being solid material. For convenience the
design domain is typically a rectangle in 2D (cuboid in 3D) and divided into a
set of square (quadrilateral) or cube (hexahedron) elements. Although
convenient, this is likely a sub-optimal configuration, as the design domain
strongly influences the resulting designs in TO problems. A detailed review of
the SIMP formulation is beyond the scope of this work, but its formulation is
well-established and can be found in [28, 29].

Addressing the limitations of conventional TO, the OIDD strategy employs the
MAP-Elites algorithm to introduce variability in the initial design stages.
Incorporating voids of differing sizes and placements within the design domain
enables a comprehensive exploration of potential configurations and their
impacts on structural performance. Whilst a pure evolution driven optimization
is appealing, it leads to non-physical solutions with disjoint regions [30].
Our methodology ensures a balanced examination of design alternatives,
optimizing for structural integrity and material efficiency.

###  II-A OIDD Algorithm

The algorithm framework, for the method is outlined in Algorithm 1, its
detailed function is as follows:

  1. 1.

Initialization of Design Domains We begin by generating an array of initial
design domains, each featuring unique void region configurations. This step
lays the groundwork for extensive design exploration by ensuring a diverse set
of starting points.

  2. 2.

Archive Creation Using MAP-Elites The MAP-Elites algorithm creates an archive
($\mathcal{A}$), a multi-dimensional grid, where each cell in the grid
represents a unique combination of design features (descriptors) such as the
location and size of void regions. This categorization promotes diversity by
ensuring different design configurations are explored and stored.

  3. 3.

Topology Optimization and Performance Evaluation Each initial design domain
undergoes topology optimization using the SIMP methodology. The performance of
each optimized design is then evaluated based on problem-specific objectives,
such as structural compliance.

  4. 4.

Integration into MAP-Elites Grid Each optimized design is placed into the
appropriate grid cell in the MAP-Elites archive ($\mathcal{A}$) based on its
descriptors. If a grid cell already contains a design (an elite), the new
design replaces the current elite if it has better performance. This ensures
that only the best-performing designs for each descriptor combination are
retained.

  5. 5.

Domain Evolution The design domains are evolved with each new generation by
applying evolutionary operators (such as mutation and crossover) to the void
region configurations. This gradual improvement in the positioning and sizing
of voids encourages continuous design enhancement.

Steps (3)-(5) are then repeated until the computational budget (maximum number
of iterations) is reached. Throughout this iterative process, the MAP-Elites
archive ($\mathcal{A}$) is continuously updated, storing and categorizing the
best designs based on their descriptors. The resulting archive showcases a
diverse set of high-performing designs, each representing a unique combination
of features.

###  II-B Encoding strategy of initialization of design domain

Figure 2: Illustration of the encoded design domain initialization. a) 2D
Design Domain Initialization b) 3D Design Domain Initialization

The initialization of the design domain impacts the strategic implementation
of the OIDD method. Our approach refines the traditional TO design process by
integrating decision variables that define void configurations, including the
number of void regions, locations, and dimensions within the initial design
domain. Introducing the number of void regions as a variable adds depth to our
analysis, allowing for a detailed investigation into how variations in void
quantity influence structural durability and functionality. Specifying each
void’s position and size further enables examining their collective impact on
load distribution and stress endurance across the structure. Figure 2
illustrates the design domain initialization within OIDD methodology. In
Figure 2, we see a 2D representation of the design domain, where void regions
are distributed in varying sizes and locations, demonstrating the initial
setup’s adaptability and diversity. Figure 2 extends this concept into three
dimensions, highlighting how void activation—depicted by the presence or
absence of cyan blocks within the structure—plays a crucial role in defining
the design’s structural integrity and potential functionality.

We apply an encoding strategy that represents the decision variables as a
concatenated string that enhances the granularity and flexibility of design
representation. This encoding strategy employs a five-bit string denoted as
$v=\\{x,y,l,w,a\\}$ for each potential void region, designed to encode the
top-left vertex coordinates $(x,y)$, the dimensions of the void (length $l$
and width $w$), and a binary indicator $a$ denoting the void’s activation
within the design domain. The fifth bit of each void region is crucial for
whether the void region actively contributes to the design ($1$) or remains
latent ($0$), thus enabling a dynamic exploration of the design space by
selectively activating or deactivating void regions.

This encoding strategy establishes a robust framework for systematically
investigating diverse design configurations. In each optimization, we set the
maximum available number of void regions, $n$. Therefore, the length of the
decision variables is $5n$ for 2D TO problems and $7n$ for 3D problems.

###  II-C Diversity Feature selection

To generate a diverse pool of designs, a set of features are required which
capture the designs’ unique properties and differences. Here we use two
features to quantify diversity arising from void configuration: Shannon
entropy and void-centroid distance.

Shannon entropy indicates the distribution of void sizes within a design,
providing a quantitative assessment of uniformity in material distribution. To
evaluate this feature for a set of decision variables, we first calculate the
volume of each void region based on its dimensions (length, width, and height,
as specified by the decision variables) and normalize these volumes relative
to the total volume of all voids within the design. Then, we apply the Shannon
entropy formula $H=-\sum(p_{i}\log p_{i})$, where $p_{i}$ is the normalized
volume of the $i$-th void. This calculation transforms the distribution of
void volumes into a single metric that encapsulates the diversity of void
sizes, offering insights into the design’s balance between structural strength
and material usage.

Beyond just void size, the spatial arrangement of voids significantly
influences structural performance. To evaluate the diversity of this feature,
we use a distribution metric that examines the distances between void
centroids, assessing the strategic dispersion of voids. This metric highlights
the clustering or uniform spread of voids and aids in understanding their
collective impact on structural performance. We calculate the centroid based
on each void’s top-left vertex and dimensions. Then, we compute the Euclidean
distances between each pair of void centroids to measure the spatial
dispersion of voids within the design domain. Finally, we employ statistical
measures (mean, variance, and standard deviation) to quantify the spatial
distribution. A lower variance indicates a more uniform distribution of voids,
while a higher variance suggests clustering or irregular dispersion.

Together, these two features provide a multidimensional characterization of
design possibilities within the MAP-Elites archive. Our strategy captures a
broad spectrum of design diversity, ensuring that each solution within the
MAP-Elites archive is distinct and informative, expanding the horizons of what
is achievable in topology optimization.

##  III Evaluation on Benchmark Problem

Figure 3: Loading and boundary conditions of MBB beam. Left: full design
domain, right: half design domain with symmetry boundary conditions.

###  III-A The MBB Problem

In this section we evaluate the performance of the Optimized Iterative Design
Diversification (OIDD) methodology on a benchmark TO problem, the MBB beam
[31]. While the problem has been thoroughly explored within the TO community,
many high-performing configurations exist. Hence, beyond demonstrating the
validity of the OIDD method, finding diverse MBB designs is also of interest
to researchers and practitioners.

The MBB problem is a structural compliance problem, similar to designing a
bridge. The optimization problem aims to minimize the structural compliance of
the beam subject to a material volume constraint, hence maximising stiffness
for a given mass. The initial design domain and boundary conditions for this
problem are depicted in Figure 3.

This example operates under a fixed termination condition set at $50$ OIDD
(outer-loop) iterations. The initial population for each run is set at $20$,
with the algorithm producing $10$ offspring in every iteration. This
configuration ensures a balanced approach between exploration and exploitation
within the design space. Each design is topology optimized for $50$ iterations
in an inner-loop, and the design domain contains $200\times 100$ elements,
with upto $10$ void regions.

Figure 4: Analysis of MBB Beam Optimization. a) Evolution of archive coverage
and corresponding optimal designs. b) Distribution of high-performing
solutions in the behavior space of MBB beam

###  III-B MBB Results

The results of the OIDD optimization on the benchmark MBB problem are
presented in Figure 4, showing the growth of the archive and the overall best
design as the optimization progresses (Figure 4), and the final archive along
with selected design (Figure 4). Overall 20 elite designs were found in the 64
cell archive.

Initially, the archive expands quickly as every cell is empty and hence any
valid entry qualifies as elite. However there are diminishing returns as the
cells fill up, making it harder and harder to find valid and elite entries
This could indicate that the solution space has been thoroughly explored or
that the algorithm’s parameters are tuned towards intensification over
diversification as the search progresses. The MBB design resulting from a
standard topology optimization run is shown in the bottom-right in Figure 4
(random domain initialization without any void regions). The other four
designs are the best performing designs chosen from the archive corresponding
to their iterations. The numbers accompanying each design represent the
compliance objective (lower is better). Evidently, several distinct designs
can be found which outperform standard TO.

Figure 4 complements this by showing the distribution of high-performing
solutions in the behavioral space of the MBB beam. Each cell in the heatmap
represents a unique combination of diversity measures values, with the color
indicating the objective value of the optimal solutions. The skewed
distribution in the heatmap, with a concentration of viable designs within
certain regions, is likely a function of the underlying physics and
constraints imposed on the beam’s design, where only specific configurations
can meet the performance criteria.

For statistical validation of the OIDD method’s efficacy, the Kruskal-Wallis
test with a $95\%$ confidence interval and the Bonferroni correction method
are employed. The comparison between standard SIMP and OIDD-enhanced SIMP
shows a statistically significant difference in performance. The average
objective value for standard SIMP over $30$ independent runs is $84.62$ with a
standard deviation of $0.81$, while for OIDD+SIMP, the average objective is
$82.30$ with a standard deviation of $0.33$. This statistically significant
difference highlights the enhanced efficiency and effectiveness of the OIDD
approach in topology optimization.

##  IV Application to Soft Robotics

The previous MBB beam example demonstrated the viability of the OIDD method to
a well-known, benchmark TO problem. Here, we demonstrate its application to
soft robotics. In the MBB and other classical TO problems, the design goal can
be readily transformed into a measurable cost function and constraints (e.g
minimizing stiffness subject to weight costraint). Soft robots, which need to
be flexible but strong and light but powerful, are challenging to formulate
into a tractable and meaningful, scalar cost function. In this section we
demonstrate the value of the OIDD method in two typical soft robotics
applications, soft grippers and pneumatic soft actuators. We demonstrate the
ability of our method to overcome local minima and generate diverse sets of
high-performing candidates for further detailed evaluation and integration.

###  IV-A Bending Soft Gripper

Figure 5: Design domain of soft gripper optimization: the finger should bend
inwards from the bottom left corner when a force is applied at the top right.
The void space is intended to maintain a clear path for the gripper to close
in a 2 fingered configuration

Figure 6: Optimization results of soft gripper problem: a) optimization
performance, showing evolution of the archive coverage and average objective
value of the designs produced in each generation, and the corresponding best
designs at selected points b) Optimal designs of the soft gripping bending
finger obtained by OIDD

We first optimize the design of a soft gripping finger (Figure 1), which bends
around an object when one side of the gripper is translated relative to the
other. The bending soft gripper problem is based on [13]. It applies a force
to one corner of the design domain, creating a compressive load on the soft
gripper, that forces it to bend inwards. This design is of interest because it
is an implementation of soft grasping with published optimized designs, and
because it has been shown that the resulting designs can be improved by
manually tuning the design domain [14]. The design domain of the problem is
presented in Figure 5. It is parameterised by $150\times 70$ elements. Here
the OIDD algorithm is run for 100 iterations, with a population size of 10.

####  IV-A1 Numerical Results

The experimental results, presented in Figure 6, illustrate the efficacy of
OIDD in generating a suite of optimized topologies tailored to the unique
demands of compliance in robotic grippers. Figure 6 traces the evolution of
archive coverage and average objective value (average of population in each
iteration) over iterations. The plot shows a steady increase in archive
coverage as the method progresses, highlighting the method’s thorough
exploration of the design domain. However, it plateaus at 55%, as resulting
regions are increasingly challenging to find or infeasible. Whilst not a
monotonic improvement, on average the optimizeer finds increasingly high
quality solutions as it progresses, improving from an average objective of
$11.5$ in the first iteration to $8.2$ in the last. The best compliance metric
seems to gradually plateau, indicating the convergence towards an optimal
solution in the middle of the process.

Figure 6 presents the resulting optimized topologies of the compliant gripper
mechanism. Notably, the designs reveal a variety of structural adaptations
driven by the constraints imposed by void regions, indicative of the
algorithm’s responsiveness to different initial conditions and constraints.

####  IV-A2 Experimental Grasp Testing

The topology optimization formulation creates high-performing bending soft
fingers, however what we are really interested in is soft grippers. Because of
the sharp discontinuity that occurs when grippers make contact with an object,
optimizing soft grippers in a realistic grasp environment is an intractable
problem. Hence, the problem is abstracted into a function of shape in free-
space (i.e maximising bending), as an approximation of the desired grasp
behavior. To evaluate the designs’ true grasp performance, we measure the peak
grasp strength (retention force) of a representative subset (9 fingers) of the
resulting designs. The designs are evaluated using an automated gantry
platform, which contains a set of stationary, rigid 3D printed test objects.
The platform uses a pneumatic cylinder to open and close candidate soft
fingers around the objects. A custom 2-bar mechanism is used in this work to
convert the horizontal motion to the desired
$45\text{\,}\mathrm{\SIUnitSymbolDegree}$ path. The mechanism and grasp
process are shown in Figure 7. In each test, the fingers are lowered around
the fixed object, closed to induce contact and then raised until contact is
broken, with grasp (vertical) force recorded throughout. 7 objects are used in
the experiment (Figure ) giving a range of shapes and sizes including smooth
spheres and jagged diamonds. The fingers are printed using Agilus 85 (Shore
85A) resin on a Stratasys J850 polyjet printer, and have dimensions $75\times
37.5\times 10$\mathrm{mm}$$

Figure 7: Experimental grasping procedure and platform. a) Testing cycle: for
each object/gripper pair the gripper is lowered around the object, closed,
then slowly raised until contact is lost. The forces are recorded throughout.
b) Set of grasped objects shown during grasping by F10 gripper. Top row (Left
to right): $40\text{\,}\mathrm{mm}$ cube, $25\text{\,}\mathrm{mm}$ cube,
$40\text{\,}\mathrm{mm}$ diameter tube, $25\text{\,}\mathrm{mm}$ diameter
tube. Bottom row: $40\text{\,}\mathrm{mm}$ diameter sphere,
$25\text{\,}\mathrm{mm}$ diameter sphere, $40\text{\,}\mathrm{mm}$ diamond

Figure 8: Experimental Grasping results. a) Sample of grippers for testing,
illustrating grasping modalities b) Complete set of grasped objects c) Heatmap
of maximized Grasp Force Measurements across Various Objects and Soft Gripper
Designs

The grasping mechanism of the grippers and the resulting heatmap of grasp
strength is shown in Figure 8. Several unique grasping mechanisms emerge from
the OIDD optimization: Designs F11, F15, F18, F19 and F21 all have long
linkages connecting the gripping region to the input force (2-bar mechanism),
these curve significantly when actuated, allowing the gripper to bend around
the object with a large contact area but relatively low force (due to strain
energy dissipating in the structure). F01 has a similar mechanism, but uses a
soft pad for gripping rather than a reinforced truss ’jaw’. Finally, F00, F10
and F17 have a shorter, stiffer linkage, which gives a larger force at the tip
for strong pinch-grasping. It can be seem that the pinch-grasps typically give
stonger grasps than those that envelop the object. However the result is not
universal, with also F18 giving strong grasps (and large displacement). Both
F10 and F18 are excellent designs for future use, with the choice depending on
the application and its requirements. Interestingly almost every gripper
tested outperforms F19, a design which is nearly identical to the published
benchmark [13], highlighting the efficacy of our method.

###  IV-B 3D Pneumatic Soft Finger

As a final example, we demonstrate the application of OIDD to a 3D pneumatic
soft finger, similar to widely used Pneunets [1]. Inflatable fingers have been
topology optimized in the past [6, 32, 11]; given the strongly nonlinear
behavior of these fingers, and their sensitivity to small design-changes, it
is likely that these are sub-optimal solutions, which can be improved using
OIDD.

To investigate this problem, we investigate the optimization of multi-jointed
soft fingers, using an approach similar to [12], The design domain of the
multi-jointed soft finger is shown in Figure 9(a), and is paramaterised by
$20\times 20\times 10$ $1\text{\,}\mathrm{mm}$ elements. The finger is
intended to bend when pressure is supplied to a central tube. As the tube
expands, the surrounding material should steer the output downwards.

The OIDD process was run for 10 generations with a population size of 10, and
resulted in 24 valid designs on the QD grid. A sample of the resulting designs
is presented in Figure 9(b), along with their displacement when simulated as a
3 jointed finger in Comsol Multiphysics. Unlike the original optimization, the
Comsol simulation captures both the geometric and loading non-linearities
present in soft fingers. Two interesting features are evident in the results:
firstly, the OIDD method has learned to ’checkerboard’ the designs. This
phenomenon, in which the optimizer converges on an alternating pattern of
filled and empty voxels, is common within TO. It exploits a flaw in the FEM
simulation to reduce material usage whilst retaining stiffness. Although not
typically a desirable outcome, finding such a solution demonstrates the
utility of OIDD in illuminating ’high-performing’ solutions even within large
search spaces. Secondly, we see the sensitivity of the performance to small
changes in design. When using a high fidelity solver, small changes in the
design not only vastly change the output, but can even invert the
displacement. Such complexity highlights the need for our multistage
optimization approach that overcomes nonlinearities and local minima to find
high-quality candidates for further evaluation. Capturing the full physics of
soft robotics in a fast simulator is an ongoing challenge for the field,
however even using coarse simulation and a highly abstracted problem, we
generate a number of high-quality results.

Figure 9: optimized pneumatic soft finger example. a) Design Domain of soft
finger joint b) Performance of representative group of results in high-
fidelity simulation. Left side: optimized single links generated by OIDD.
Right: Simulated inflation of corresponding soft fingers in 3 link
configuration of $0.5\text{\,}\mathrm{MPa}$ rubber with
$0.5\text{\,}\mathrm{MPa}$ inflation pressure

##  V Conclusion

In this research, we introduced the Optimized Initial Design Domain (OIDD)
methodology, which significantly advances topology optimization. This novel
hybrid design optimization method exploits global-search evolution and fine-
grained topology optimization to find diverse and structurally efficient
designs. By strategically adjusting the initial design domain, our methodology
facilitates the generation of solutions that are both structurally efficient
and considerate of material constraints.

In benchmark optimization problems and sim-to-sim and experimental evaluation
in soft robotic design problems, we show that OIDD finds higher-performing
designs than SIMP alone and that the resulting designs outperform benchmarks
in physical experimental evaluation. Physical testing is essential to our
research, as the substantial correlation between simulated optimization
results and experimental outcomes highlights the OIDD method’s robustness and
capacity to bridge the gap between simulation and reality reliably.

In future work, the OIDD methodology may be further refined to streamline the
design process for soft robots, leading to the innovation of soft robotic
design. The impact of the OIDD method on this field is anticipated to open up
unprecedented opportunities in how soft robots are imagined and utilized.

## Acknowledgement

This project has received funding from the European Union’s Horizon 2020
research and innovation programme under the Marie Skłodowska-Curie grant
agreement No 101034337.

## References

  * [1] F. Ilievski, A. D. Mazzeo, R. F. Shepherd, X. Chen, and G. M. Whitesides, “Soft robotics for chemists,” _Angewandte Chemie International Edition_ , vol. 50, no. 8, pp. 1890–1895, 2011. [Online]. Available: https://onlinelibrary.wiley.com/doi/abs/10.1002/anie.201006464
  * [2] F. Renda, M. Giorelli, M. Calisti, M. Cianchetti, and C. Laschi, “Dynamic model of a multibending soft robot arm driven by cables,” _IEEE Transactions on Robotics_ , vol. 30, no. 5, pp. 1109–1122, 2014.
  * [3] M. S. Xavier, C. D. Tawk, A. Zolfagharian, J. Pinskier, D. Howard, T. Young, J. Lai, S. M. Harrison, Y. K. Yong, M. Bodaghi, and A. J. Fleming, “Soft Pneumatic Actuators: A Review of Design, Fabrication, Modeling, Sensing, Control and Applications,” _IEEE Access_ , vol. 10, pp. 59 442–59 485, 2022.
  * [4] A. Miriyev and M. Kovač, “Skills for physical artificial intelligence,” _Nature Machine Intelligence_ , vol. 2, no. 11, pp. 658–660, 2020. [Online]. Available: http://dx.doi.org/10.1038/s42256-020-00258-y
  * [5] J. Pinskier and D. Howard, “Task-Mediated Design: A latent space for dexterous soft robotic design,” _IOP Conference Series: Materials Science and Engineering_ , vol. 1292, no. 1, p. 12009, oct 2023. [Online]. Available: https://dx.doi.org/10.1088/1757-899X/1292/1/012009
  * [6] ——, “From bioinspiration to computer generation: Developments in autonomous soft robot design,” _Advanced Intelligent Systems_ , vol. 4, no. 1, p. 2100086, 2022.
  * [7] N. Cheney, R. MacCurdy, J. Clune, and H. Lipson, “Unshackling evolution,” _ACM SIGEVOlution_ , vol. 7, no. 1, pp. 11–23, 2014.
  * [8] T.-H. Wang, P. Ma, A. Spielberg, Z. Xian, H. Zhang, J. B. Tenenbaum, D. Rus, and C. Gan, “SoftZoo: A Soft Robot Co-Design Benchmark For Locomotion in Diverse Environments,” in _ICLR 2023_ , 2023.
  * [9] P. Schegg, E. Ménager, E. Khairallah, D. Marchal, J. Dequidt, P. Preux, and C. Duriez, “SofaGym: An Open Platform for Reinforcement Learning Based on Soft Robot Simulations,” _Soft Robotics_ , vol. 10, no. 2, pp. 410–430, 2023.
  * [10] J. Pinskier, X. Wang, L. Liow, Y. Xie, P. Kumar, M. Langelaar, and D. Howard, “Diversity-based topology optimization of soft robotic grippers,” _Advanced Intelligent Systems_ , vol. n/a, no. n/a, p. 2300505. [Online]. Available: https://onlinelibrary.wiley.com/doi/abs/10.1002/aisy.202300505
  * [11] J. Pinskier, P. Kumar, M. Langelaar, and D. Howard, “Automated design of pneumatic soft grippers through design-dependent multi-material topology optimization,” in _2023 IEEE International Conference on Soft Robotics (RoboSoft)_ , 2023, pp. 1–7.
  * [12] H. Zhang, A. S. Kumar, F. Chen, J. Y. H. Fuh, and M. Y. Wang, “Topology optimized multimaterial soft fingers for applications on grippers, rehabilitation, and artificial hands,” _IEEE/ASME Transactions on Mechatronics_ , vol. 24, no. 1, pp. 120–131, 2019.
  * [13] C.-H. Liu, T.-L. Chen, C.-H. Chiu, M.-C. Hsu, Y. Chen, T.-Y. Pai, W.-G. Peng, and Y.-P. Chiang, “Optimal design of a soft robotic gripper for grasping unknown objects,” _Soft Robotics_ , vol. 5, no. 4, pp. 452–465, 2018. [Online]. Available: https://doi.org/10.1089/soro.2017.0121
  * [14] C.-H. Liu, S.-Y. Yang, and Y.-C. Shih, “Optimal design of a highly self-adaptive gripper with multi-phalange compliant fingers for grasping irregularly shaped objects,” _IEEE Robotics and Automation Letters_ , vol. 8, no. 11, pp. 7026–7033, 2023.
  * [15] P. Ma, T. Du, J. Z. Zhang, K. Wu, A. Spielberg, R. K. Katzschmann, and W. Matusik, “DiffAqua: A Differentiable Computational Design Pipeline for Soft Underwater Swimmers with Shape Interpolation,” _ACM Transactions on Graphics_ , vol. 40, no. 4, 2021.
  * [16] Y. Yao, Y. Chen, L. He, and P. Maiolino, “Design Optimization for Bellow Soft Pneumatic Actuators in Shape-Matching,” _2023 IEEE International Conference on Soft Robotics, RoboSoft 2023_ , pp. 1–7, 2023.
  * [17] M. P. Bendsøe and N. Kikuchi, “Generating optimal topologies in structural design using a homogenization method,” _Computer methods in applied mechanics and engineering_ , vol. 71, no. 2, pp. 197–224, 1988.
  * [18] O. Sigmund and K. Maute, “Topology optimization approaches: A comparative review,” _Structural and multidisciplinary optimization_ , vol. 48, no. 6, pp. 1031–1055, 2013.
  * [19] H. A. Eschenauer and N. Olhoff, “Topology optimization of continuum structures: a review,” _Appl. Mech. Rev._ , vol. 54, no. 4, pp. 331–390, 2001.
  * [20] A. Prathyusha and G. R. Babu, “A review on additive manufacturing and topology optimization process for weight reduction studies in various industrial applications,” _Materials Today: Proceedings_ , vol. 62, pp. 109–117, 2022.
  * [21] R. Picelli, S. Ranjbarzadeh, R. Sivapuram, R. S. Gioria, and E. C. Silva, “Topology optimization of binary structures under design-dependent fluid-structure interaction loads,” _Structural and Multidisciplinary Optimization_ , vol. 62, no. 4, pp. 2101–2116, 2020.
  * [22] T. J. Ikonen, G. Marck, A. Sóbester, and A. J. Keane, “Topology optimization of conductive heat transfer problems using parametric L-systems,” _Structural and Multidisciplinary Optimization_ , vol. 58, no. 5, pp. 1899–1916, 2018.
  * [23] T. Lowe and J. Pinskier, “Tree Reconstruction Using Topology Optimisation,” _Remote Sensing_ , vol. 15, no. 1, jan 2023.
  * [24] J.-H. Zhu, W.-H. Zhang, and L. Xia, “Topology optimization in aircraft and aerospace structures design,” _Archives of computational methods in engineering_ , vol. 23, pp. 595–622, 2016.
  * [25] S. Mukherjee, D. Lu, B. Raghavan, P. Breitkopf, S. Dutta, M. Xiao, and W. Zhang, “Accelerating large-scale topology optimization: State-of-the-art and challenges,” _Archives of Computational Methods in Engineering_ , pp. 1–23, 2021.
  * [26] Z. Jihong, Z. Han, W. Chuang, Z. Lu, Y. Shangqin, and W. Zhang, “A review of topology optimization for additive manufacturing: Status and challenges,” _Chinese Journal of Aeronautics_ , vol. 34, no. 1, pp. 91–110, 2021.
  * [27] J. Mouret and J. Clune, “Illuminating search spaces by mapping elites,” _CoRR_ , vol. abs/1504.04909, 2015.
  * [28] M. P. Bendsøe, “Optimal shape design as a material distribution problem,” _Structural optimization_ , vol. 1, pp. 193–202, 1989.
  * [29] M. P. Bendsøe and O. Sigmund, _Topology optimization: theory, methods, and applications_. Springer, 2003.
  * [30] D. Guirguis, N. Aulig, R. Picelli, B. Zhu, Y. Zhou, W. Vicente, F. Iorio, M. Olhofer, W. Matusiks, C. A. Coello Coello, and K. Saitou, “Evolutionary Black-Box Topology Optimization: Challenges and Promises,” _IEEE Transactions on Evolutionary Computation_ , vol. 24, no. 4, pp. 613–633, 2020.
  * [31] K. Gao, D. M. Doc, S. Chu, G. Wu, H. A. Kim, and C. A. Featherston, “Robust topology optimization of structures under uncertain propagation of imprecise stochastic-based uncertain field,” _CoRR_ , vol. abs/2201.11513, 2022.
  * [32] F. Chen and M. Y. Wang, “Design optimization of soft robots: A review of the state of the art,” _IEEE Robotics & Automation Magazine_, vol. 27, no. 4, pp. 27–43, 2020.