11institutetext: School of Computer Science, Shanghai Key Laboratory of
Intelligent Information Processing, Fudan University, China
22institutetext: Academy for Engineering and Technology, Fudan University,
China
22email<EMAIL_ADDRESS>33institutetext: Children’s Hospital of Fudan University, National Children’s
Medical Center, Shanghai, China
33email<EMAIL_ADDRESS>

# Boosting COVID-19 Severity Detection with Infection-aware Contrastive Mixup
Classification

Junlin Hou 11 Jilan Xu 11 Nan Zhang 22 Yuejie Zhang 11
Xiaobo Zhang 33 Rui Feng 112233

###### Abstract

This paper presents our solution for the 2nd COVID-19 Severity Detection
Competition. This task aims to distinguish the Mild, Moderate, Severe, and
Critical grades in COVID-19 chest CT images. In our approach, we devise a
novel infection-aware 3D Contrastive Mixup Classification network for severity
grading. Specifically, we train two segmentation networks to first extract the
lung region and then the inner lesion region. The lesion segmentation mask
serves as complementary information for the original CT slices. To relieve the
issue of imbalanced data distribution, we further improve the advanced
Contrastive Mixup Classification network by weighted cross-entropy loss. On
the COVID-19 severity detection leaderboard, our approach won the first place
with a Macro F1 Score of 51.76%. It significantly outperforms the baseline
method by over 11.46%.

###### Keywords:

COVID-19 severity detection, chest CT images, infection-aware contrastive
mixup classification

##  1 Introduction

The Coronavirus Disease 2019 SARS-CoV-2 (COVID-19) is an ongoing pandemic,
which spreads fast worldwide since the end of 2019 [35]. To date (July, 2022),
the number of infected people is still increasing rapidly. In clinical
practice, thoracic computed tomography (CT) has been recognized to be a
reliable complement to RT-PCR assay [7]. From CT scans, a detailed overview of
COVID-19 radiological patterns can be clearly presented, including ground
glass opacities, rounded opacities, enlarged intra-infiltrate vessels, and
later more consolidations [5]. In fighting against COVID-19, two important
clinical tasks can be conducted in chest CT scans, namely identification and
severity assessment. Early identification is vital to the slowdown of viral
transmission, whereas severity assessment is significant for clinical
treatment planning.

Recently, deep learning approaches have achieved excellent performance in
discriminating COVID-19 from non-pneumonia or other types of pneumonia [3, 16,
12, 34]. For instance, Chen et al. [3] detected the suspicious lesions for
COVID-19 diagnosis using a UNet++ model. Javaheri et al.[16] designed the
CovidCTNet to differentiate COVID-19 from other lung diseases. Xu et al. [36]
developed a deep learning system with a location-attention mechanism to
categorize COVID-19, Influenza-A viral pneumonia, and healthy subjects.
Despite the great success in COVID-19 identification, the automatic assessment
of COVID-19 severity in CT images is still a very challenging task.

Figure 1: Examples of COVID-19 CT images with different severity levels from
the COV19-CT-DB dataset. The red arrows indicate the locations of the main
lesion regions. In the mild case, the lesions only account for small regions
in the lung. In the critical case, the lesions spread over the periphery of
the lung and are indistinguishable from the outer tissue.

In this paper, we present a deep-learning based solution for the 2nd COV19D
Competition of the Workshop “AI-enabled Medical Image Analysis – Digital
Pathology & Radiology/COVID19 (AIMIA)”, which occurs in conjunction with the
European Conference on Computer Vision (ECCV) 2022. This paper mainly focuses
on the COVID-19 Severity Detection Challenge. As illustrated in Fig. 1, the
goal of this task is to categorize the severity of COVID-19 into four stages,
including Mild, Moderate, Severe, and Critical. Each severity category is
determined by the presence of ground glass opacities and the pulmonary
parenchymal involvement. This task is of great significance in real-world
scenarios. Especially for patients with severe or critical grades, extensive
care and attention are necessary in the clinical practice. The main challenge
of the severity detection task is three-fold: (1) Compared with the binary
classification task (COVID vs. non-COVID), the severity detection requires
more fine-grained lesion features to determine the final grade. (2) The
accurate annotation of grades requires domain knowledge and it is labor-
expensive. The limited amount of data makes the task difficult for deep-
learning based methods. (3) In the original CT volume, it is hard to identify
the lesions of interest from substantial non-lesion voxels.

In order to address the aforementioned challenges, we propose a novel
infection-aware Contrastive Mixup Classification network. This framework is
constructed in a segmentation-classification manner. In particular, we
initially extract the lung regions and segment the inner lesion regions using
two individual segmentation networks. The obtained lesion segmentation mask
serves as complementary information for fine-grained classification. The
location, as well as the number of lesions, are both crucial in determining
the severity grade. After lesion segmentation, an advanced 3D Contrastive
Mixup Classification network (CMC-COV19D) in the previous work [11] is
applied. It won the first price in the ICCV 2021 COVID-19 Diagnosis
Competition of AI-enabled Medical Image Analysis Workshop [21]. The CMC-COV19D
framework introduces contrastive representation learning to discover more
discriminative representations of COVID-19 cases. It includes a joint training
loss that combines the classification loss, mixup loss, and contrastive loss.
We further improve the classification loss with weight balancing strategy to
enhance the attention on minor classes (i.e. moderate and critical).
Experimental results on the COVID-19 severity detection challenge shows that
our approach achieves a Marco F1 Score of 51.76%, surpassing all the
participants.

The remainder of this paper is organized as follows. In Section 2, we
summarize related works on segmentation of lung regions and lesions, and
COVID-19 severity assessment. Section 3 introduces our proposed infection-
aware Contrastive Mixup Classification framework in detail. Section 4
describes the two datasets. Extensive experiments are conducted in Section 5
to evaluate the performance of our network. The conclusion is given in Section
6.

##  2 Related Work

###  2.1 Segmentation of lung regions and lesions

Segmentation is an important step in AI-based COVID-19 image processing and
analysis [32]. It extracts the regions of interest (ROI), including lung,
lobes, and infected regions in CT scans for further diagnosis. A large number
of researches [39, 1, 14] utilized U-Net [30] to segment lung fields and
lesions. For example, Qi et al. [29] used U-Net to delineate the lesions in
the lung and extract radiometric features of COVID-19 patients with the
initial seeds given by a radiologist for predicting hospital stay. Moreover,
Jin et al. [17] proposed an COVID-19 screening system, in which the lung
regions are first detected by UNet++ [41]. The Inf-Net [6] aimed to segment
the COVID-19 infection regions for quantifying and evaluating the disease
progression. Shan et al. [31] proposed a VB-Net for segmentation of lung, lung
lobes and lung infection, which provided accurate quantification data for
medical studies. Motivated by the above approaches, we train segmentation
networks to extract infection-aware clues for severity grading.

###  2.2 COVID-19 severity assessment

The study of COVID-19 severity assessment plays an essential role in clinical
treatment planning. Tang et al. [33] established a random forest model based
on quantitative features for COVID-19 severity assessment (non-severe or
severe). Chaganti et al. [2] presented a method that automatically segmented
and quantified abnormal CT patterns commonly presented in COVID-19. Huang et
al. [14] assessed a quantitative CT image parameter defined as the percentage
of lung opacification (QCT-PLO) and classified the severity into four classes
(mild, moderate, severe and critical). Pu et al. [28] developed an automated
system to detect COVID-19, quantify the extent of disease, and assess the
progression of the disease. Feng et al. [8] designed a novel Lesion Encoder
framework to detect lesions in chest CT scans and encode lesion features for
automatic severity assessment. He et al. [10] proposed a synergistic learning
framework for automated severity assessment of COVID-19 in 3D CT images. In
our work, we first identify the infected regions, and then utilize segmented
lesion masks as complementary information for the original CT slices.

##  3 Methodology

Figure 2: Overview of our infection-aware CMC network, comprised of the
infection segmentation stage and the COVID-19 severity classification stage.

Fig. 2 illustrates the overview of our infection-aware Contrastive Mixup
Classification (CMC) framework in a combined segmentation-classification
manner. The framework is divided into two stages. The first stage is the
infection segmentation that generates lesion masks of a CT volume slice by
slice (Fig. 2(a)). The lesion masks of an input CT volume help the
classification model focus on important regions and better classify COVID-19
severity. The second stage is to take a CT volume and its 3D lesion masks as
the input and output the COVID-19 severity prediction (Fig. 2(b)). We adopt
our previous work, the CMC network, as the classification model, which has
shown excellent COVID-19 detection performance.

In the following subsections, we will elaborate the infection segmentation
procedure in detail, and then describe the CMC network for COVID-19 severity
classification.

###  3.1 Infection segmentation

As shown in Fig. 2(a), given an input CT volume
$x=\\{\mathbbm{x}_{t}\\}_{t=1}^{T}$, we perform slice-level infection
segmentation by the following two steps.

Step 1: lung region extraction. For each slice
$\mathbbm{x}_{t}\in\mathbb{R}^{H\times W}$ in the CT volume, we train a lung
segmentation model $M_{1}$ to predict the lung region mask
$M_{1}(\mathbbm{x}_{t};\theta)\in\mathbb{R}^{H\times W}$, where $\theta$ are
the network parameters. Extracting the lung regions of the CT volume removes
the redundant background region as the background varies across different
patients and screening devices. Moreover, lung extraction eases the subsequent
lesion segmentation model as the lesions usually appear at the periphery of
the lung. We empirically show that with simple training recipes, the lung
segmentation model yields high segmentation accuracy. After training, the
segmented CT slice $\hat{\mathbbm{x}_{t}}\in\mathbb{R}^{H\times W}$ is
obtained by multiplying the lung segmentation mask with the original CT slice:

$\hat{\mathbbm{x}_{t}}=M_{1}(\mathbbm{x}_{t};\theta)\odot\mathbbm{x}_{t}$ (1)

where $\odot$ denotes the Hadamard product.

Step 2: lesion segmentation. Given each CT slice $\hat{\mathbbm{x}_{t}}$, we
obtain the infection mask $I_{t}=M_{2}(\hat{\mathbbm{x}_{t}};\phi)$ by
training another segmentation model $M_{2}$ parameterized by $\phi$. We refer
the infection mask to the binary map where 1 indicates the pixel with COVID-19
infection and 0 represents the non-infected region. Notably, $M_{2}$ is
trained on the CT slices with the segmented lung region, and thus the noisy
background and artifacts are avoided. This reduces the model’s potential false
positive predictions over the background. The input to the classification
model is defined as the concatenation of the infection mask and the original
CT image:

$\bar{x}_{t}=[I_{t},\mathbbm{x}_{t}]=[M_{2}(M_{1}(\mathbbm{x}_{t};\theta)\odot\mathbbm{x}_{t};\phi),\mathbbm{x}_{t}].$
(2)

####  3.1.1 Segmentation models.

We employ several state-of-the-art models for lung and infection segmentation,
including DeepLabV3 [4], FPN [25], U-Net [30], and U-Net++ [41]. Different
backbones such as ResNet [9] and DenseNet [13] with ImageNet pre-trained
weights are adopted as the encoder layers of segmentation models. All
segmentation models are trained and validated on the COVID19-CT-Seg dataset
[26]. The models that perform best on the validation set are then transferred
to the COV19-CT-DB dataset to generate lung and infection masks for each CT
volume.

####  3.1.2 Loss function.

We use the Dice loss [27] as the objective function to optimize lung and
infection segmentation networks. For each category $c$, the Dice loss is
defined as follows:

$\mathcal{L}_{Dice}^{c}=1-2\times\frac{\sum_{n=1}^{N}y_{n}^{c}\hat{y}_{n}^{c}}{\sum_{n=1}^{N}(y_{n}^{c}+\hat{y}_{n}^{c})+\epsilon},$
(3)

where $y_{n}$ and $\hat{y_{n}}$ denote the label and the prediction of each
pixel $n$, respectively. $\epsilon$ is a small constant to avoid zero
division.

###  3.2 Infection-aware COVID-19 severity classification

As illustrated in Fig. 2(b), the CT volume $X\in\mathbb{R}^{T\times H\times
W}$ and its infection mask $I\in\mathbb{R}^{T\times H\times W}$ are
concatenated per channel to produce $\bar{X}\in\mathbb{R}^{2\times T\times
H\times W}$ as the input. We adopt the Contrastive Mixup Classification (CMC)
network for COVID-19 severity detection.

The CMC network is mainly comprised of the following four components.

  * •

A data augmentation module $A(\cdot)$. This module transforms an input CT
volume $x$ into an augmented volume $\tilde{x}$ randomly. In our work, two
augmented volumes are generated from each input CT scan.

  * •

A feature extractor $E(\cdot)$. It maps the augmented CT sample $\tilde{x}$ to
a representation vector $r=E(\tilde{x})\in\mathbb{R}^{d_{e}}$ in
$d_{e}$-dimensional latent space. We adopt an inflated 3D ResNeSt-50 to
extract features of each CT scan.

  * •

A small projection network $P(\cdot)$. This network is used to map the high-
dimension vector $r$ to a relative low-dimension vector
$z=P(r)\in\mathbb{R}^{d_{p}}$. A MLP (multi-layer perception) can be employed
as the projection network.

  * •

A classifier network $C(\cdot)$. It classifies the vector
$r\in\mathbb{R}^{d_{e}}$ to the COVID-19 severity prediction using fully
connected layers.

We train the CMC network using a well-designed adaptive loss, composed of
contrastive loss, mixup loss, and weighted classification loss.

####  3.2.1 Contrastive loss.

The CMC network employs the contrastive learning as an auxiliary task to learn
more discriminative representations of each COVID-19 severity categories.
Formally, given a minibatch of $2N$ augmented CT volumes and the labels
$\\{(\tilde{x}_{i},\tilde{y}_{i})\\}_{i=1,\dots,2N}$, we define the positives
as any augmented CT samples from the same category, while those from different
classes are considered as negative pairs [19]. Let $i\in\\{1,\dots,2N\\}$ be
the index of an arbitrary augmented sample, the contrastive loss function is
defined as:

$\mathcal{L}_{con}^{i}=\frac{-1}{2N_{\tilde{y}_{i}}-1}\sum_{j=1}^{2N}\mathbbm{1}_{i\neq
j}\cdot\mathbbm{1}_{\tilde{y}_{i}=\tilde{y}_{j}}\cdot\log\frac{\exp(z_{i}^{T}\cdot
z_{j}/\tau)}{\sum_{k=1}^{2N}\mathbbm{1}_{i\neq k}\cdot\exp(z_{i}^{T}\cdot
z_{k}/\tau)},$ (4)

where $N_{\tilde{y}_{i}}$ is the number of samples in a minibatch that share
the same label $\tilde{y}_{i}$, $\mathbbm{1}\in\\{0,1\\}$ is an indicator
function, and $\tau$ denotes a temperature parameter.

####  3.2.2 Mixup loss.

To boost the generalization ability of the model, the mixup strategy [38] is
also adopted in the training procedure. For each augmented CT volume
$\tilde{x}_{i}$, we generate a mixed sample $\tilde{x}^{mix}_{i}$ and its
mixed label $\tilde{y}^{mix}_{i}$ as:

$\tilde{x}^{mix}_{i}=\lambda\tilde{x}_{i}+(1-\lambda)\tilde{x}_{p},~{}\tilde{y}^{mix}_{i}=\lambda\tilde{y}_{i}+(1-\lambda)\tilde{y}_{p},$
(5)

where $p$ is a random index. The mixup loss is defined as the cross-entropy
loss of mixed samples:

$\mathcal{L}^{i}_{mix}=\mathrm{CrossEntropy}(\tilde{x}^{mix}_{i},\tilde{y}^{mix}_{i}).$
(6)

####  3.2.3 Weighted classification loss.

In addition to the mixup loss on the mixed samples, we also employ the cross-
entropy loss on raw samples for severity detection. It is observed from Table
1 that there are imbalance data distributions on the COV19-CT-DB dataset. To
alleviate this problem, we adopt the weighted cross entropy loss. Each sample
is scaled by the weight proportional to the inverse of the percentage of the
sample for each class $c$ in the training set, denoted as
$\alpha=[\alpha^{1},\dots,\alpha^{c}]$. Therefore, the weighted classification
loss is defined as:

$\mathcal{L}_{clf}^{i}=-\alpha\tilde{y}_{i}^{T}\log\hat{y}_{i},$ (7)

where $\tilde{y}_{i}$ denotes the one-hot vector of ground truth label, and
$\hat{y}_{i}$ is predicted probability of the sample $x_{i}$ $(i=1,\dots,2N)$.

####  3.2.4 Adaptive joint loss.

Finally, we merge the contrastive loss, mixup loss, and weighted
classification loss into a combined objective function with learnable weights
[18]:

$\mathcal{L}=\frac{1}{2N\sigma_{1}^{2}}\sum_{i=1}^{2N}\mathcal{L}^{i}_{con}+\frac{1}{2N\sigma_{2}^{2}}\sum_{i=1}^{2N}(\mathcal{L}^{i}_{mix}+\mathcal{L}^{i}_{clf})+\log\sigma_{1}+\log\sigma_{2},$
(8)

where $\sigma_{1},\sigma_{2}$ are utilized to learn the adaptive weights of
the three losses.

##  4 Dataset

###  4.1 COV19-CT-DB Database

The COV19-CT-Database (COV19-CT-DB) [20] provides chest CT scans that are
marked for the existence of COVID-19. It contains about 1,650 COVID and 6,100
non-COVID chest CT scan series from over 1,150 patients and 2,600 subjects.
The CT scans were collected in the period from September 1, 2020 to November
30, 2021. Each CT scan was annotated by 4 experienced medical experts, which
showed a high degree of agreement (around 98%). The number of slices of each
3D CT scan ranging from 50 to 700. The variation in the number of slices is
due to the context of CT scanning.

In particular, the COV19-CT-DB further provides the severity annotations on a
small portion of the COVID-19 cases. The descriptions and number of samples of
the four severity categories are shown in detail in Table 1. The severity
categories include Mild, Moderate, Severe, and Critical, in the range from 1
to 4. The training set contains, in total, 258 3D CT scans (85 Mild cases, 62
Moderate cases, 85 Severe cases, and 26 Critical cases). The validation set
consists of 61 3D CT scans (22 Mild cases, 10 Moderate cases, 22 Severe cases,
and 5 Critical cases). The testing set includes 265 scans and the labels are
not available during the challenge.

Table 1: The number of samples and descriptions of four severity categories on
the COV19-CT-DB database.

Category | Severity | Training | Validation | Description
---|---|---|---|---
1 | Mild | 85 | 22 |  Few or no ground glass opacities. Pulmonary parenchymal involvement $\leq$ 25% or absence
2 | Moderate | 62 | 10 |  Ground glass opacities. Pulmonary parenchymal involvement 25-50%
3 | Severe | 85 | 22 |  Ground glass opacities. Pulmonary parenchymal involvement 50-75%
4 | Critical | 26 | 5 |  Ground glass opacities. Pulmonary parenchymal involvement $\geq$ 75%

###  4.2 COVID-19-CT-Seg dataset

The COVID-19-CT-Seg dataset [26] provides 20 public COVID-19 CT scans from the
Coronacases Initiative and Radiopaedia. All the cases contain COVID-19
infections with the proportion ranging from 0.01% to 59%. The left lung, right
lung, and infections (i.e., all visibly affected regions of the lungs) were
annotated, refined and verified by junior annotators, radiologists, and a
senior radiologist, progressively. The whole lung mask includes both normal
and pathological regions. In total, there are more than 300 infections with
over 1,800 slices.

##  5 Experimental Results

###  5.1 Implementation details

####  5.1.1 Lung and infection segmentation.

We employ multiple segmentation models [4, 25, 30, 41] with the ImageNet pre-
trained encoder [9, 13] to perform lung and infection segmentation on 2D CT
slices. The input resolution is resized to $256\times 256$. Random crop,
random horizontal flip, vertical flip, and random rotation are applied as
forms of data augmentation to reduce overfitting. The initial learning rate is
set to 1e-3 and then divided by 10 at $30\%$ and $80\%$ of the total number of
training epochs. The segmentation models are trained for 100 epochs with the
Stochastic Gradient Descent (SGD) optimizer. The batch size is 8. We select
two models that perform best on the validation set as the final lung and
infection segmentation models.

####  5.1.2 COVID-19 severity detection.

We employ an inflated 3D ResNest50 as the backbone for COVID-19 severity
detection. Each CT volume is resampled from $T\times 512\times 512$ to
$64\times 256\times 256$, where $T$ denotes the number of slices. Data
augmentations include random resized crop and random contrast changes. Other
augmentations such as flip and rotation are also tried, but no significant
improvement is yielded. We optimize the networks using the Adam algorithm with
a weight decay of 1e-5. The initial learning rate is set to 1e-4 and then
divided by 10 at $30\%$ and $80\%$ of the total number of training epochs. Our
models are trained for 100 epochs with the batch size 4. Our methods are
implemented in PyTorch and run on four NVIDIA Tesla V100 GPUs.

###  5.2 Evaluation metrics

For the lung and infection segmentation, we report mean Intersection over
Union (mIoU), and IoU of the foreground. For the COVID-19 severity detection,
we adopt the Macro F1 Score as the evaluation metric for overall comparison.
The score is defined as the unweighted average of the class-wise/label-wise F1
Scores. We also show the F1 Score for each severity category. Furthermore, we
utilize a model explanation technique, namely Class Activation Mapping (CAM)
[40], to conduct visual interpretability analysis [15]. The CAM can identify
the important regions that are most relevant to the predictions.

###  5.3 Results of lung and infection segmentation

Table 2: The segmentation results on the COVID-19-CT-Seg dataset (unit: %). Model | Encoder | Lung segmentation | Infection segmentation
---|---|---|---
mIoU | lung IoU | mIoU | infection IoU
FPN | ResNet-34 | 96.81 | 94.50 | 77.88 | 56.46
Deeplabv3 | ResNet-34 | 95.01 | 91.44 | 78.67 | 58.10
UNet | ResNet-34 | 96.96 | 94.75 | 79.67 | 59.93
UNet | DenseNet-121 | 97.54 | 95.75 | 77.52 | 55.73
UNet++ | ResNet-34 | 97.37 | 95.46 | 80.36 | 61.36
UNet++ | DenseNet-121 | 97.56 | 95.79 | 80.04 | 60.71
Figure 3: An illustration of the segmentation results on the COV19-CT-DB
dataset.

The lung and infection segmentation results on the COVID-19-CT-Seg dataset are
shown in Table 2. Among all segmentation models, the UNet++ models achieve
better performance on both lung and infection segmentation tasks. The UNet++
with ResNet-34 encoder reaches the highest infection segmentation results with
80.36% mean IoU and 61.36% IoU for infections. The Unet++ with DenseNet-121
encoder performs best on the lung segmentation task. It obtains 97.56% mIoU
and 95.79% IoU for lung regions.

After the segmentation on the COVID-19-CT-Seg dataset, we utilize the two
trained models to inference the lung and infection regions on the CT scans
from COV19-CT-DB dataset. As the COV19-CT-DB dataset lacks the segmentation
annotations, we illustrate several representative visual results in Fig. 3.
For each severity category, the segmentation results of two slices from a 3D
CT scan are presented in the first four rows. As can be apparently seen, our
lung segmentation model can predict the whole lung regions accurately, which
shows its strong generalization ability. In addition, our infection
segmentation model can inference most infection areas from mild to critical
precisely.

We also analyze the failure cases, which are shown in the last two rows of
Fig. 3. According to our observation, the incomplete segmentation results
mainly occur in the following two cases. (1) In the early stage, the low-
density infections with fuzzy boundaries are difficult to identify. (2) The
appearance of consolidation may be similar to bones, which is also hard for
the model to recognize. Therefore, it may be inaccurate to make severity
predictions by directly calculating the volume ratio between the segmented
infections and lung regions. Therefore, we make full use of the segmentation
results to provide auxiliary information for the original CT scan inputs.

###  5.4 Results on the COVID-19 severity detection challenge

Table 3: The comparison results on the validation set of COVID-19 severity
detection challenge (unit: %).

ID | Methods | Con | Mixup | Macro F1 | F1
---|---|---|---|---|---
Mild | Moderate | Severe | Critical
1 | CNN-RNN [20] | - | - | 63.00 | - | - | - | -
2 | ResNeSt-50 [37] |  |  | 65.47 | 82.05 | 61.54 | 68.29 | 50.00
3 | ResNeSt-50 [37] |  | $\checkmark$ | 67.30 | 71.79 | 75.00 | 60.87 | 61.54
4 | CMC [11] | $\checkmark$ | $\checkmark$ | 71.86 | 79.07 | 78.26 | 75.56 | 54.55
5 | Lung-aware CMC | $\checkmark$ | $\checkmark$ | 70.62 | 78.05 | 66.67 | 71.11 | 66.67
6 | Infection-aware CMC | $\checkmark$ | $\checkmark$ | 77.03 | 83.72 | 66.67 | 74.42 | 83.33

Table 4: The leaderboard on the COVID19 Severity Detection Challenge (unit: %). Rank | Teams | Macro F1 | F1
---|---|---|---
Mild | Moderate | Severe | Critical
1 | FDVTS (Ours) | 51.76 | 58.97 | 44.53 | 58.89 | 44.64
2 | Code 1055 | 51.48 | 61.14 | 34.06 | 61.91 | 48.83
3 | CNR-IEMN | 47.11 | 55.67 | 37.88 | 55.46 | 39.46
4 | MDAP | 46.00 | 58.14 | 42.29 | 43.78 | 40.03
5 | etro | 44.92 | 57.69 | 29.57 | 62.85 | 29.57
6 | Jovision-Deepcam | 41.49 | 62.20 | 40.24 | 38.59 | 24.93
7 | ResNet50-GRU [20] | 40.30 | 61.10 | 39.47 | 37.07 | 23.54

####  5.4.1 Results on the validation set.

Table 3 shows the results of the baseline model and our methods on the
validation set of COVID-19 severity detection challenge. The baseline model is
based on a CNN-RNN architecture [20] that performs 3D CT scan analysis,
following the work [22, 24, 23] on developing deep neural architectures for
predicting COVID-19. It achieves 63.00% Marco F1 Score. In our experiments, a
simple 3D ResNeSt-50 model reaches better performance with 65.47% Macro F1
compared to the baseline. The Mixup technique helps to boost the detection
result by $\sim$2% improvement. When we adopt the CMC architecture with
Contrastive learning and Mixup, it further improves the detection performance
with 71.86% Macro F1 Score. In addition, the CMC model significantly increases
the F1 Scores on Moderate and Severe categories, which are easily confused.
The experimental results demonstrate the effectiveness of Contrastive learning
and Mixup modules.

Furthermore, we report the performance of our proposed lung-aware CMC and
infection-aware CMC in the 5th and 6th rows in Table 3. It can be seen that
the combination of original images and lung region masks does not
significantly benefit the severity detection results. However, the infection-
aware CMC which takes the concatenation of original images and infection
region masks as the input significantly boost the performance to 77.03% Macro
F1 Score. Additionally, this model shows the best or comparable results on the
Mild (83.72%), Moderate (66.67%), Severe (74.42%), and Critical (83.33%)
categories.

Figure 4: The visualization of model outputs on CT scan of different severity
categories.

####  5.4.2 The leaderboard on the COVID-19 severity detection challenge.

In the COVID-19 severity detection challenge, we ensemble the CMC and
infection-aware CMC models as our final model. The comparison results on the
testing dataset are shown in Table 4. Our model ranks first in the challenge,
surpassing the second team by 0.28% on the Macro F1 Score. Compared to other
methods, our model shows relatively stable performance on each category,
indicating that our model has learned the discriminative representations of
different severity categories.

####  5.4.3 Visualization results.

We exploit the Class Activation Mapping (CAM) [40] to show the attention maps
for COVID-19 severity detection. Four CT scan series of different categories
are selected from the validation set, and the visualization results are shown
in Figure 4. For each group, the first row shows a series of CT slices, and
the second and third rows present the CAMs from CMC and infection-aware CMC
models, respectively.

Though the CMC model makes correct predictions, the attention maps just
roughly indicate the suspicious area (see subfig. (a)), and it even over-
focuses on irrelevant regions (e.g. bones) rather than the infections (see
subfig. (b)(c)(d)). In contrast, our proposed infection-aware CMC model
greatly improves the localization ability of suspicious lesions. Most
infection areas can be located precisely, and the irrelevant regions are
largely excluded. These highlighted areas in the attention maps demonstrate
the reliability and interpretability of the detection results from our model.

##  6 Conclusions

In this paper, we present our solution for COVID-19 severity detection
challenge in the 2nd COV19D Competition. Our proposed infection-aware 3D CMC
network is composed of two stages, namely infection segmentation and COVID-19
severity classification. In the first stage, two segmentation networks are
trained to extract lung fields as well as segment infected regions. The
infection masks further provide auxiliary information for the original CT
scans. In the second stage, the CMC network takes the CT scan and its lesion
masks as the input and outputs the severity prediction. The experimental
results on the competition leaderboard demonstrate the effectiveness of our
infection-aware CMC network.

## Acknowledgement

This work was supported by the Scientific & Technological Innovation 2030 -
“New Generation AI” Key Project (No. 2021ZD0114001; No. 2021ZD0114000), and
the Science and Technology Commission of Shanghai Municipality (No.
21511104502; No. 21511100500; No. 20DZ1100205). Yuejie Zhang, Xiaobo Zhang,
and Rui Feng are corresponding authors.

## References

  * [1] Cao, Y., Xu, Z., Feng, J., Jin, C., Han, X., Wu, H., et al.: Longitudinal assessment of covid-19 using a deep learning-based quantitative ct pipeline: Illustration of two cases. Radiology: Cardiothoracic Imaging 2(2), e200082 (2020)
  * [2] Chaganti, S., Grenier, P., Balachandran, A., Chabin, G., Cohen, S., Flohr, T., Georgescu, B., Grbic, S., Liu, S., Mellot, F., et al.: Automated quantification of ct patterns associated with covid-19 from chest ct. Radiology: Artificial Intelligence 2(4) (2020)
  * [3] Chen, J., Wu, L., Zhang, J., Zhang, L., Gong, D., Zhao, Y., et al.: Deep learning-based model for detecting 2019 novel coronavirus pneumonia on high-resolution computed tomography. Scientific Reports 10(1), 1–11 (2020)
  * [4] Chen, L.C., Zhu, Y., Papandreou, G., Schroff, F., Adam, H.: Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Proceedings of the European conference on computer vision (ECCV). pp. 801–818 (2018)
  * [5] Chung, M., Bernheim, A., Mei, X., Zhang, N., Huang, M., Zeng, X., Cui, J., Xu, W., Yang, Y., Fayad, Z.A., et al.: Ct imaging features of 2019 novel coronavirus (2019-ncov). Radiology (2020)
  * [6] Fan, D.P., Zhou, T., Ji, G.P., Zhou, Y., Chen, G., Fu, H., Shen, J., Shao, L.: Inf-net: Automatic covid-19 lung infection segmentation from ct images. IEEE Transactions on Medical Imaging 39(8), 2626–2637 (2020)
  * [7] Fang, Y., Zhang, H., Xie, J., Lin, M., Ying, L., Pang, P., Ji, W.: Sensitivity of chest ct for covid-19: comparison to rt-pcr. Radiology (2020)
  * [8] Feng, Y.Z., Liu, S., Cheng, Z.Y., Quiroz, J.C., Rezazadegan, D., Chen, P.K., Lin, Q.T., Qian, L., Liu, X.F., Berkovsky, S., et al.: Severity assessment and progression prediction of covid-19 patients based on the lesionencoder framework and chest ct. Information 12(11), 471 (2021)
  * [9] He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition. pp. 770–778 (2016)
  * [10] He, K., Zhao, W., Xie, X., Ji, W., Liu, M., Tang, Z., Shi, Y., Shi, F., Gao, Y., Liu, J., et al.: Synergistic learning of lung lobe segmentation and hierarchical multi-instance classification for automated severity assessment of covid-19 in ct images. Pattern recognition 113, 107828 (2021)
  * [11] Hou, J., Xu, J., Feng, R., Zhang, Y., Shan, F., Shi, W.: Cmc-cov19d: Contrastive mixup classification for covid-19 diagnosis. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 454–461 (2021)
  * [12] Hou, J., Xu, J., Jiang, L., Du, S., Feng, R., Zhang, Y., Shan, F., Xue, X.: Periphery-aware covid-19 diagnosis with contrastive representation enhancement. Pattern Recognition 118, 108005 (2021)
  * [13] Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition. pp. 2261–2269 (2017)
  * [14] Huang, L., Han, R., Ai, T., Yu, P., Kang, H., Tao, Q., Xia, L.: Serial quantitative chest ct assessment of covid-19: a deep learning approach. Radiology: Cardiothoracic Imaging 2(2) (2020)
  * [15] Ioannou, G., Papagiannis, T., Tagaris, T., Alexandridis, G., Stafylopatis, A.: Visual interpretability analysis of deep cnns using an adaptive threshold method on diabetic retinopathy images. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 480–486 (2021)
  * [16] Javaheri, T., Homayounfar, M., Amoozgar, Z., Reiazi, R., Homayounieh, F., Abbas, E., Laali, A., Radmard, A.R., Gharib, M.H., Mousavi, S.A.J., et al.: Covidctnet: An open-source deep learning approach to identify covid-19 using ct image. arXiv preprint arXiv:2005.03059 (2020)
  * [17] Jin, S., Wang, B., Xu, H., Luo, C., Wei, L., Zhao, W., et al.: Ai-assisted ct imaging analysis for covid-19 screening: building and deploying a medical ai system in four weeks. MedRxiv (2020)
  * [18] Kendall, A., Gal, Y., Cipolla, R.: Multi-task learning using uncertainty to weigh losses for scene geometry and semantics. In: 2018 IEEE Conference on Computer Vision and Pattern Recognition. pp. 7482–7491 (2018)
  * [19] Khosla, P., Teterwak, P., Wang, C., Sarna, A., Tian, Y., Isola, P., Maschinot, A., Liu, C., Krishnan, D.: Supervised contrastive learning. In: Annual Conference on Neural Information Processing Systems 2020 (2020)
  * [20] Kollias, D., Arsenos, A., Kollias, S.: Ai-mia: Covid-19 detection & severity analysis through medical imaging. arXiv preprint arXiv:2206.04732 (2022)
  * [21] Kollias, D., Arsenos, A., Soukissian, L., Kollias, S.: Mia-cov19d: Covid-19 detection through 3-d chest ct image analysis. arXiv preprint arXiv:2106.07524 (2021)
  * [22] Kollias, D., Bouas, N., Vlaxos, Y., Brillakis, V., Seferis, M., Kollia, I., Sukissian, L., Wingate, J., Kollias, S.: Deep transparent prediction through latent representation analysis. arXiv preprint arXiv:2009.07044 (2020)
  * [23] Kollias, D., Tagaris, A., Stafylopatis, A., Kollias, S., Tagaris, G.: Deep neural architectures for prediction in healthcare. Complex & Intelligent Systems 4(2), 119–131 (2018)
  * [24] Kollias, D., Vlaxos, Y., Seferis, M., Kollia, I., Sukissian, L., Wingate, J., Kollias, S.D.: Transparent adaptation in deep medical image diagnosis. In: TAILOR. pp. 251–267 (2020)
  * [25] Lin, T.Y., Dollár, P., Girshick, R., He, K., Hariharan, B., Belongie, S.: Feature pyramid networks for object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 2117–2125 (2017)
  * [26] Ma, J., Wang, Y., An, X., Ge, C., Yu, Z., Chen, J., Zhu, Q., Dong, G., He, J., He, Z., Cao, T., Zhu, Y., Nie, Z., Yang, X.: Towards data-efficient learning: A benchmark for covid-19 ct lung and infection segmentation. Medical Physics 48(3), 1197–1210 (2021). https://doi.org/https://doi.org/10.1002/mp.14676
  * [27] Milletari, F., Navab, N., Ahmadi, S.A.: V-net: Fully convolutional neural networks for volumetric medical image segmentation. In: 2016 fourth international conference on 3D vision (3DV). pp. 565–571. IEEE (2016)
  * [28] Pu, J., Leader, J.K., Bandos, A., Ke, S., Wang, J., Shi, J., Du, P., Guo, Y., Wenzel, S.E., Fuhrman, C.R., et al.: Automated quantification of covid-19 severity and progression using chest ct images. European radiology 31(1), 436–446 (2021)
  * [29] Qi, X., Jiang, Z., Yu, Q., Shao, C., Zhang, H., Yue, H., et al.: Machine learning-based ct radiomics model for predicting hospital stay in patients with pneumonia associated with sars-cov-2 infection: A multicenter study. medRxiv (2020)
  * [30] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: International Conference on Medical image computing and computer-assisted intervention. pp. 234–241. Springer (2015)
  * [31] Shan, F., Gao, Y., Wang, J., Shi, W., Shi, N., Han, M., et al.: Lung infection quantification of covid-19 in ct images with deep learning. arXiv preprint arXiv:2003.04655 (2020)
  * [32] Shi, F., Wang, J., Shi, J., Wu, Z., Wang, Q., Tang, Z., et al.: Review of artificial intelligence techniques in imaging data acquisition, segmentation and diagnosis for covid-19. IEEE Reviews in Biomedical Engineering pp. 1–1 (2020)
  * [33] Tang, Z., Zhao, W., Xie, X., Zhong, Z., Shi, F., Liu, J., et al.: Severity assessment of coronavirus disease 2019 (covid-19) using quantitative features from chest ct images. arXiv preprint arXiv:2003.11988 (2020)
  * [34] Wang, S., Kang, B., Ma, J., Zeng, X., Xiao, M., Guo, J., et al.: A deep learning algorithm using ct images to screen for corona virus disease (covid-19). European radiology pp. 1–9 (2021)
  * [35] WHO: Coronavirus disease (covid-19) pandemic. https://www.who.int/emergencies/diseases/novel-coronavirus-2019 (2022)
  * [36] Xu, X., Jiang, X., Ma, C., Du, P., Li, X., Lv, S., et al.: A deep learning system to screen novel coronavirus disease 2019 pneumonia. Engineering 6(10), 1122–1129 (2020)
  * [37] Zhang, H., Wu, C., Zhang, Z., Zhu, Y., Lin, H., Zhang, Z., Sun, Y., He, T., Mueller, J., Manmatha, R., et al.: Resnest: Split-attention networks. arXiv preprint arXiv:2004.08955 (2020)
  * [38] Zhang, H., Cisse, M., Dauphin, Y.N., Lopez-Paz, D.: mixup: Beyond empirical risk minimization. arXiv preprint arXiv:1710.09412 (2017)
  * [39] Zheng, C., Deng, X., Fu, Q., Zhou, Q., Feng, J., Ma, H., Liu, W., Wang, X.: Deep learning-based detection for covid-19 from chest ct using weak label. MedRxiv (2020)
  * [40] Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., Torralba, A.: Learning deep features for discriminative localization. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 2921–2929 (2016)
  * [41] Zhou, Z., Rahman Siddiquee, M.M., Tajbakhsh, N., Liang, J.: Unet++: A nested u-net architecture for medical image segmentation. In: Deep learning in medical image analysis and multimodal learning for clinical decision support, pp. 3–11. Springer (2018)


# Nonequilibrium design strategies for functional colloidal assemblies

Avishek Das  Department of Chemistry, University of California, Berkeley, CA,
94720, USA  David T. Limmer<EMAIL_ADDRESS>Department of Chemistry,
University of California, Berkeley, CA, 94720, USA  Chemical Sciences
Division, LBNL, Berkeley, CA, 94720, USA  Material Sciences Division, LBNL,
Berkeley, CA, 94720, USA  Kavli Energy NanoSciences Institute, University of
California, Berkeley, CA, 94720, USA

###### Abstract

We use a nonequilibrium variational principle to optimize the steady-state,
shear-induced interconversion of self-assembled nanoclusters of DNA-coated
colloids. Employing this principle within a stochastic optimization algorithm
allows us to discover design strategies for functional materials. We find that
far-from-equilibrium shear flow can significantly enhance the flux between
specific colloidal states by decoupling trade-offs between stability and
reactivity required by systems in equilibrium. For isolated nanoclusters, we
find nonequilibrium strategies for amplifying transition rates by coupling a
given reaction coordinate to the background shear flow. We also find that
shear flow can be made to selectively break detailed balance and maximize
probability currents by coupling orientational degrees of freedom to
conformational transitions. For a microphase consisting of many nanoclusters,
we study the flux of colloids hopping between clusters. We find that a shear
flow can amplify the flux without a proportional compromise on the microphase
structure. This approach provides a general means of uncovering design
principles for nanoscale, autonomous, functional materials driven far from
equilibrium.

††preprint: APS/123-QED

Self-assembly is a dynamic process and its optimal design involves
simultaneously stabilizing a target structure while preserving a high rate of
conversion into it from a fixed set of precursors 1; 2; 3; 4. In and near
thermal equilibrium, fundamental trade-offs exist between these two design
criteria making them difficult to fulfill concurrently 5. For example, strong,
specific interactions are needed to stabilize select structures over others,
but strong interactions result in the slow relaxation of defects, and highly
specific interactions can stifle growth 6. It is known that far-from-
equilibrium conditions may generally accelerate rare reactive events 7; 8, and
this has been leveraged in simple models of self-assembly 9; 10. However, the
extent to which nonequilibrium conditions can aid the design of self-
assembling systems by breaking constraints between the factors that stabilize
structures and those that aid in the dynamics of their formation is largely
unknown. Moreover, there is currently a lack of general principles to inform
strategies employing nonequilibrium conditions to assemble materials with
unique structure or function, beyond what is possible in equilibrium. Specific
examples in colloidal systems however point to the potential utility of such
strategies 11; 12; 13. Here we present a framework for learning the optimal
design principles for a general nonequilibrium self-assembling system, and
demonstrate how the continuous injection of energy significantly expands the
available design space of functional materials assembly.

Functional materials cycle between multiple conformations in order to achieve
a dynamical task. Any functional structure like an enzyme or a molecular motor
must not only be stable, but also necessarily able to disassemble and
reassemble in a dynamic nonequilibrium steady-state 14; 15. Such materials
typically form internal reaction networks of disassembled intermediates with
multiple nonzero reactive fluxes, driven by external forces that are
autonomous 16; 17; 18; 19. Autonomous driving is time-independent,
energetically cheap to implement and acts nonspecifically on the entire system
20. Functional materials efficiently transduce chemical and mechanical energy
by optimally coupling the external dissipative forces to specific edges in
their internal reaction networks. This may modify reactive fluxes and break
detailed balance in those reaction coordinates that are then coupled to their
output work. Thus, optimizing the dynamics of self-assembly for probability
fluxes in and out of a target structure is the path to designing functional
materials. However, a framework for doing so is complicated as typical
relationships concerning stability and response valid in equilibrium no longer
hold.

Using a nonequilibrium variational framework, we develop a route to discover
design rules for driven self-assembly. This framework uses insights from large
deviation theory and stochastic thermodynamics 21; 22. In particular, we probe
the dynamics and outcome of self-assembly by analyzing order parameters over
trajectories from molecular dynamics simulations 23; 24; 25. Previous work on
optimizing the reactivity of self-assembled colloidal clusters have used
Kramers approximations valid only at equilibrium 26 or have looked at single
transient realizations of reactive barrier crossing 27. The design of optimal
disassembly and reassembly, i.e., nonzero probability fluxes and currents in
and out of functional structures in nonequilibrium steady-states, remains
unexplored. Employing a variational principle, we adapt a previous algorithm
used to study the optimal design of passive DNA-coated colloids, to design
instead for enhanced reactive fluxes. We study the effectiveness of this
approach in isolated colloidal nanoclusters and in a microphase of many
nanoclusters, both in a sheared nonequilibrium steady-state. Using the inverse
design algorithm and rationalizing the discovered designs in parameter space,
we find nonequilibrium strategies for breaking trade-offs between structure
and dynamics.

## Model of DNA coated colloids

We have modeled the self-assembly of DNA-labeled colloidal nanoparticles via
overdamped Langevin dynamics simulations of $N$ particles, interacting
pairwise with a coarse-grained potential $V(r)$, with $r$ being the center-to-
center distance between two colloids, see Fig. 1c. The potential consists of a
volume-exclusion repulsion modeled with a Weeks-Chandler-Andersen (WCA)
potential 28 with an exclusion diameter of $\sigma$, and selective short-range
attractive interactions modeled with a Morse potential with amplitude $D_{ij}$
between pairwise particle indices $i$ and $j$. Such a potential has been
demonstrated previously to accurately represent selective attractions between
two DNA-labeled surfaces 29; 30. Self-assembly with this model can be designed
to yield nanoclusters of pre-specified sizes and symmetry, by tuning $D_{ij}$
with respect to $k_{\mathrm{B}}T$, or Boltzmann’s constant times temperature
31; 32; 21. We consider this self-assembling system evolving within a
nonequilibrium steady-state under a shear flow with a constant shear flow rate
$f$. The shear flow displaces the particles by competing with the natural
diffusive velocity gradient scale for the system,
$f^{*}=k_{\mathrm{B}}T/\sigma^{2}$. Given the mechanical driving force of
shear performs dissipative work on the system, it may change the relative
stability of configurations as well as the rate of interconversions between
them. We investigate whether optimal design principles can be found that can
tune the dynamics of self-assembly independently from configurational
probabilities for pre-specified target order parameters.

## Equilibrium constraints on self-assembly dynamics

Thermal equilibrium constrains the dynamics of rates of interconversion
between stable self-assembled structures to the yields of those structures.
Both quantities can be viewed as trajectory-averaged quantities. We denote a
simulated trajectory of particle positions $\mathbf{r}^{N}(t)$ as a function
of time $t$, by $X=\\{\mathbf{r}^{N}(0),\mathbf{r}^{N}(\delta
t),\dots,\mathbf{r}^{N}(t_{f})\\}$, with $t_{f}$ being the duration of the
trajectory and $\delta t$ the simulation time step. We take $t_{f}$ much
longer than the free colloidal diffusion timescale
$t^{*}=\gamma\sigma^{2}/k_{\mathrm{B}}T$, where $\gamma$ is the friction
coefficient for the free colloids in solution, such that the colloids relax
into a steady-state and trajectory averages of order parameters are
independent of $t_{f}$. The expected yield, $Y_{\mathrm{A}}$, of any target
structure A over this trajectory can be computed from a trajectory average of
the indicator function,

$Y_{\mathrm{A}}=\frac{1}{t_{f}}\int_{0}^{t_{f}}\mathbf{1}_{\mathrm{A}}[r^{N}(t)]dt\,,$
(1)

where $\mathbf{1}_{\mathrm{A}}[r^{N}(t)]$ returns 1 when the system is in
state A and is 0 otherwise. For computing the interconversion rates, we first
define the probability flux from state A to state B, $q_{\mathrm{AB}}$. The
steady state flux can be evaluated from a two-time correlation function
between the indicator functions for state A and B over a lag-time $\tau$,

$q_{\mathrm{AB}}=\frac{1}{\tau(t_{f}-\tau)}\int_{\tau}^{t_{f}}\mathbf{1}_{\mathrm{A}}[\mathbf{r}^{N}(t-\tau)]\mathbf{1}_{\mathrm{B}}[\mathbf{r}^{N}(t)]dt\,,$
(2)

which is a time-scaled, joint probability of being in A at some time and being
in B after time $\tau$. This flux directly counts the number of times A
transfers to B over a trajectory. If the lag-time $\tau$ is larger than the
relaxation time within the A state, but much smaller than the typical waiting
time for the transition, $q_{\mathrm{AB}}$ reports on the rate constant
$k_{\mathrm{AB}}$ as $k_{\mathrm{AB}}=q_{\mathrm{AB}}/Y_{\mathrm{A}}$,
independent of $\tau$ 33.

In thermal equilibrium, probability fluxes are balanced, such that
$q_{\mathrm{AB}}=q_{\mathrm{BA}}$ and the probability current
$j_{\mathrm{AB}}\equiv q_{\mathrm{AB}}-q_{\mathrm{BA}}=0$. This implies that
the ratio of forward and backward rates of interconversion $k_{\mathrm{AB}}$
and $k_{\mathrm{BA}}$ is strictly coupled to the relative stability of A and
B, $k_{\mathrm{AB}}/k_{\mathrm{BA}}=Y_{\mathrm{B}}/Y_{\mathrm{A}}$. A
configuration that is more energetically stable is necessarily less reactive
at equilibrium, and thus the rate of error correction and annealing towards
the globally stable state is reduced. This trade-off is the reason for the
existence of an optimal zone for self-assembly in or near thermal equilibrium
5. Far-from-equilibrium conditions may be designed to stabilize otherwise
unstable or exotic structures, but may not always break detailed balance, if
the driving force does not couple to the relevant reactive mode. We describe
next how to design far-from-equilibrium steady-states to tune rate constants
and structural stabilities independently, and how to break detailed balance in
any target coordinate.

Figure 1: Optimal probability flux between polytetrahedron and octahedron. (a)
Schematic of isomerization reaction. Volume exclusion due to the colloids are
shown in blue, and tunable pairwise bonds are shown in red. (b) Optimal
alphabets for high yield and high flux. (c) Pairwise interaction potential.
(d) and (e) Average values of observable $\langle
O\rangle\in\\{Y,k\tau,q\tau\\}$ with respect to shear, keeping the alphabets
fixed at $\mathbf{S}$ and $\mathbf{AS}$.

## Optimal design of function

We use an iterative optimization algorithm to find the best choice of design
parameters $\\{D_{ij}\\}$ and shear flow rate $f$ that maximizes fluxes
$q_{\mathrm{AB}}$ or currents $j_{\mathrm{AB}}$ for a given choice of states A
and B. We construct a cost function for optimization, $\langle\Omega\rangle$,
that encodes the minimal change to a reference system to maximize either
observable by minimizing the Kullback–Leibler divergence between the steady-
states of the reference and the controlled system 21. For the overdamped
dynamics we consider, this is given by
$\langle\Omega\rangle=\left\langle\lambda
O-\sum_{i=1}^{N}(\mathbf{u}_{i}-\mathbf{F}_{i})^{2}/4\gamma
k_{\mathrm{B}}T\right\rangle$, where $\lambda$ is a biasing parameter, $O$ is
a time-averaged static or dynamic order parameter, $q_{\mathrm{AB}}$ or
$j_{\mathrm{AB}}$ for example, $\mathbf{u}_{i}$ is the total force that the
$i$-th particle experiences, $\mathbf{F}_{i}$ is the WCA force from the
interparticle potential, and $\langle\dots\rangle$ denote an average over the
steady-state. The second term, deriving from the Kullback-Leibler divergence
between trajectory ensembles with and without the added force
$\mathbf{u}_{i}-\mathbf{F}_{i}$ 22, acts as a regularizer for the optimization
in regions of design space where the first term is either zero or degenerate.
We optimize $\langle\Omega\rangle$ through stochastic gradient descent by
computing its explicit gradients with respect to any design parameter $c$,
from a generalized fluctuation-response relation

$\frac{\partial\langle\Omega\rangle}{\partial
c}=\left\langle\frac{\partial\Omega}{\partial\mathbf{u}}.\frac{\partial\mathbf{u}}{\partial
c}\right\rangle+\int_{0}^{\infty}\left\langle\delta\Omega(t)\delta\left(\frac{\partial\dot{\ln}P[X]}{\partial\mathbf{u}}.\frac{\partial\mathbf{u}}{\partial
c}\right)(0)\right\rangle dt$ (3)

where $P[X]$ is the Onsager-Machlup probability of trajectory $X$34; 35. We
choose initial values for the design parameters such that $\langle O\rangle$
is nonzero, simulate a steady-state molecular dynamics trajectory to converge
the terms in Eq. 3, change the design parameters by taking a gradient descent
step, and keep iterating till we converge to a locally optimal design. In
practice, we find that as the gradients are stochastic, noise enhances the
exploration of parameter space and helps anneal rapidly into the global
optimum 21. We nevertheless test the uniqueness of each solution basin by
starting the optimization from different points in parameter space, such as
from high-yield designs for $\mathrm{A}$ and $\mathrm{B}$, or from a
nonspecific attraction that results in moderate yields of both. During
optimization, we constrain the magnitude of the bond energies to stay less
than $10k_{\mathrm{B}}T$ so that there are no long-lived kinetic traps that
prevent the relaxation of the system into a steady-state 36. In case of a
breakdown in ergodicity, the steady-state gradient expression in Eq. 3 would
not be accurate, instead the gradients of probability of finite duration
trajectories would need to be used 8. Further details about the variational
algorithm and a pseudocode are in the Supporting Information (SI).

## Symmetric and asymmetric strategies for maximal flux

We first apply this algorithm to the interconversion between polytetrahedral
(A: $\mathrm{C}_{\mathrm{2v}}$) and octahedral (B: $\mathrm{O}_{\mathrm{h}}$)
configurations in a sheared steady-state with $N=6$ particles, for a fixed
choice of lag-time $\tau=0.025t^{*}$, slightly longer than typical transition
timescales. The optimal design that maximizes yields $Y_{\mathrm{A}}$ and
$Y_{\mathrm{B}}$ are known to be the Maximal Alphabet $\mathbf{A}$ and
$\mathbf{B}$ for the $D_{ij}$ matrix, shown schematically in Fig. 1b, in which
a uniform attractive energy stabilizes each contact pair 21; 36. Alphabet
matrices possess a degeneracy to permutations of particle indices, a symmetry
that the variational optimization spontaneously breaks 21. In order to
interpret physically meaningful distinctions between alphabets and to
interpolate between them, we have permuted them to be the least different (see
SI for details).

Optimization for maximal $q_{\mathrm{AB}}$ yields $f/f^{*}=50$, the highest
allowed value during optimization, and two distinct $D_{ij}$ alphabet
solutions, denoted $\mathbf{S}$ and $\mathbf{AS}$. Maximizing
$q_{\mathrm{BA}}$ however yields only the alphabets $\mathbf{S}$. Figure 1d
shows a slice through the design space in the direction of varying $f$ but
keeping $D_{ij}$ fixed. Far-from-equilibrium conditions amplify the fluxes by
increasing the rates of interconversion in both directions. At $f=0$, detailed
balance constraints $q_{\mathrm{AB}}=q_{\mathrm{BA}}$ as seen by the trade-
offs in $(Y_{\mathrm{A}},k_{\mathrm{AB}})$ and
$(Y_{\mathrm{B}},k_{\mathrm{BA}})$. The $\mathbf{S}$ matrix maximizes both
$q_{\mathrm{AB}}$ and $q_{\mathrm{BA}}$ symmetrically, such that probability
flows predominately between states A and B even far-from-equilibrium. This is
accomplished with alphabet $\mathbf{S}$ being intermediate in bond-strength to
the alphabets $\mathbf{A}$ and $\mathbf{B}$. $\mathbf{S}$ preserves the common
bonds in $\mathbf{A}$ and $\mathbf{B}$ and codes for weak bond energies for
the unique bond in each structure.

With the other solution $\mathbf{AS}$, shear flow amplifies the flux in the
forward direction more than the reverse, breaking detailed balance. Figure 1e
shows how this is achieved by starting with a $Y_{\mathrm{A}}$ higher than the
previous case and thus a smaller $k_{\mathrm{AB}}$, at equilibrium. At high
shear, the $k_{\mathrm{AB}}$ is amplified to similar absolute values as with
$\mathbf{S}$, without a proportional reduction in $Y_{\mathrm{A}}$. The
solution $\mathbf{AS}$ codes for a dynamical bonding arrangement different
from $\mathbf{A}$ or $\mathbf{B}$. This class of asymmetric high-flux
solutions is generally found as a locally optimal design for other pairs of
nanoclusters as well. We note that the alphabet in question is not asymmetric,
as it is generated from pairwise conservative interactions, but rather that
the alphabet causes $q_{\mathrm{AB}}$ and $q_{\mathrm{BA}}$ to be
asymmetrically amplified, breaking detailed balance.

In both cases, the fluxes reach a maximum with $f$ before turning over and
decaying at large shear rates that are nevertheless experimentally achievable
in microfluidic devices 37. This turnover arises from a competition between
the enhancement of diffusive exploration of the internal degrees of freedom
till moderate flow rates, and the destabilization of bonds at high flow rates.
This turnover is similar in origin to a turnover previously reported in shear-
induced crystallization rates of hard spheres at high supersaturation, where
the cohesive free energy is much larger than $k_{\mathrm{B}}T$ 38.

Figure 2: Optimal breakdown of detailed balance induced by shear flow. (a)
Optimized alphabet for high current. (b) Schematic of the isomerization
intermediate. (c) Reactive displacement trajectories $J_{\mathrm{AB}}(t)$. (d)
Average values of observable $\langle O\rangle\in\\{Y,k\tau,q\tau\\}$ with
respect to shear, keeping the alphabet fixed at $\mathbf{C}$. (e) Probability
distribution of the orientation of the nanocluster, $e_{x}$, while in steady-
state in $\mathrm{A}$ (black circles), immediately after an
$\mathrm{A}\to\mathrm{B}$ reaction (black triangles), steady-state in
$\mathrm{B}$ (orange circles), and immediately after a
$\mathrm{B}\to\mathrm{A}$ reaction (orange triangles). Blue arrows in inset
denote orientational relaxation. (f) Difference of covariance of shear work
$W$ and the time spent in 11-bonded state $\tau_{\mathrm{I}}$, during forward
and backward transitions.

## Strategy for maximal probability current

For the same polytetrahedron (A) and octahedron (B) clusters, we have also
found the optimal $D_{ij}$ matrices and shear flow rates $f$ that maximize the
probability currents $j_{\mathrm{AB}}$ and $j_{\mathrm{BA}}$. For
$j_{\mathrm{AB}}$, we find the optimal solution to be $f/f^{*}$ close to $50$
and alphabet $\mathbf{C}$, shown in Fig. 2a. For $j_{\mathrm{BA}}$
optimization, we do not find any design where $j_{\mathrm{BA}}$ is positive
and the optimization algorithm always finds the trivial maximum with low
$D_{ij}$ where $Y_{\mathrm{A}}=Y_{\mathrm{B}}=0$. A slice through the
parameter space in the $f$ direction, keeping alphabet $\mathbf{C}$ fixed, is
shown in Fig. 2d. The algorithm has found a design to break detailed balance
maximally in this pre-specified direction, by decoupling constraints between
$(Y_{\mathrm{A}},k_{\mathrm{AB}})$ and $(Y_{\mathrm{B}},k_{\mathrm{BA}})$.
When we compare these observables to those with the symmetric high-flux
alphabet $\mathbf{S}$ in Fig. 1d, we find that the high current with
$\mathbf{C}$ arises due to $Y_{\mathrm{B}}$ having a quicker turnover than
before. An equivalent explanation is $k_{\mathrm{BA}}$ does not increase as
expected from detailed balance considerations. As a result, structural
constraints have been decoupled optimally from rate constraints in
nonequilibrium. The gap between $Y_{\mathrm{A}}$ and $Y_{\mathrm{B}}$ in log-
scale, exactly balanced at equilibrium by the gap between $k_{\mathrm{AB}}$
and $k_{\mathrm{BA}}$, has been designed to no longer be so at the optimal
shear.

Broken detailed balance results in a reactive displacement over time
$J_{\mathrm{AB}}(t)=\int_{\tau}^{t+\tau}\left[\mathbf{1}_{\mathrm{A}}(t^{{}^{\prime}}-\tau)\mathbf{1}_{\mathrm{B}}(t^{{}^{\prime}})-\mathbf{1}_{\mathrm{B}}(t^{{}^{\prime}}-\tau)\mathbf{1}_{\mathrm{A}}(t^{{}^{\prime}})\right]dt^{{}^{\prime}}$,
where the full arguments of the indicators have been suppressed for brevity.
The expectation of this observable counts the difference between the expected
number of transitions happening in the forward and backward directions. In
Fig. 2c we have shown a typical trajectory of the reactive displacement with
time, at zero and optimal shear rate, for the alphabet $\mathbf{C}$. The
approximately uniform and rapid growth of $J_{\mathrm{AB}}$ with time, and the
comparable magnitudes of fluctuations in the two curves, shows how the shear
is preferentially amplifying the reactive flux in the forward direction and
that this happens through a higher number of small jumps rather than through
large rare events. This suggests that the reactive cycle involved in breaking
detailed balance in the $\mathrm{A}\to\mathrm{B}$ coordinate consists of all
edges with comparable reaction rates and comparable changes in configuration
space.

Shear flow is a nonspecific mechanical force acting on all particles, yet it
is able to drive a reactive cycle in the internal coordinates of the
nanocluster. This is achieved through autonomous relaxations in the
orientation of the nanoclusters. We demonstrate this by plotting probability
distributions of the x-component of the normalized principal eigenvector of
the gyration tensor of the nanocluster, $e_{x}$, in Fig. 2e. The $x$ component
of the gyration tensor $e_{x}$ measures the orientational alignment of the
cluster in the flow direction. Aspherical clusters like $\mathrm{A}$ have
larger typical value of $e_{x}$ than approximately spherical clusters like
$\mathrm{B}$, which does not align in the flow. We find that immediately after
reactions in both directions, orientations of the clusters are significantly
different from their steady-state values. Orientational motion aided by shear
flow and conformational transitions thus form a reactive cycle with a nonzero
current.

The distinction between the forward and backward transition paths with
alphabet $\mathbf{C}$ in the presence of shear flow can be quantified
mechanistically. Both clusters have 12 bonds, but to transition between the
two states requires breaking a bond on the concave face of the
$\mathrm{C}_{\mathrm{2v}}$ structure, passing through an 11-bonded
intermediate as shown in Fig. 2b, and then rebonding of two different
particles on that face. The optimized alphabet $\mathbf{C}$ stabilizes this
motion by weakening the bond in $\mathrm{A}$ located directly opposite to its
concave face. We have computed the effect of shear flow on the transition
paths through the covariance between the time spent in the 11-bonded
intermediate, $\tau_{\mathrm{I}}$, and the work done by shear flow,
$W=\sum_{i=1}^{N}\int_{t_{1}}^{t_{2}}\mathbf{f}_{i}^{\mathrm{S}}(\mathbf{r}_{i}).\dot{\mathbf{r}_{i}}dt$,
which is integrated between $t_{1}$ and $t_{2}$, the start and end of forward
and backward transition paths. The difference in covariance
$\chi=\langle\delta\tau_{\mathrm{I}}\delta
W\rangle_{\mathrm{AB}}-\langle\delta\tau_{\mathrm{I}}\delta
W\rangle_{\mathrm{BA}}$between forward and reverse reaction paths, is plotted
in Fig. 2f, with respect to increasing shear. The difference in the covariance
increases initially and then peaks near the optimal shear flow for high
current. This implies that even at the level of individual trajectories,
spending time in the 11-bonded intermediate state is correlated with the
effect of shear more in the trajectory ensemble of forward reactions than the
reverse ones. Past the turnover, the difference becomes smaller and then
negative, with fluctuations that grow with further increasing shear,
suggesting participation of competing sheared reaction pathways involving more
broken bonds.

Figure 3: Design for maximal probability current. (Top row) Minimal model of
shear-induced reactive cycle for a two-dimensional colloidal dimer. (a)
Reactive cycle comprising of shear-dependent reactions followed by shear-
dependent orientational relaxations. (b) Quadratic response of the probability
current to an increasing shear flow. (c) Profile of reactive currents in
7-particle clusters with changing shear, keeping the optimized alphabets
fixed. Solid (dashed) lines represent forward (backward) reactive flux. Colors
correspond to the specific edges in the reaction network in (d).

Detailed studies of the local stability of these strategies in design space
(SI) show that detailed balance is preserved all along linear interpolations
in parameter space between $\mathbf{A}\to\mathbf{S}\to\mathbf{B}$ even in a
high shear flow, with the highest flux occurring at $\mathbf{S}$ as found by
the optimization algorithm. Employing similar interpolations between
$\mathbf{S}\to\mathbf{AS}\to\mathbf{C}$, we find that $q_{\mathrm{AB}}$
increases monotonically between $\mathbf{C}\to\mathbf{AS}$, which explains why
$\mathbf{AS}$ can be discovered by gradient descent when starting from
$\mathbf{C}$. The close connection between solutions $\mathbf{C}$ and
$\mathbf{AS}$ suggests that the existence of a nontrivial high current locally
optimal solution in any reaction direction depends on the existence of an
asymmetric high flux solution with a positive current.

Figure 4: Independent tuning of hopping flux and microphase structure by shear
flow. (a) Pairwise interaction potential. (Inset) Schematic illustration of
hopping in a microphase. (Center and right columns) Color maps of (c) $Y_{6}$
at $f/f^{*}=1$, (d) $q_{6}\tau$ at $f/f^{*}=1$, (e) $Y_{6}$ at $f/f^{*}=25$
and (f) $q_{6}\tau$ at $f/f^{*}=25$, with respect to changing assembly and
disassembly barriers. Corresponding color scales are indicated next to each
color map. (b) Trajectories of the yield of particles of color $c$ as a
function of time, with different noise realizations.

## Generality of nonequilibrium strategies

We can better understand the coupling between shear-driven orientation and the
conformational change through a minimal model of a colloidal dimer in shear
flow $\mathbf{f}_{i}^{\mathbf{S}}=fz_{i}\mathbf{x}_{i}$ constrained to the
$x-z$ plane for simplicity. We assume that the dimer isomerizes between two
states $\alpha_{1}$ and $\alpha_{2}$, with center-to-center distances $L_{1}$
and $L_{2}$ respectively, and equilibrium rates $k_{12}=k_{0}$ and
$k_{21}=k_{0}e^{\beta\Delta V}$, where $\Delta V$ is the potential energy
difference between $\alpha_{2}$ and $\alpha_{1}$. This is schematically
illustrated in Fig. 3a. We assume $L_{2}=L_{1}+\Delta L$, such that a shear
flow may stretch the dimer and increase the rate of isomerizing from
$\alpha_{1}$ to $\alpha_{2}$. We assume the reactions occur at fixed
orientation, and in between reactions, the orientation of the dimer relaxes in
the shear flow. The orientation is quantified by the angle $\theta\in(0,2\pi)$
where $\cos\theta\equiv\hat{\mathbf{e}}\cdot\hat{\mathbf{x}}$ with
$\hat{\mathbf{e}}$ being the dimer axis. The torque due to the shear flow
depends on the state $\alpha_{j\in(1,2)}$ and is given by
$\Gamma=fL_{j}^{2}[1-\cos(2\theta)]/4$, which together with an angular
diffusion constant $D_{\theta}$, relaxes the distribution $P(\theta)$. In the
limit of small $fL_{j}^{2}/D_{\theta}$, the distribution $P_{j}(\theta)$ for
each state $\alpha_{j\in(1,2)}$ can be computed as a perturbation over free
diffusion, $P_{j}(\theta)=\\{1+[8D_{\theta}p\sin 2\theta-p^{2}(\cos
4\theta-4\cos 2\theta)]/16D_{\theta}^{2}\\}N_{j}/2\pi$, where $p=fL_{j}^{2}/4$
and $N_{j}$ is a normalization factor 39. Longer clusters thus align more with
shear flow, similar to Fig. 2e. We further assume that the yields of
$\alpha_{1}$ and $\alpha_{2}$ do not change from equilibrium, such that
$N_{1}/N_{2}=e^{\beta\Delta V}$, consistent with our explicit model findings.

Given an orientation, the stretching force is $\Pi=fL_{j}\sin(2\theta)/4$ and
upto linear order in $\Delta L$, the rates of extension and compression are
affected as $k_{12}=k_{0}e^{\beta\Pi\Delta L\cos\phi_{1}}$ and
$k_{21}=k_{0}e^{\beta(\Delta V+\Pi\Delta L\cos\phi_{2})}$ respectively 7,
where $\phi_{j\in(1,2)}$ is the angle between the reaction coordinate and the
stretching mode. We can now compute the forward and backward fluxes as
probability times reaction rate integrated over all $\theta$. The steady-state
reactive current from $\alpha_{1}$ to $\alpha_{2}$ becomes

$j_{12}=\frac{k_{0}L^{3}}{64D_{\theta}^{2}[1+e^{-\beta\Delta V}]}\Delta
L(\cos\phi_{1}-\cos\phi_{2})f^{2}$ (4)

in the limit of small $\Delta L/L$ where $L=(L_{1}+L_{2})/2$. The quadratic
response of the current in the limit of small $\phi_{j}$ is illustrated in
Fig. 3b. Hence shear flow is able to produce a reactive current by coupling
internal conformation with global orientation in a four-stroke cycle, with the
sign of the current depending only on the geometry of the reaction
coordinates. While simple, this model explains the observed features in the
self-assembly of nanoclusters, such as why there exist no solutions for
positive current in some reaction directions. Based on our minimal model,
bidirectional currents are only possible when there exist multiple reaction
mechanisms, i.e., different sets of $(\phi_{1},\phi_{2})$. Further the dimer
model predicts that the magnitude of the current $j_{12}$ increases with
increasing $\Delta V$, i.e., with a more stable $\alpha_{1}$, when the
timescale arising from $k_{0}$ is held constant. This rationalizes the
observation that the alphabet $\mathbf{C}$ in Fig. 2d that drives current
$j_{AB}$ has a higher $Y_{A}/Y_{B}$ at equilibrium, compared to alphabet
$\mathbf{S}$.

A high reactive flux simultaneously in both directions is achieved through
having a low free-energy barrier such that the reaction timescale is faster
than shear-induced rotations, with the fluxes being amplified symmetrically by
a shear flow. A high reactive current is achieved through having a favorable
reaction coordinate geometry and an intermediate barrier height, to slow down
the reactive timescale and couple it to to rotational timescale. For
nanocluster rearrangements where the free energy barrier and reaction
coordinate geometry are not available, the variational algorithm can directly
optimize for the reactive flux and current. We note that this method of
converting mechanical driving to chemical work through switchable geometry is
reminiscent of the blood-clotting protein von Willebrand factor that uses the
shear flow to unfold 40. The shear-induced enhancement of reactive flux and
current in our theory is contingent only on two aspects, the alignment of
aspherical clusters along a shear flow, and the modulation of reaction rates
by dissipative driving. Both are general results with numerical and
experimental verification in colloidal suspensions 41; 42; 7, polymers 43; 44
and biomolecules 45; 46.

The generality of the principle can be tested by studying probability fluxes
and currents for interconversions between four rigid nanoclusters with $N=7$
DNA-labeled particles, as shown in Fig. 3c) and d), with additional results in
the SI. These are the four highest yield clusters at equilibrium for a
nonspecific short-range pairwise attraction 21; 36. We find that the
nonequilibrium strategies discovered for interconverting the polytetrahedron
and octahedron nanoclusters are also applicable more widely. When pairwise
interaction matrices are set to the optimized alphabets, most reactive fluxes
are amplified by a far-from-equilibrium shear flow due to a larger enhancement
in rate constants compared to the decay in yields. The far-from-equilibrium
optima exhibit qualitatively different dynamics than a near-equilibrium
perturbative regime. For high probability flux in pairwise transitions, in
most cases we find locally optimal symmetric alphabets that preserve detailed
balance even far-from-equilibrium. The symmetric solutions are always locally
optimal in both forward and reverse directions of the transitions. As before,
these solutions result from restricting transitions between only these two
tagged configurations, and preventing leakage to competing structures. In many
cases we find distinct locally optimal asymmetric alphabets that break
detailed balance at large $f$. The alphabets for this strategy are typically
only optimal for reactive flux in one reaction direction. When optimized for
high probability current, we find many locally optimal alphabets that channel
the shear flow into breaking detailed balance in a particular reaction
direction, and thus maximizes the difference between forward and backward
reactive flux. These alphabets are distinct from, but closely related to, the
asymmetric high-flux alphabets. For any given reaction, optimal high-current
solutions exist if and only if there are optimal asymmetric high-flux
solutions that have a positive current.

## Particle hopping flux in a sheared microphase

Extending these strategies to a macroscopic system of DNA-labeled colloids
straightforwardly requires a prohibitive $\mathcal{O}(N^{2})$ scaling of the
necessary alphabet size, as any colloid in a distinct nanocluster must not
attract bonding partners from adjacent nanoclusters 36; 32. This attraction is
a nanoscale manifestation of surface tension that tries to relax the system
towards a bulk condensate. If however a long-range repulsion arising from a
screened Coulomb interaction is combined with the short-range DNA-labeled
attractive forces, large systems can self-assemble into a microphase at
equilibrium, characterized by the spontaneous emergence of an intermediate
length-scale at which nanoclusters are stabilized 47; 48. Microphase
separation or self-limited assembly is often used in biological systems to
regulate the volumes of viral capsids, cells and membraneless organelle 49;
50; 51.

Self-limited clusters in biological systems are not static structures but
highly fluxional. Nonequilibrium surface reactions are used by cells to
stabilize microphase droplets 52; 53; 54, and components of self-limited
clusters continuously exchange with the surrounding solute pool in a dynamic
steady-state 55. Here we focus on how to tune the dynamics of particle
exchange between nanoclusters given we have a microphase separated state. By
studying the optimal design of a particle hopping flux between multiple
nanoclusters, we demonstrate that a sheared nonequilibrium steady-state can
break equilibrium constraints relating stability and flux.

We start with a microphase separated state at equilibrium, self-assembled from
DNA-labeled colloids having a pairwise repulsive Yukawa potential
$V(r)=\epsilon\exp(-\kappa r)/r$ in addition to the previous short-ranged
interactions. We are interested in the yield and reactive flux of compact
nanoclusters consisting of 6 colloids and with at least 9 bonds, regardless of
its shape. Such nanoclusters are denoted as state A. The cutoff on the number
of bonds ensures we are accurately counting reactions involving hopping of
particles between clusters instead of complete dissociation of a cluster. We
calculate the average yield $Y_{\mathrm{6}}$ at any time $t$, by checking
whether each particle is in a cluster of type A, as
$Y_{\mathrm{6}}=\int_{0}^{t_{f}}\sum_{i=1}^{N}\mathbf{1}_{i\in\mathrm{A}}(t)dt/Nt_{f}$.
For counting particle hops we take a lag-time $\tau=0.025t^{*}$ slightly
larger than the timescale of typical adsorption and desorption transitions of
colloids from nanoclusters. With this, our hopping flux between 6-colloid
clusters becomes

$q_{6}=\frac{1}{N\tau(t_{f}-\tau)}\int_{\tau}^{t_{f}}\sum_{i=1}^{N}\mathbf{1}_{i\in\mathrm{A}^{{}^{\prime}}}(t-\tau)\mathbf{1}_{i\in\mathrm{A}^{{}^{\prime\prime}}}(t)dt$
(5)

where $\mathrm{A}^{{}^{\prime}}$ and $\mathrm{A}^{{}^{\prime\prime}}$ refer to
distinct clusters of type A with the only common particle being that with
index $i$. This condition ensures that only the exchange of a single particle
at a time, i.e. a hop, is being counted, as illustrated in Fig. 4a (inset).

As we are not recording the shapes of clusters so long that they are compact,
we do not use an alphabet in our design space, but rather a nonspecific value
of the attraction $D$ that competes against the repulsion energy scale
$\epsilon$ over a lengthscale of $\kappa^{-1}$. We constrain the total
pairwise force and potential to go to zero at a cutoff distance of $5\sigma$
with a shifted forces approximation 56. This constraint leaves the potential
energy function with two relevant effective features, the energy barrier to
bond formation or the assembly barrier $\Delta E_{\mathrm{a}}$, and the energy
barrier to bond breaking or the disassembly barrier $\Delta E_{\mathrm{d}}$,
shown in Fig. 4a. We study the effect of far-from-equilibrium conditions on
$Y_{6}$ and $q_{6}$ by variationally optimizing $D$, $\epsilon$ and $\kappa$,
keeping the shear flow rate fixed at $f/f^{*}=1$ and 25. We also compute the
yield and reactive flux landscape by directly sweeping over the design
parameters such that we can elucidate their dependence on $\Delta
E_{\mathrm{a}}$ and $\Delta E_{\mathrm{d}}$.

## Breaking equilibrium constraints on hopping flux

At equilibrium, large values of both barriers $\Delta E_{\mathrm{d}}$ and
$\Delta E_{\mathrm{a}}$ stabilizes the microphase, where their difference
$\Delta E_{\mathrm{d}}-\Delta E_{\mathrm{a}}$ is the energy gain on forming a
new bond starting from a dispersed state. The reactive flux for the bond
formation and breakage, however, is dependent on both of these energy
barriers, slowing exponentially as they increase. As can be anticipated from
detailed balance, near equilibrium changing $\Delta E_{\mathrm{d}}$ or $\Delta
E_{\mathrm{a}}$ to optimize yield degrades the reactive flux and vice-versa.
This trade-off is shown near thermal equilibrium, $f/f^{*}=1$, in Fig. 4c and
4d in maps of $Y_{6}$ and $q_{6}$, for a system of $N=90$ colloids at a
packing fraction $\phi=0.02$, where the yield and reactive flux both exhibit
narrow peaks at distinct parameters.

Far-from-equilibrium, however, the constraint between yield and reactive flux
is decoupled, and both have broad regimes of near optimal behavior that
overlap one another. Shear flow thus allows an optimal regime in parameter
space having high yield of a microphase as well as a high reactive flux of
particles hopping between nanoclusters. This occurs by expanding the regime of
stability of the microphase over a broader $\Delta E_{\mathrm{a}}$ range than
near equilibrium (see Fig. 4e and 4f) through stabilizing nonrigid
nanoclusters 21. These nonrigid clusters are compact with $\geq 9$ bonds in
total, but have dangling bonds that are energetically more susceptible to
break, contributing to a high hopping flux. We find that though far-from-
equilibrium shear flow reduces the yield of the nanoclusters by 30% from the
near-equilibrium regime, the highest reactive flux is amplified by more than
600%. This points at a large amplification in the underlying rate constants by
the breakdown of equilibrium constraints. We also show in the SI that the
enhancement of hopping flux is explained by the alignment of nonrigid clusters
in the shear flow and the enhancement of diffusion constant in the flow
direction. These findings are in qualitative agreement with the shear-induced
flux enhancement in our dimer model. In all cases the variational algorithm
optimizes the parameters accurately to arrive close to the optima given it is
started from a point in parameter space where either observable is measurably
non-zero (SI).

We have demonstrated an application of the increased hopping flux between
nanoclusters by an example of an autocatalytic switching reaction. We assume
that colloids in the nanoclusters are energetically identical but each carry a
color, $c$ or $nc$, similar to phosphorylation tags on biomolecules or
competing conformations in proteins 55; 57. At $t=0$ in a steady-state only
one particle is initialized as $c$ with the rest being $nc$. Every time a
$c$-colored particle is in a cluster of type A, it instantly converts every
other particle in the cluster to $c$, similar to models for cooperative
switching in biomolecules 58. The yield $Y_{c}$ of particles of color $c$ is
expected to increase fastest when colloids from type A clusters hop fastest
directly to other type A clusters in a steady-state. Keeping $D$, $\epsilon$
and $\kappa$ fixed at the optimal value for the highest microphase yield
$Y_{6}$ at $f/f^{*}=1$, we show example trajectories of $Y_{c}$ for
$f/f^{*}=0$ and $25$ with a system size $N=300$, in Fig. 4b. We find that the
far-from-equilibrium system indeed shows a higher rate of increase of $Y_{c}$
over the equilibrium scenario. At this optimal design the mechanical and
nonspecific shear driving has been optimally channeled into amplifying
reactive flux in the hopping reaction coordinate.

We note that the reactive fluxes we discussed are autonomously driven within
the sheared steady-state assembly. The design principles we found rely on
coupling an external nonspecific driving force to an internal reaction
coordinate through an autonomously self-organizing degree of freedom, the
geometry. The accompanying algorithm can numerically discover such design
rules. Our insights on how such motifs can break equilibrium constraints in
specific reaction coordinates can help design molecular machines that
efficiently transduce energy through internal rearrangements 20. This work
also opens a promising window towards designing dynamical phases of functional
materials, and shows how to tune fluxionality without compromising structure,
by going to nonperturbative far-from-equilibrium regimes 59.

Materials and Methods A pseudocode of our optimization algorithm, a detailed
description of the numerical simulations, learning curves, and optimal
alphabets are available in the SI.

Acknowledgements This work has been supported by NSF Grant CHE-1954580. AD was
also supported by the Philomathia Graduate Student Fellowship from the Kavli
Energy Nanoscience Institute at UC Berkeley.

Data availability A source code for the optimization algorithm and all data
that reproduce the findings of this study are openly available on Zenodo at
https://doi.org/10.5281/zenodo.8031998.60

## References

  * Whitesides and Grzybowski (2002) G. M. Whitesides and B. Grzybowski, Science 295, 2418 (2002).
  * Jacobs  _et al._ (2015) W. M. Jacobs, A. Reinhardt, and D. Frenkel, Proceedings of the National Academy of Sciences 112, 6313 (2015).
  * Fullerton and Jack (2016) C. J. Fullerton and R. L. Jack, The Journal of Chemical Physics 145, 244505 (2016).
  * Coropceanu  _et al._ (2022) I. Coropceanu, E. M. Janke, J. Portner, D. Haubold, T. D. Nguyen, A. Das, C. P. Tanner, J. K. Utterback, S. W. Teitelbaum, ¸. M. H. Hudson, _et al._ , Science 375, 1422 (2022).
  * Whitelam and Jack (2015) S. Whitelam and R. L. Jack, Annual review of physical chemistry 66, 143 (2015).
  * Whitelam  _et al._ (2009) S. Whitelam, E. H. Feng, M. F. Hagan, and P. L. Geissler, Soft Matter 5, 1251 (2009).
  * Kuznets-Speck and Limmer (2021) B. Kuznets-Speck and D. T. Limmer, Proceedings of the National Academy of Sciences 118 (2021).
  * Das  _et al._ (2022) A. Das, B. Kuznets-Speck, and D. T. Limmer, Physical Review Letters 128, 028005 (2022).
  * Whitelam (2018) S. Whitelam, The Journal of chemical physics 149, 104902 (2018).
  * Nguyen  _et al._ (2021) M. Nguyen, Y. Qiu, and S. Vaikuntanathan, Annual Review of Condensed Matter Physics 12 (2021).
  * Charbonneau and Reichman (2007) P. Charbonneau and D. Reichman, Physical Review E 75, 050401 (2007).
  * Chremos  _et al._ (2010) A. Chremos, K. Margaritis, and A. Z. Panagiotopoulos, Soft Matter 6, 3588 (2010).
  * Stenhammar  _et al._ (2013) J. Stenhammar, A. Tiribocchi, R. J. Allen, D. Marenduzzo, and M. E. Cates, Physical review letters 111, 145702 (2013).
  * Wang  _et al._ (2009) Y. Wang, H. Xu, and X. Zhang, Advanced Materials 21, 2849 (2009).
  * Atkin-Smith and Poon (2017) G. K. Atkin-Smith and I. K. Poon, Trends in cell biology 27, 151 (2017).
  * Grzybowski  _et al._ (2017) B. A. Grzybowski, K. Fitzner, J. Paczesny, and S. Granick, Chemical Society Reviews 46, 5647 (2017).
  * Murugan  _et al._ (2012) A. Murugan, D. A. Huse, and S. Leibler, Proceedings of the National Academy of Sciences 109, 12034 (2012).
  * Knoch and Speck (2017) F. Knoch and T. Speck, Physical Review E 95, 012503 (2017).
  * Albaugh and Gingrich (2022) A. Albaugh and T. R. Gingrich, Nature communications 13, 1 (2022).
  * Seifert (2012) U. Seifert, Rep. Prog. Phys. 75, 126001 (2012).
  * Das and Limmer (2021) A. Das and D. T. Limmer, The Journal of chemical physics 154, 014107 (2021).
  * Das and Limmer (2019) A. Das and D. T. Limmer, The Journal of chemical physics 151, 244123 (2019).
  * Hagan and Chandler (2006) M. F. Hagan and D. Chandler, Biophysical journal 91, 42 (2006).
  * Newton  _et al._ (2015) A. C. Newton, J. Groenewold, W. K. Kegel, and P. G. Bolhuis, Proceedings of the National Academy of Sciences 112, 15308 (2015).
  * Jack  _et al._ (2007) R. L. Jack, M. F. Hagan, and D. Chandler, Physical Review E 76, 021119 (2007).
  * Goodrich  _et al._ (2021) C. P. Goodrich, E. M. King, S. S. Schoenholz, E. D. Cubuk, and M. P. Brenner, Proceedings of the National Academy of Sciences 118 (2021).
  * Miskin  _et al._ (2016) M. Z. Miskin, G. Khaira, J. J. de Pablo, and H. M. Jaeger, Proc. Natl. Acad. Sci. USA 113, 34 (2016).
  * Weeks  _et al._ (1971) J. D. Weeks, D. Chandler, and H. C. Andersen, The Journal of chemical physics 54, 5237 (1971).
  * Rogers and Crocker (2011) W. B. Rogers and J. C. Crocker, Proceedings of the National Academy of Sciences 108, 15687 (2011).
  * Manoharan (2015) V. N. Manoharan, Science 349 (2015).
  * Meng  _et al._ (2010) G. Meng, N. Arkus, M. P. Brenner, and V. N. Manoharan, Science 327, 560 (2010).
  * Zeravcic  _et al._ (2014) Z. Zeravcic, V. N. Manoharan, and M. P. Brenner, Proceedings of the National Academy of Sciences 111, 15918 (2014).
  * Chandler (1978) D. Chandler, The Journal of Chemical Physics 68, 2959 (1978).
  * Onsager and Machlup (1953) L. Onsager and S. Machlup, Phys. Rev. 91, 1505 (1953).
  * Warren and Allen (2012) P. B. Warren and R. J. Allen, Phys. Rev. Lett. 109, 250601 (2012).
  * Hormoz and Brenner (2011) S. Hormoz and M. P. Brenner, Proceedings of the National Academy of Sciences 108, 5193 (2011).
  * Dou  _et al._ (2017) Y. Dou, B. Wang, M. Jin, Y. Yu, G. Zhou, and L. Shui, Journal of Micromechanics and Microengineering 27, 113002 (2017).
  * Richard and Speck (2015) D. Richard and T. Speck, Scientific reports 5, 1 (2015).
  * Gardiner  _et al._ (1985) C. W. Gardiner _et al._ ,  _Handbook of stochastic methods_ , Vol. 3 (springer Berlin, 1985).
  * Schneider  _et al._ (2007) S. Schneider, S. Nuschele, A. Wixforth, C. Gorzelanny, A. Alexander-Katz, R. Netz, and M. F. Schneider, Proceedings of the National Academy of Sciences 104, 7899 (2007).
  * Vermant and Solomon (2005) J. Vermant and M. J. Solomon, Journal of Physics: Condensed Matter 17, R187 (2005).
  * Toyabe  _et al._ (2007) S. Toyabe, H.-R. Jiang, T. Nakamura, Y. Murayama, and M. Sano, Physical Review E 75, 011122 (2007).
  * Wu  _et al._ (2006) M. W. Wu, R. A. Register, and P. M. Chaikin, Physical Review E 74, 040801 (2006).
  * Alexander-Katz  _et al._ (2006) A. Alexander-Katz, M. Schneider, S. Schneider, A. Wixforth, and R. Netz, Physical review letters 97, 138101 (2006).
  * Hill  _et al._ (2006) E. K. Hill, B. Krebs, D. G. Goodall, G. J. Howlett, and D. E. Dunstan, Biomacromolecules 7, 10 (2006).
  * Liphardt  _et al._ (2002) J. Liphardt, S. Dumont, S. B. Smith, I. Tinoco Jr, and C. Bustamante, Science 296, 1832 (2002).
  * Zhuang and Charbonneau (2016) Y. Zhuang and P. Charbonneau, The Journal of Physical Chemistry B 120, 7775 (2016).
  * Hagan and Grason (2021) M. F. Hagan and G. M. Grason, Reviews of Modern Physics 93, 025008 (2021).
  * Caspar and Klug (1962) D. L. Caspar and A. Klug, in  _Cold Spring Harbor symposia on quantitative biology_ , Vol. 27 (Cold Spring Harbor Laboratory Press, 1962) pp. 1–24.
  * Tanaka  _et al._ (2008) S. Tanaka, C. A. Kerfeld, M. R. Sawaya, F. Cai, S. Heinhorst, G. C. Cannon, and T. O. Yeates, science 319, 1083 (2008).
  * Courchaine  _et al._ (2016) E. M. Courchaine, A. Lu, and K. M. Neugebauer, The EMBO journal 35, 1603 (2016).
  * Zwicker  _et al._ (2015) D. Zwicker, A. A. Hyman, and F. Jülicher, Physical Review E 92, 012317 (2015).
  * Söding  _et al._ (2020) J. Söding, D. Zwicker, S. Sohrabi-Jahromi, M. Boehning, and J. Kirschbaum, Trends in Cell Biology 30, 4 (2020).
  * Kirschbaum and Zwicker (2021) J. Kirschbaum and D. Zwicker, Journal of the Royal Society Interface 18, 20210255 (2021).
  * Brackley  _et al._ (2017) C. A. Brackley, B. Liebchen, D. Michieletto, F. Mouvet, P. R. Cook, and D. Marenduzzo, Biophysical journal 112, 1085 (2017).
  * Toxvaerd and Dyre (2011) S. Toxvaerd and J. C. Dyre, “Communication: Shifted forces in molecular dynamics,” (2011).
  * Michaels  _et al._ (2020) T. C. Michaels, A. Šarić, S. Curk, K. Bernfur, P. Arosio, G. Meisl, A. J. Dear, S. I. Cohen, C. M. Dobson, M. Vendruscolo, _et al._ , Nature chemistry 12, 445 (2020).
  * Sourjik and Berg (2004) V. Sourjik and H. C. Berg, Nature 428, 437 (2004).
  * Whitelam  _et al._ (2012) S. Whitelam, R. Schulman, and L. Hedges, Physical review letters 109, 265506 (2012).
  * Das and Limmer (2023) A. Das and D. T. Limmer, “Nonequilibrium design strategies for functional colloidal assemblies,”  (2023).