# NAR-Former V2: Rethinking Transformer for Universal Neural Network
Representation Learning

Yun Yi${}^{1},$ Haokui Zhang2,3 Rong Xiao2 Nannan Wang1 Xiaoyu Wang2
1Xidian University 2Intellifusion 3Harbin Institute of Technology, Shenzhen
<EMAIL_ADDRESS><EMAIL_ADDRESS>
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>This work was
done while Yun Yi was an intern at Intellifusion

###### Abstract

As more deep learning models are being applied in real-world applications,
there is a growing need for modeling and learning the representations of
neural networks themselves. An efficient representation can be used to predict
target attributes of networks without the need for actual training and
deployment procedures, facilitating efficient network deployment and design.
Recently, inspired by the success of Transformer, some Transformer-based
representation learning frameworks have been proposed and achieved promising
performance in handling cell-structured models. However, graph neural network
(GNN) based approaches still dominate the field of learning representation for
the entire network. In this paper, we revisit Transformer and compare it with
GNN to analyse their different architecture characteristics. We then propose a
modified Transformer-based universal neural network representation learning
model NAR-Former V2. It can learn efficient representations from both cell-
structured networks and entire networks. Specifically, we first take the
network as a graph and design a straightforward tokenizer to encode the
network into a sequence. Then, we incorporate the inductive representation
learning capability of GNN into Transformer, enabling Transformer to
generalize better when encountering unseen architecture. Additionally, we
introduce a series of simple yet effective modifications to enhance the
ability of the Transformer in learning representation from graph structures.
Our proposed method surpasses the GNN-based method NNLP by a significant
margin in latency estimation on the NNLQP dataset. Furthermore, regarding
accuracy prediction on the NASBench101 and NASBench201 datasets, our method
achieves highly comparable performance to other state-of-the-art methods.

##  1 Introduction

With the maturity of deep learning technology, an increasing number of deep
network models of various sizes and structures are being proposed and
implemented in academic research and industrial applications. In this process,
the rapid deployment of networks and the design of new networks that meet task
requirements are significant. To address this issue, researchers propose using
machine learning models to solve the deployment and design problems of the
models themselves. One popular strategy is encoding the input neural network
and utilizing the resulting neural network representation to predict a
specific target attribute directly without actually executing the evaluation
program. In recent years, we have witnessed success in accelerating model
deployment and design processes with the help of neural network
representations [2; 3; 11; 10; 9; 8; 20]. Taking the advantages of latency
predictors [2; 3; 7; 6; 20; 31], significant time cost and expertise efforts
can be saved by not having to carry out the time-consuming process of
compilation, deployment, inference, and latency evaluation when engineers
choose networks for application. Through the use of accuracy predictors [2; 9;
10; 12; 20; 8; 26], researchers can avoid the resource-intensive process of
network training and instead perform a forward inference process to evaluate
the accuracy of a multitude of networks. This measure dramatically reduces the
time cost associated with network design.

Although the vanilla Transformer is designed for natural language processing,
Transformer architecture has found widespread adoption across diverse fields
owing to its strengths in global modeling and parallelizable computation [4;
5; 14; 19; 34; 30]. Very recently, several researchers have attempted to learn
appropriate representations for neural networks via Transformer [2; 9]. These
methods have indeed achieved leading performance on relevant tasks.
Nevertheless, they are mainly designed for encoding the architecture of cells
(basic micro units of repeatable neural networks) in cell-structured networks.
As shown in the latency prediction experiment in NAR-Former [2], poor
generalization performance occurs when the depth of the input architecture
reaches hundreds of layers. In the development process of neural network
representation learning, Graph neural network (GNN) [32; 35] is also a
promising technique for learning neural network representations [3; 24; 23;
22; 20]. They model the input neural architecture as a directed acyclic graph
(DAG) and operate on the graph-structured data, which comprises the node
information matrix and adjacency matrix. Recently, the NNLP [3] introduced a
dedicated latency prediction model based on GNNs, which is capable of encoding
the complete neural network having hundreds of layers and achieving a cutting-
edge advance.

In fact, both cell-structured architectures and complete neural networks are
widely used in various applications. Cell-structured models offer good
scalability, allowing for easy scaling by adding or removing cells. This
adaptability makes them suitable for addressing problems of different
complexities and data sizes, while also facilitating incremental model
development and deployment.omplete neural networks provide better flexibility
in connectivity and can achieve higher accuracy in certain cases. Furthermore,
in some cases, such as in latency estimation, encoding the complete network is
necessary. To handle various network architectures in different tasks, both
GNN-based and Transformer-based models are necessary. However, this issue of
utilizing multiple architectures can introduce constraints that may not be
conducive to practical applications. For instance, when a designed network
requires specific attributes, having similar model structures and high-
accuracy predictions for different attributes can reduce code redundancy and
improve work efficiency.

In this paper, we build upon the research conducted in NAR-Former [2] and
present a novel framework called NAR-Former V2 for universal neural network
representation learning. Our framework can handle cell-structured networks and
learning representations for the entire network. To accomplish this, we
incorporate graph-specific properties into the vanilla Transformer and
introduce a graph-aided attention-based Transformer block. This approach
combines the strengths of both Transformer and graph neural networks (GNNs).
Extensive experiments are conducted to evaluate our proposed framework.
Results show that:(1) our method can be applied to predict different
attributes, can outperform the state-of-the-art method in latency prediction
on the NNLQP dataset [3], and can achieve promising results in accuracy
sorting prediction on the NAS-Bench-101 and NAS-Bench-201 datasets [1; 21];
(2) our method has good scalability, which is capable of encoding network
having only a few operations or complete neural networks that have hundreds of
operations.

##  2 Related work

###  2.1 Representation and attribute prediction of neural networks

Neural network representation learning is the base for evaluating the
attributes of different networks via machine learning models. Early methods
[33; 26] construct representation models for learning neural network
representation based on LSTM and MLP. Peephole [33] inputs the embedding of
each layer to LSTM to predict accuracy, which neglects the topological
structure and is limited to handling only sequential architectures. Later, in
order to better capture the structural information of the network, an accuracy
predictor [8] uses a binary path encoding with a length equal to the number of
possible paths from input to output given in terms of operations, where the
element at the corresponding position of the path presenting in the input
network is set to 1. When the neural network is regarded as a directed acyclic
graph, the adjacency matrix describes the connection between nodes, so it is
naturally used to encode the topological structure of the neural network. NAS-
Bench-101 [1] proposed to encode the given neural network as a concatenated
vector of a flat adjacency matrix and a list of node labels. Many other
methods [3; 7; 11; 20; 22; 23] realize accuracy and latency prediction by
directly inputting the original two-dimensional adjacency matrix together with
the node information matrix to GNN, which can realize the explicit encoding of
the input network topology. Recently, other methods have focused on enhancing
the original GNN or introducing transformers to obtain more meaningful neural
network representations [16; 12; 9; 2].

###  2.2 Transformer

Transformer [34] is a self-attention-based neural network architecture that
has revolutionized natural language processing [15; 30; 17] and has been
adopted in many other fields [4; 5; 14; 2; 9; 18; 19]. Transformer has
recently been successfully introduced into neural network representation
learning [9; 2]. TNASP [9] inputs the sum of operation type embedding matrix
and Laplacian matrix into standard Transformer. NAR-Former [2], on the other
hand, encodes each operation and connection information of this operation into
token and inputs all tokens into a proposed multi-stage fusion transformer.
Excellent attribute prediction results have been achieved on cell-based
dataset by using these methods. However, the strong long-range modeling
ability of the self-attention mechanism may also result in subtle local
variation affecting the representation of all tokens. Due to the potential
impact of this feature on the generalization ability, although NAR-Former [2]
has made attempts to encode complete neural networks, the results are still
unsatisfactory.

###  2.3 Graph neural network

GNNs are designed to handle graph-structured data, which is a fundamental
representation for many real-world problems such as social network analysis
and recommendation systems [27; 29]. Given that neural networks can be viewed
as graphs, GNN-based models have emerged as a prominent and widely adopted
approach for neural network representation learning [3; 7; 11; 20; 22; 23].
GNNs show generalization ability through a simple mechanism of aggregating
information from neighbors. For instance, the recently proposed GNN-based
model [3] can obtain representations of neural networks with hundreds of
layers and achieves new state-of-the-art results in latency prediction, even
if the input network structure has not been seen during training.
Nevertheless, the simple structural characteristics of GNNs, which contribute
to its strong generalization ability, also lead to the need for further
improvement in the performance of methods based on original GNN in cellular
structure and complete neural network representation learning. Therefore, it
is a promising approach for neural network representation learning to combine
the Transformer and GNN to leverage the strengths of both models.

Figure 1: Diagrams of three modules. (a) The vanilla Transformer block [25].
(b) The GNN layer111https://pytorch-
geometric.readthedocs.io/en/latest/generated/torch_geometric.nn.conv.SAGEConv.html
with mean aggregator [32]. (c) The proposed graph-aided attention Transformer
block.

##  3 Method

###  3.1 Motivation

As mentioned in the Sec. 1, Transformer-based models have demonstrated
remarkable performance in encoding and learning representations of neural
networks when the input is in the form of cells. However, when dealing with
complete deep neural networks (DNNs) consisting of hundreds of layers, and the
depth of the input data is unknown during training, they may sometimes exhibit
poorer performance compared to GNN-based methods. Additionally, as highlighted
in [3], real-world applications often show significant differences in the
topologies and depths between training and test samples. Consequently,
representation learning models must possess strong generalization abilities
for handling unseen data. In this regard, GNN-based models appear to achieve
better performance.

This observation has prompted us to reconsider the two types of inputs, namely
cells and complete DNNs, as well as the two representation learning models,
the Transformer and GNN. Through a detailed comparative analysis of the
structures of the Transformer and GNN, we speculate that the insufficient
generalization capability of Transformer-based methods may be attributed to
its structure and computation characteristics. As we know, the self-attention
structure in transformers is a crucial design that allows for the effective
extraction of global features in a data-driven manner. However, this structure
becomes a double-edged sword when learning network representations. For input
neural networks with depths of hundreds of layers, the Transformer’s
impressive capability to capture global information can sometimes lead to
excessive sensitivity. This stems from the fact that the Transformer models
interactions between all tokens using its self-attention mechanism, treating
the entire sequence as a fully connected graph. This dense attention mechanism
can give rise to a particular issue: even a subtle variation, such as reducing
the kernel size in layer "i" from $5\times 5$ to $3\times 3$, can affect the
representation of all other layers, ultimately leading to significant
differences in the final representation. As a result of this issue, the
trained model may be biased toward fitting the training data. Consequently,
when the model is employed for inferring architectures outside the training
data distribution, it yields inferior results and demonstrates poorer
generalization performance. The corresponding experiments are presented in
Sec. 4.4.

###  3.2 Transformer grafted with GNN

Fig.1 shows the vanilla transformer block, GNN block, and our proposed graph-
aided attention Transformer block. As shown in Fig.1 (a), the vanilla
Transformer block has two major parts:

$\displaystyle\hat{H}^{l}=\mathrm{SelfAttn}(\mathrm{LN}(H^{l-1}))+H^{l-1},$
(1) $\displaystyle H^{l}=\mathrm{FFN}(\mathrm{LN}(\hat{H}^{l}))+\hat{H}^{l},$
(2)

where $H^{l}$ is the feature for the layer $l$. $\hat{H}^{l}$ is an
intermediate result. SelfAttn, FFN, and LN refer to self-attention, feed-
forward network, and layer normalization, respectively. GNN block just has one
major part, where the representation is updated following:

$\displaystyle\hat{H}^{l}=\mathrm{GraphAggre}(H^{l-1},A)+W^{l}_{r}H^{l-1},$
(3) $\displaystyle H^{l}=\mathrm{L_{2}}(\hat{H}^{l}),$ (4)

where $\mathrm{GraphAggre}(H^{l-1},A)=W_{a}^{l}(\mathrm{Norm}(A)H^{l-1})$.
$A\in\mathbb{R}^{N\times N}$ is the adjacency matrix, and $\mathrm{L}_{2}$
denotes the l2-normalization function. $W$ with different superscripts and
subscripts represents different learnable transformation matrices.

Comparing formulas (3) and (4) with formulas (1) and (2), we can observe two
major differences between the Transformer block and GNN block:

  * •

The Transformer utilizes self-attention to fuse information from a global
perspective, while the GNN uses graph aggregation to fuse neighbor information
based on the adjacency matrix.

  * •

The Transformer block includes an additional FFN (Feed-Forward Network)
component, which enhances information interaction between channels.

The advantage of self-attention lies in its data-driven structure, allowing
for flexible adjustment of information fusion weights based on the input data.
On the other hand, the advantage of GNN is that graph aggregation focuses on
the topological structure. These two advantages are not contradictory to each
other. Consequently, we have naturally come up with an idea to combine the
strengths of self-attention and graph aggregation. This approach inherits the
flexibility of self-attention while benefiting from the good generalization
capability of graph aggregation.

To implement this idea, we consider the neural network encoded as a graph,
with the operations or layers in the network treated as nodes. Assuming the
graph has $N$ nodes, the transformer layer we have designed for universal
neural network representation learning (Fig. 1 (c)) is calculated as follows:

$\displaystyle\widetilde{H}^{l}=\mathrm{TAEnhance}(H^{l-1},D),$ (5)
$\displaystyle\hat{H}^{l}=\mathrm{L_{2}}(\mathrm{GraphAttn}(\widetilde{H}^{l},A)+W_{r}^{l}\widetilde{H}^{l}),$
(6) $\displaystyle H^{l}=\mathrm{GFFN}(\mathrm{LN}(\hat{H}^{l}))+\hat{H}^{l},$
(7)

where $D\in\mathbb{R}^{N\times 1}$ is a vector that records the number of
nodes directly connected to each node. The Graph-aided Attention (GraphAttn)
module is responsible for performing attention calculations using the
properties of the graph structure to adjust global self-attention. The Type-
aware Enhancement module (TAEnhance) is utilized to further enhance the
representation. We introduce Grouped Feed Forward Network (GFFN) by
introducing group linear transformation into the original FFN. In the
following sections, we will provide a detailed introduction to each component.

##### Graph-aided attention

In the proposed graph-aided attention, we employ the adjacency matrix to
govern the attention calculation range. Moreover, the adjacency matrix
characterizes the inter-layer connection relationships within the neural
network, enabling the model to acquire topology knowledge. Hence, we define
this module as the Graph-aided Attention module:

$\displaystyle X^{l}=\mathrm{Sigmoid}(W_{q}^{l}\widetilde{H}^{l}+b_{q}^{l}),$
(8) $\displaystyle S^{l}=(X^{l}X^{lT}/\sqrt{d})\odot A,$ (9) $\displaystyle
Z^{l}=W_{a}^{l}(\mathrm{Norm}(S^{l})\widetilde{H}^{l})+b_{a}^{l}.$ (10)

The $\mathrm{Norm}(\cdot)$ means that for each node, the attention weights
between it and other nodes are transformed to (0, 1) by dividing it by the
sum. The character $b$ with different superscripts and subscripts represents
different learnable biases. The $d$ refers to feature dimension of $X^{l}$.
Note that simply using the adjacency matrix to control the attention map in
self-attention is insufficient. To make this approach work, we have discarded
the original softmax operation in self-attention and replaced it with a linear
attention mechanism. This is because the softmax operation tends to focus
excessively on the current node while neglecting neighboring nodes.
Consequently, we have inserted a sigmoid activation function before the linear
attention to ensure that all values in the attention map are positive. For
further comparisons with the original self-attention, please refer to the
supplementary.

##### Type-Aware enhancement module

The connection between a layer and other layers is related to the type of that
layer. Therefore, the number of connected layers in each layer can be used to
assist the model in learning the type of layer. By fully utilizing the
internal characteristics of this graph-structured data, it is beneficial to
improve the learned representations. The enhanced representation is obtained
by:

$\mathrm{TAEnhance}(H^{l-1},D)=\mathrm{Sigmoid}(W_{d}^{l}D+b_{d}^{l})\odot
H^{l-1}.$ (11)

###  3.3 Universal representation learning for neural network

Figure 2: Overview of attribute prediction model.

In this subsection, we will present the construction of a comprehensive
framework for neural network encoding and representation based on the proposed
enhanced Transformer block. We will also explain how this framework can be
utilized to predict network attributes. The overall system, as depicted in
Fig. 2, is composed of three consecutive stages: neural network encoding,
representation learning, and attribute prediction.

##### Neural network encoding

We have taken inspiration from the tokenizer used in NAR-Former [2] and made
certain modifications to encode the input neural network. For a given network
consisting of $N$ layers or operations, whether it is a cell architecture or a
complete DNN, we represent it as a sequence feature comprising vectors
corresponding to each layer:
$T=(t_{1},t_{2},\cdots,t_{N})\in\mathbb{R}^{N\times C}$. Each vector
encapsulates both the operation and position information:
$t_{i}=(t_{i}^{\mathrm{op}},t_{i}^{\mathrm{pos}})\in\mathbb{R}^{C}$.

Following the encoding scheme proposed in NAR-Former [2], we use the position
encoding formula [34; 13] to transform the single real-valued numbers (e.g.
operation labels and node position indices) of relevant information into a
higher-dimensional space. We denote this mapping scheme as
$f_{\mathrm{PE}}(\cdot)$.

For the node position encoding $t_{i}^{\mathrm{pos}}$, since our improved
transformer can obtain the topology information of the network with the help
of the adjacency matrix, we only encode the self-position of the node with
$f_{\mathrm{PE}}(\cdot)$. For the operation encoding $t_{i}^{\mathrm{op}}$,
there are slight differences in the specific encoding content for input
networks of different scales. If the input is in the form of a cell
architecture [1; 21], there are usually no more than ten different options for
each architecture operation. In this case, we directly assign category labels
to all possible operations, and then use the function $f_{\mathrm{PE}}(\cdot)$
to encode the label of the operation to obtain $t_{i}^{\mathrm{op}}$. However,
when a complete DNN is used as input [3], more abundant operational
information can be extracted and encoded. In this case, we first use one-hot
vectors, which ensure the same distance between different categories, to
encode the type of operation (e.g. convolution, batch normalization, ReLU,
concatenation). Then use the function $f_{\mathrm{PE}}(\cdot)$ to encode the
properties (e.g. kernel size, number of groups) of the operation, which is
then concatenated with the one-hot type vector as $t_{i}^{\mathrm{op}}$.

##### Representation learning

The model for learning neural network representations $H^{K}$ is constructed
by stacking multiple instances of our proposed enhanced Transformer blocks.
These improvements, specifically tailored for neural network data, allow the
model to learn representations that are more meaningful and exhibit enhanced
generalization capabilities.

##### Attributes predicting

Taking the representation $H^{K}$ as input, the target attribute can be
predicted by using the predicting head:

$\hat{y}=-\mathrm{logsigmoid}(\mathrm{FC(ReLU(FC(ReLU(FC(}H^{K}))))).$ (12)

Currently, among the various attributes of the network, accuracy and latency
are the two main types of predicted objects. Because they have extremely high
acquisition costs, and are the primary manifestation of network performance
and efficiency.

For latency prediction, due to the strong correlation between batch size,
memory access, parameter quantity, and FLOPs with network latency, the
encoding corresponding to these characteristics and the representation $H^{K}$
are input into the predicting head together. To train the whole latency
prediction model, the mean square error (MSE) function is adopted to measure
the difference between predicted results and the ground truths.

For accuracy prediction, in addition to MSE loss function, architecture
consistency loss (AC_loss) and sequence ranking related loss (SR_loss)
proposed by NAR-Former [2] are also used. Following NAR-Former, we employed a
hierarchical fusion strategy in accuracy prediction experiments. We use a
simplified approach, which computes the weighted sum of the outputs of each
transformer layer with adaptive weights.

Table 1: Latency prediction on NNLQP [3]. Training and test sets have the same
distribution.

Test Model |  MAPE$\downarrow$ |  Acc(10%)$\uparrow$
---|---|---
NNLP [3] | Ours |  NNLP [3] | Ours
avg / best | avg / best | avg / best | avg / best
All | 3.47% / 3.44% | 3.07% / 3.00% | 95.25% / 95.50% | 96.41% / 96.30%
AlexNet | 6.37% / 6.21% | 6.18% / 5.97% | 81.75% / 84.50% | 81.90% / 84.00%
EfficientNet | 3.04% / 2.82% | 2.34% / 2.22% | 98.00% / 97.00% | 98.50% / 100.0%
GoogleNet | 4.18% / 4.12% | 3.63% / 3.46% | 93.70% / 93.50% | 95.95% / 95.50%
MnasNet | 2.60% / 2.46% | 1.80% / 1.70% | 97.70% / 98.50% | 99.70% / 100.0%
MobileNetV2 | 2.47% / 2.37% | 1.83% / 1.72% | 99.30% / 99.50% | 99.90% / 100.0%
MobileNetV3 | 3.50% / 3.43% | 3.12% / 2.98% | 95.35% / 96.00% | 96.75% / 98.00%
NasBench201 | 1.46% / 1.31% | 1.82% / 1.18% | 100.0% / 100.0% | 100.0% / 100.0%
SqueezeNet | 4.03% / 3.97% | 3.54% / 3.34% | 93.25% / 93.00% | 95.95% / 96.50%
VGG | 3.73% / 3.63% | 3.51% / 3.29% | 95.25% / 96.50% | 95.85% / 96.00%
ResNet | 3.34% / 3.25% | 3.11% / 2.89% | 98.40% / 98.50% | 98.55% / 99.00%

##  4 Experiments

In this section, we conduct experiments on NNLQP [3], NAS-Bench-101 [1], and
NAS-Bench-201 [21] to evaluate the performance of our NAR-Former V2. A series
of ablation experiments were performed to corroborate the effectiveness of our
design details. More details about implementation and analysis will be
provided in the supplementary materials.

###  4.1 Implementation details

##### Model details

For latency experiments, the number of GraphAttn-based Transformer blocks is
set to 2, which is the same as the baseline [3]. As for accuracy predicting,
we fix the number of Transformer blocks to 6 to align with the standard
Transformer used in the baseline [2].

##### Training details

All experiments were trained using the Adam optimizer. We used a linear
learning rate decay strategy with a warm-up, in which the learning rate
uniformly increased to 0.001 during the first 10% of the training steps and
then gradually decayed to 0. The batch size was fixed at 16. Our models are
trained on a machine with a GeForce RTX 3090 GPU. To avoid randomness, each
model was trained for 12 times and the two experiments with the best and worst
indicators were discarded.

###  4.2 Latency prediction

We conduct latency prediction on the recently released NNLQP dataset [3],
which comprises 20000 complete deep learning networks and their corresponding
latencies on the target hardware. This dataset has 10 different types of
networks (referring to the first column of Tab. 1), with 2000 networks per
type. Following NNLP [3], we use Mean Absolute Percentage Error (MAPE) and
Error Bound Accuracy (Acc($\delta$)) to measure the deviations between latency
predictions and ground truths. The lower the MAPE, the higher the prediction
accuracy, while the opposite is true for Acc($\delta$).

Here, we considered two different scenarios. In the first scenario, the
training and testing sets are from the same distribution. We constructed the
training set with the first 1800 samples from each of the ten network types,
and the remaining 2000 networks were used as the testing set. The detailed
results are shown in Tab. 1. When testing with all test samples, the average
MAPE of our method is 0.4% lower than that of NNLP [3], and the average
Acc(10%) is 1.16% higher than that of NNLP. When tested on various types of
network data separately, except for the NASBench201 family, our method
consistently outperforms NNLP. This indicates that our improved transformer,
which utilizes the structural characteristics of the graph, has learned more
reasonable representations than the original GNN.

Table 2: Latency prediction on NNLQP [3]. “Test Model = AlexNet” means that
only AlexNet models are used for testing, and the data from the other 9 model
families are used for training. The best results refer to the lowest MAPE and
corresponding ACC (10%) in 10 independent experiments.

Metric | Test Model | FLOPs | FLOPs | nn-Meter | TPU | BRP |  NNLP [3] | Ours
---|---|---|---|---|---|---|---|---
+MAC | [6] | [7] |  -NAS [20] | (avg / best) | (avg / best)
MAPE$\downarrow$ | AlexNet | 44.65% | 15.45% | 7.20% | 10.55% | 31.68% | 10.64% / 9.71% | 24.28% / 18.29%
EfficientNet | 58.36% | 53.96% | 18.93% | 16.74% | 51.97% | 21.46% / 18.72% | 13.20% / 11.37%
GoogleNet | 30.76% | 32.54% | 11.71% | 8.10% | 25.48% | 13.28% / 10.90% | 6.61% / 6.15%
MnasNet | 40.31% | 35.96% | 10.69% | 11.61% | 17.26% | 12.07% / 10.86% | 7.16% / 5.93%
MobileNetV2 | 37.42% | 35.27% | 6.43% | 12.68% | 20.42% | 8.87% / 7.34% | 6.73% / 5.65%
MobileNetV3 | 64.64% | 57.13% | 35.27% | 9.97% | 58.13% | 14.57% / 13.17% | 9.06% / 8.72%
NasBench201 | 80.41% | 33.52% | 9.57% | 58.94% | 13.28% | 9.60% / 8.19% | 9.21% / 7.89%
ResNet | 21.18% | 18.91% | 15.58% | 20.05% | 15.84% | 7.54% / 7.12% | 6.80% / 6.44%
SqueezeNet | 29.89% | 23.19% | 18.69% | 24.60% | 42.55% | 9.84% / 9.52% | 7.08% / 6.56%
VGG | 69.34% | 66.63% | 19.47% | 38.73% | 30.95% | 7.60% / 7.17% | 15.40% / 14.26%
Average | 47.70% | 37.26% | 15.35% | 21.20% | 30.76% | 11.55% / 10.27% | 10.55% / 9.13%
Acc(10%)$\uparrow$ | AlexNet | 6.55% | 40.50% | 75.45% | 57.10% | 15.20% | 59.07% / 64.40% | 24.65% / 28.60%
EfficientNet | 0.05% | 0.05% | 23.40% | 17.00% | 0.10% | 25.37% / 28.80% | 44.01% / 50.20%
GoogleNet | 12.75% | 9.80% | 47.40% | 69.00% | 12.55% | 36.30% / 48.75% | 80.10% / 83.35%
MnasNet | 6.20% | 9.80% | 60.95% | 44.65% | 34.30% | 55.89% / 61.25% | 73.46% / 81.60%
MobileNetV2 | 6.90% | 8.05% | 80.75% | 33.95% | 29.05% | 63.03% / 72.50% | 78.45% / 83.80%
MobileNetV3 | 0.05% | 0.05% | 23.45% | 64.25% | 13.85% | 43.26% / 49.65% | 68.43% / 70.50%
NasBench201 | 0.00% | 10.55% | 60.65% | 2.50% | 43.45% | 60.70% / 70.60% | 63.13% / 71.70%
ResNet | 26.50% | 29.80% | 39.45% | 27.30% | 39.80% | 72.88% / 76.40% | 77.24% / 79.70%
SqueezeNet | 16.10% | 21.35% | 36.20% | 25.65% | 11.85% | 58.69% / 60.40% | 75.01% / 79.25%
VGG | 4.80% | 2.10% | 26.50% | 2.60% | 13.20% | 71.04% / 73.75% | 45.21% / 45.30%
Average | 7.99% | 13.20% | 47.42% | 34.40% | 21.34% | 54.62% / 60.65% | 62.70% / 67.40%

The second scenario has more practical application significance, that is, the
network type needed to be inferred is not seen during the training progress.
There are ten sets of experiments in this part, with each set taking one type
of network as the test set, while all samples from the other nine types of
networks are used as the training set. As shown in Tab. 2, it can be seen that
using only FLOPs and memory access information to predict latency is not
enough. Suffering from the gap between the accumulation of kernel delays and
the actual latency, kernel-based methods (TPU[7] and nn-Meter[6]) perform
worse than the GNN-based model NNLP that directly encodes and predicts the
entire network. Benefiting from considering the entire input network and
grafting GNN into the transformer, our method achieves the best MAPE and
Acc(10%) on the average indicators of 10 experimental groups. Compared with
the second-best method NNLP, the average Acc(10%) of our method has an marked
increase of 8.08%.

Table 3: Accuracy prediction on NAS-Bench-101 [1]. “SE” denotes the self-
evolution strategy proposed by TNASP [9].

Backbone | Method | Training Samples
---|---|---
0.1% | 0.1% | 1%
(424) | (424) | (4236)
Test Samples
100 | all | all
CNN |  ReNAS [10] | 0.634 | 0.657 | 0.816
LSTM |  NAO [28] | 0.704 | 0.666 | 0.775
NAO+SE | 0.732 | 0.680 | 0.787
GNN |  NP [22] | 0.710 | 0.679 | 0.769
NP + SE | 0.713 | 0.684 | 0.773
CTNAS [11] | 0.751 | - | -
Transformer |  TNASP [9] | 0.752 | 0.705 | 0.820
TNASP + SE | 0.754 | 0.722 | 0.820
NAR-Former [2] | 0.801 | 0.765 | 0.871
| NAR-Former V2 | 0.802 | 0.773 | 0.861

###  4.3 Accuracy prediction

####  4.3.1 Experiments on NAS-Bench-101

NAS-Bench-101 [1] provides 423624 different cell architectures and the
accuracies of the complete neural network constructed based on each cell on
different datasets. Following [9], 0.1% and 1% of the whole data is used as
training set and another 200 samples are used for validation. We use Kendall’s
Tau [36] to evaluate the correlation between the predicted sequence and the
real sequence, and a higher value indicates a better results. The Kendall’s
Tau is calculated on the whole dataset or 100 testing samples. We report the
average results of our predictor in 10 repeated experiments. Results are shown
in Tab. 3. When only 424 samples were available for training, our method
achieves the highest Kendall’s Tau. We achieves 0.773 when tested using the
whole testing set, which is 0.8% and 8.9% higher than the transformer-based
model [2] and GNN-based model [22], respectively. This proves that the
modifications we made to the transformer based on inspiration from GNN are
effective.

Table 4: Accuracy prediction on NAS-Bench-201 [21]. “SE” denotes the self-
evolution strategy proposed by TNASP [9].

Backbone | Model | Training Samples
---|---|---
(781) | (1563)
5% | 10%
LSTM |  NAO [28] | 0.522 | 0.526
NAO + SE | 0.529 | 0.528
GNN |  NP [22] | 0.634 | 0.646
NP + SE | 0.652 | 0.649
Transformer |  TNASP [9] | 0.689 | 0.724
TNASP + SE | 0.690 | 0.726
NAR-Former [2] | 0.849 | 0.901
| NAR-Former V2 | 0.874 | 0.888

####  4.3.2 Experiments on NAS-Bench-201

NAS-Bench-201 [21] is another cell-based dataset, which contains 15625 cell-
accuracy pairs. Following [9], 5% and 10% of the whole data is used as
training set and another 200 samples are used for validation. We use Kendall’s
Tau [36] computed on the whole dataset as the evaluation metric in this part.
Average results of our predictor of 10 runs are reported. Results are shown in
Tab. 4. The conclusion of this experiment is similar to Sec. 4.3.1. When
compared with the second-best method, a substantial improvement (2.5%) of
Kendall’s Tau can be seen in the setting of training with 781 samples.

_Compared to NAR-Former, NAR-Former V2 achieves comparable accuracy prediction
performance with fewer parameters. In latency prediction experiments, NNLP
outperforms NAR-Former by a significant margin, and NAR-Former V2 exhibits a
clear advantage over NNLP (a direct comparison experiment is provided in the
supplementary material). In summary, by incorporating the strengths of GNN,
the universal representation learning framework NAR-Former V2 is significantly
enhanced. NAR-Former V2 addresses the shortcomings of NAR-Former, which was
overly sensitive when handling complete network structures, while still
retaining the outstanding performance of NAR-Former when handling cell-
structured networks._

###  4.4 Ablation studies

Table 5: Ablation studies on NNLQP [3]. "PE" denotes position encoding.

Row | Structure | Op | Op | Graph- | GFFN | TA- | MAPE$\downarrow$ | Acc(10%)$\uparrow$ | Acc(5%)$\uparrow$
---|---|---|---|---|---|---|---|---|---
Type | Attributes | Attn | Ehance
1(Baseline) | GNN | One-hot | Real Num | - | - | - | 3.48 | 95.26 | 77.80
2 | GNN | PE | PE | - | - | - |  3.43(-0.05) |  95.11(-0.15) |  79.58(+1.78)
3 | GNN | One-hot | PE | - | - | - |  3.33(-0.15) |  95.57(+0.31) |  80.19(+2.39)
4 | Transformer | One-hot | PE | ✓ | - | - |  3.20(-0.28) |  96.00(+0.74) |  81.86(+4.06)
5 | Transformer | One-hot | PE | ✓ | ✓ | - |  3.20(-0.28) |  96.06(+0.80) |  81.76(+3.96)
6 | Transformer | One-hot | PE | ✓ | ✓ | ✓ |  3.07(-0.41) |  96.41(+1.15) |  82.71(+4.91)

Table 6: The influence of using different attentions. Test on EfficientNet.

Attention |  MAPE$\downarrow$ |  ACC(10%)$\uparrow$
---|---|---
Global | 16.88% | 36.32%
Local | 13.20% | 44.01%

In this section, we conducted a series of ablation experiments on the NNLQP
dataset to investigate the impact of various modifications. The results from
Rows (2) and (3) in Table 5 indicate that for type encoding without numerical
relationships, using one-hot vectors with equidistant properties across
different categories is more suitable. Comparing Row (3) in Table 5 with Row
(4), we observe that introducing GNN characteristics into the Transformer
improves the model’s ability to learn effective representations and achieve
more accurate predictions compared to using the original GNN. When replacing
the FFN with the GFFN module with eight groups (Row (5)), the number of model
parameters reduces to approximately one eighth of that in Row (4), without a
significant decrease in prediction accuracy. Compared to Row (5), Row (6)
demonstrates an increase of 0.35% in ACC(10%) and 0.95% in ACC(5%). This
confirms the role of the type-aware enhancement module in further refining and
enhancing the rationality of the representations.

To verify our hypothesis regarding the generalization ability of the network
and the effectiveness of the proposed graph-aided attention, we conducted
comparative experiments in scenarios where the training and testing data have
different distributions. The results of these experiments are presented in
Table 6. In order to perform the experiment on global attention, we excluded
the step of multiplying the adjacency matrix $A$ in Equation 9, and instead
replaced $S^{l}$ with $X^{l}X^{lT}/\sqrt{d}$. Results in Table 6 demonstrate
that incorporating the adjacency matrix to restrict the scope of attention
calculation is indeed beneficial for latency prediction on unseen data. The
model utilizing graph-aided attention exhibited a significant improvement of
7.68% in ACC(10%) compared to the model using global attention.

##  5 Conclusion

In this paper, we combine the strengths of Transformer and GNN to develop a
universal neural network representation learning model. This model is capable
of effectively processing models of varying scales, ranging from several
layers to hundreds of layers. Our proposed model addresses the limitations of
previous Transformer-based methods, which exhibited excessive sensitivity when
dealing with complete network structures. However, it still maintains
exceptional performance when handling cell-structured networks. In future
work, we will focus on optimizing the design of the representation learning
framework and applying it to a broader range of practical applications. Such
as using the proposed model to search for the best mixed precision model
inference strategies.

## References

  * [1] C. Ying, A. Klein, E. Christiansen, E. Real, K. Murphy, and F. Hutter, “Nas-bench-101: Towards reproducible neural architecture search,” in _International Conference on Machine Learning_. PMLR, 2019, pp. 7105–7114.
  * [2] Y. Yi, H. Zhang, W. Hu, N. Wang, and X. Wang, “Nar-former: Neural architecture representation learning towards holistic attributes prediction,” _arXiv preprint arXiv:2211.08024_ , 2022.
  * [3] L. Liu, M. Shen, R. Gong, F. Yu, and H. Yang, “Nnlqp: A multi-platform neural network latency query and prediction system with an evolving database,” in _51 International Conference on Parallel Processing - ICPP_ , ser. ICPP ’22. Association for Computing Machinery, 2022. [Online]. Available: https://doi.org/10.1145/3545008.3545051
  * [4] Z. Liu, J. Ning, Y. Cao, Y. Wei, Z. Zhang, S. Lin, and H. Hu, “Video swin transformer,” in _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 2022, pp. 3202–3211.
  * [5] Z. Liu, H. Hu, Y. Lin, Z. Yao, Z. Xie, Y. Wei, J. Ning, Y. Cao, Z. Zhang, L. Dong _et al._ , “Swin transformer v2: Scaling up capacity and resolution,” in _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 2022, pp. 12 009–12 019.
  * [6] L. L. Zhang, S. Han, J. Wei, N. Zheng, T. Cao, Y. Yang, and Y. Liu, “nn-meter: towards accurate latency prediction of deep-learning model inference on diverse edge devices,” in _Proceedings of the 19th Annual International Conference on Mobile Systems, Applications, and Services_ , 2021, pp. 81–93.
  * [7] S. Kaufman, P. Phothilimthana, Y. Zhou, C. Mendis, S. Roy, A. Sabne, and M. Burrows, “A learned performance model for tensor processing units,” _Proceedings of Machine Learning and Systems_ , vol. 3, pp. 387–400, 2021\.
  * [8] C. White, W. Neiswanger, and Y. Savani, “Bananas: Bayesian optimization with neural architectures for neural architecture search,” in _Proceedings of the AAAI Conference on Artificial Intelligence_ , vol. 35, 2021, pp. 10 293–10 301.
  * [9] S. Lu, J. Li, J. Tan, S. Yang, and J. Liu, “Tnasp: A transformer-based nas predictor with a self-evolution framework,” _Advances in Neural Information Processing Systems_ , vol. 34, pp. 15 125–15 137, 2021.
  * [10] Y. Xu, Y. Wang, K. Han, Y. Tang, S. Jui, C. Xu, and C. Xu, “Renas: Relativistic evaluation of neural architecture search,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2021, pp. 4411–4420.
  * [11] Y. Chen, Y. Guo, Q. Chen, M. Li, W. Zeng, Y. Wang, and M. Tan, “Contrastive neural architecture search with neural architecture comparators,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2021, pp. 9502–9511.
  * [12] Z. Chen, Y. Zhan, B. Yu, M. Gong, and B. Du, “Not all operations contribute equally: Hierarchical operation-adaptive predictor for neural architecture search,” in _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 2021, pp. 10 508–10 517.
  * [13] B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng, “Nerf: Representing scenes as neural radiance fields for view synthesis,” _Communications of the ACM_ , vol. 65, no. 1, pp. 99–106, 2021\.
  * [14] Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” in _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 2021, pp. 10 012–10 022.
  * [15] S. Mehta, M. Ghazvininejad, S. Iyer, L. Zettlemoyer, and H. Hajishirzi, “Delight: Deep and light-weight transformer,” _arXiv preprint arXiv:2008.00623_ , 2020.
  * [16] X. Ning, Y. Zheng, T. Zhao, Y. Wang, and H. Yang, “A generic graph-based neural architecture encoding scheme for predictor-based nas,” in _European Conference on Computer Vision_. Springer, 2020, pp. 189–204.
  * [17] C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu, “Exploring the limits of transfer learning with a unified text-to-text transformer,” _The Journal of Machine Learning Research_ , vol. 21, no. 1, pp. 5485–5551, 2020.
  * [18] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly _et al._ , “An image is worth 16x16 words: Transformers for image recognition at scale,” _arXiv preprint arXiv:2010.11929_ , 2020.
  * [19] A. Gulati, J. Qin, C.-C. Chiu, N. Parmar, Y. Zhang, J. Yu, W. Han, S. Wang, Z. Zhang, Y. Wu _et al._ , “Conformer: Convolution-augmented transformer for speech recognition,” _arXiv preprint arXiv:2005.08100_ , 2020.
  * [20] L. Dudziak, T. Chau, M. Abdelfattah, R. Lee, H. Kim, and N. Lane, “Brp-nas: Prediction-based nas using gcns,” _Advances in Neural Information Processing Systems_ , vol. 33, pp. 10 480–10 490, 2020.
  * [21] X. Dong and Y. Yang, “Nas-bench-201: Extending the scope of reproducible neural architecture search,” _arXiv preprint arXiv:2001.00326_ , 2020.
  * [22] W. Wen, H. Liu, Y. Chen, H. Li, G. Bender, and P.-J. Kindermans, “Neural predictor for neural architecture search,” in _European Conference on Computer Vision_. Springer, 2020, pp. 660–676.
  * [23] W. Li, S. Gong, and X. Zhu, “Neural graph embedding for neural architecture search,” in _Proceedings of the AAAI Conference on Artificial Intelligence_ , vol. 34, 2020, pp. 4707–4714.
  * [24] H. Shi, R. Pi, H. Xu, Z. Li, J. Kwok, and T. Zhang, “Bridging the gap between sample-based and one-shot neural architecture search with bonas,” _Advances in Neural Information Processing Systems_ , vol. 33, pp. 1808–1819, 2020.
  * [25] Q. Wang, B. Li, T. Xiao, J. Zhu, C. Li, D. F. Wong, and L. S. Chao, “Learning deep transformer models for machine translation,” _arXiv preprint arXiv:1906.01787_ , 2019.
  * [26] C. Liu, B. Zoph, M. Neumann, J. Shlens, W. Hua, L.-J. Li, L. Fei-Fei, A. Yuille, J. Huang, and K. Murphy, “Progressive neural architecture search,” in _Proceedings of the European conference on computer vision (ECCV)_ , 2018, pp. 19–34.
  * [27] R. Ying, R. He, K. Chen, P. Eksombatchai, W. L. Hamilton, and J. Leskovec, “Graph convolutional neural networks for web-scale recommender systems,” in _Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining_, 2018, pp. 974–983.
  * [28] R. Luo, F. Tian, T. Qin, E. Chen, and T.-Y. Liu, “Neural architecture optimization,” _Advances in neural information processing systems_ , vol. 31, 2018.
  * [29] J. Qiu, J. Tang, H. Ma, Y. Dong, K. Wang, and J. Tang, “Deepinf: Social influence prediction with deep learning,” in _Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining_, 2018, pp. 2110–2119.
  * [30] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” _arXiv preprint arXiv:1810.04805_ , 2018.
  * [31] T. Chen, T. Moreau, Z. Jiang, L. Zheng, E. Yan, H. Shen, M. Cowan, L. Wang, Y. Hu, L. Ceze _et al._ , “$\\{$TVM$\\}$: An automated $\\{$End-to-End$\\}$ optimizing compiler for deep learning,” in _13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18)_ , 2018, pp. 578–594.
  * [32] W. Hamilton, Z. Ying, and J. Leskovec, “Inductive representation learning on large graphs,” _Advances in neural information processing systems_ , vol. 30, 2017.
  * [33] B. Deng, J. Yan, and D. Lin, “Peephole: Predicting network performance before training,” _arXiv preprint arXiv:1712.03351_ , 2017.
  * [34] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” _Advances in neural information processing systems_ , vol. 30, 2017.
  * [35] F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner, and G. Monfardini, “The graph neural network model,” _IEEE transactions on neural networks_ , vol. 20, no. 1, pp. 61–80, 2008.
  * [36] P. K. Sen, “Estimates of the regression coefficient based on kendall’s tau,” _Journal of the American statistical association_ , vol. 63, no. 324, pp. 1379–1389, 1968.


# Assessing YOLACT++ for real time and robust instance segmentation
of medical instruments in endoscopic procedures

Juan Carlos Ángeles Cerón1, Leonardo Chang1, Gilberto Ochoa Ruiz1, Sharib Ali2
1Tecnológico de Monterrey, School of Engineering and Sciences, Av. Eugenio
Garza Sada 2501, Monterrey, N.L., México 64849, Mexico2 Institute of
Biomedical Engineering and Big Data Institute, Old Road Campus, University of
Oxford, Oxford, UKC<EMAIL_ADDRESS><EMAIL_ADDRESS>*Preprint under review for EMBC’21 following IEEE guidelines

###### Abstract

Image-based tracking of laparoscopic instruments plays a fundamental role in
computer and robotic-assisted surgeries by aiding surgeons and increasing
patient safety. Computer vision contests, such as the Robust Medical
Instrument Segmentation (ROBUST-MIS) Challenge, seek to encourage the
development of robust models for such purposes, providing large, diverse, and
high-quality datasets. To date, most of the existing models for instance
segmentation of medical instruments were based on two-stage detectors, which
provide robust results but are nowhere near to the real-time, running at 5
frames-per-second (fps) at most. However, for the method to be clinically
applicable, a real-time capability is utmost required along with high
accuracy. In this paper, we propose the addition of attention mechanisms to
the YOLACT architecture to allow real-time instance segmentation of
instruments with improved accuracy on the ROBUST-MIS dataset. Our proposed
approach achieves competitive performance compared to the winner of the 2019
ROBUST-MIS challenge in terms of robustness scores, obtaining 0.313 MI_DSC and
0.338 MI_NSD while reaching real-time performance at $\mathbf{>}$45 fps.

Keywords: Deep learning; laparoscopy; segmentation; surgical instrument, real-
time.

##  I Introduction

Computer-assisted minimally invasive surgery such as endoscopy has grown in
popularity over the past years. However, due to the nature of these
procedures, issues like limited field-of-view, extreme lighting conditions,
lack of depth information, and difficulty in manipulating operating
instruments demand strenuous amounts of effort from the surgeons [1]. Surgical
data science applications could provide physicians with context-aware
assistance during minimally invasive surgery to overcome these limitations and
increase patient safety. One of the main forms of assistance is providing
accurate tracking of medical instruments using instance segmentation methods.
These systems are expected to be a crucial component in tasks ranging from
surgical navigation, skill analysis, complication prediction, and other
computer-integrated surgery (CIS) applications [2].

Nonetheless, instrument tracking methods are often deployed in difficult
scenarios such as bleeding, over or underexposure, smoke, and reflections [3].
The net effect of these issues increases the missed detection rates in
endoscopic surveillance, hampering the adoption of AI-based tools in this
context [4]. Therefore, the development of robust techniques that can be
effectively deployed in real endoscopy interventions is very much necessary.

Endoscopic computer vision contests, such as the Robust Medical Instrument
Segmentation (ROBUST-MIS) Challenge [1] represent important and necessary
efforts to encourage the development of robust models for surgical instrument
segmentation. They integrate the developments in computer-assisted surgeries
and benchmark the generalization capabilities of the developed methods on
different clinical scenarios. Furthermore, they provide large-high-quality
datasets to overcome one of the main bottlenecks of developing robust
methodologies, which is the lack of annotated data.

Previous approaches for instance segmentation submitted to the 2019 ROBUST-MIS
challenge, were exclusively based on two-stage detectors such as Mask R-CNN
[5]. While these models exhibited good performances in terms of robustness,
they all suffered from very high inference times averaging around 5 fps,
preventing them from achieving real-time performances. Realistically, real-
time performance is mandatory in order to fully exploit the capabilities of
tracking applications in live surgeries.

In order to overcome these inference limitations while maintaining a robust
performance in terms of tool segmentation results, we propose a new approach
based on YOLACT++ [6] equipped with attention modules on the multi-scale
outputs of the CNN backbone and Feature Pyramid Network (FPN). The increased
representation power achieved by using attention allows the extraction of more
discriminant features while suppressing the less effective ones.

In this work, we evaluate the Criss-cross Attention Module (CCAM) [7]. CCAM,
which is depicted in Figure 1, recursively integrates global context across
feature maps in a fast and clever criss-cross fashion. By integrating this
attention mechanism, our proposed model outperforms previous approaches in the
state-of-the-art by a slight margin, but it attains real-time performances,
which makes our method clinically applicable both in inference time and
robustness.

Figure 1: Diagram of the criss-cross attention module. For each position in
the input feature map, the attention module generates a sparse attention map
by aggregating information in the horizontal and vertical axes. After the
second operation, each pixel has extracted context from all the pixels.

The rest of the paper is organized as follows. In Section II we discuss the
ROBUST-MIS dataset, the context of the challenge, and the official metrics to
assess robustness. Section III introduces our modifications to the YOLACT
model to incorporate attention; also, we discuss our training and validation
processes. In Section IV we present our results and we discuss areas for
further improvements. Finally, Section V concludes the article.

##  II Materials and methods

###  II-A Materials

The Heidelberg Colorectal Data Set [2] served as a basis for the ROBUST-MIS
challenge. It comprises 30 surgical procedures from three different types of
surgeries and includes detailed segmentation maps for the surgical instruments
in more than 10,000 laparoscopic video frames. The generalization and
performance of the submitted algorithms are typically assessed in three
stages, with increasing levels of difficulty:

  * •

Stage 1: Test data is taken from the procedures from which the training data
was extracted.

  * •

Stage 2: Test data is taken from the same type of surgery as the training data
but procedures not included in the training.

  * •

Stage 3: Test data is taken from a different but similar type of surgery
compared to the training data.

The detailed case distribution for each stage is presented in Table I.

TABLE I: Case distribution for each stage of the challenge [1].  Procedure | Training | Testing
---|---|---
|  | Stage 1 | Stage 2 | Stage 3
Proctocolectomy |  | 2,943
---
| 325
---
| 255
---
0
Rectal resection |  | 3,040
---
| 338
---
| 289
---
0
Sigmoid resection* | 0 | 0 | 0 |  | 2,880
---
TOTAL |  | 5,983
---
| 663
---
| 514
---
| 2,880
---

###  II-B Metrics

The two multi-instance segmentation metrics were used to assess the
performance of the models. Multiple Instance Dice Similarity Coefficient
(MI_DSC) and Multiple Instance Normalized Surface Dice (MI_NSD). The DSC [8]
is defined as the harmonic mean of precision and recall:

$DSC(Y,\hat{Y}):=\frac{2\mid Y\cap\hat{Y}\mid}{\mid Y\mid+\mid\hat{Y}\mid},$
(1)

Where $Y$ indicates the ground truth annotation and $\hat{Y}$ the
corresponding prediction of an image frame.

Unlike DSC, which measures the overlap of volumes, the NSD measures the
overlap of two mask borders [9]. The metric uses a threshold that is related
to the inter-rater variability of the annotators. According to [1], their
calculations resulted in a threshold of $\tau:=13$ for the challenge’s data
set. To calculate the MI_DSC and MI_NSD, matches of instrument instances were
computed. Then, the resulting metric scores per instrument instance per image
were aggregated by the mean.

Note that the challenge reports robustness and accuracy rankings. However, to
compute accuracy, it is mandatory to know the per image results per
participant, which are not available due to privacy issues. For this reason,
we will be reporting only robustness rankings.

The robustness rankings pay particular attention in stage 3 of the challenge
since it was built to test generalization, and focus on the worst-case
performance of methods. For this reason, MI_DSC and MI_NSD are aggregated by
the 5% percentile instead of by the mean or median [1].

###  II-C Data preprocessing

A total of 996 frames with no visible instruments were removed from the
training set, leaving 4,987 usable frames. An 85-15 percent split was made for
training and validation purposes from this subset, respectively.

Data augmentation techniques were heavily applied to introduce as much
variability as possible and increase the model’s performance. The augmentation
techniques used are random photometric distortions, random scaling, random
sample cropping, and random mirroring.

##  III Proposed model

###  III-A Architecture of the proposed model

In order to improve the robustness of the real-time YOLACT architecture used
in our proposal, we introduce attention modules on the multi-scale outputs of
the ResNet-101 backbone and the output features of the FPN (see Figure 2).
Attention enables the network to focus on the most relevant features and avoid
redundant use of information.

Our attention mechanism of choice was Criss-cross Attention Module (CCAM) [7],
specifically because of its fast, computationally efficient ($N\sqrt{N}$), and
low GPU memory usage. These characteristics are crucial in order to introduce
as little time-processing overhead as possible into the model and preserve
real-time performance.

Figure 2: Proposed YOLACT++ architecture with criss-cross attention modules in
ResNet-101 backbone + FPN. Note that certain modules are removed in some
experiments. See Section III for further details.

CCAM captures global contextual information in a similar fashion to the non-
local module [10] by generating an attention map for each pixel in the input
feature map and then transforming it into a refined feature map. However,
instead of generating attention maps for each pixel-pair which is
computationally expensive, for each pixel in a feature map, CCAM aggregates
contextual information only in its horizontal and vertical directions, as
shown in Figure 1. By consecutively stacking two criss-cross attention
modules, each pixel can collect contextual information from all pixels in a
given feature map. Next, the contextually rich feature is concatenated and
convolved with the original feature maps for feature fusion. Our experiments
consisted of systematically adding the attention modules in two strategic
locations in the network: first, in the backbone’s output features, next in
the FPN output features, and ultimately on both locations. As a result, we
came up with three models, _CCAM-Backbone_ , _CCAM-FPN_ , and _CCAM-Full_ ,
plus the attentionless baseline _Base YOLACT++_.

###  III-B Training and validation

We trained the models for up to 100,000 iterations with a learning rate of
0.001, momentum of 0.9, weight decay of $5\text{\times}{10}^{-4}$, and batch
size 16 in an NVIDIA DGX-1 system. The performance was assessed using the
evaluation code for the challenge [11]. Moreover, the rankings were computed
using the R package _challengeR_ [12].

##  IV Results and discussion

Figure 3 shows dot-and-boxplots of the metric values for each algorithm over
all test cases in stage 3 of the challenge. Among the three model variations
to which we added attention modules, CCAM-Backbone achieved the best results
in terms of robustness. This result indicates that the contextually enriched
feature maps from the ResNet-101 backbone are powerful enough to generate more
accurate mask prototypes and coefficients in the YOLACT architecture and
ultimately better segmentation outputs.

From our results, we can determine that adding attention mechanisms on the FPN
outputs only increases the model performance slightly when compared to the
baseline. However, this slight improvement becomes non-significant when
considering the strict aggregated challenge metrics.

Considering the good performance of CCAM-Backbone and the minor improvements
of CCAM-FPN, one might believe that combining both configurations in CCAM-Full
would lead to better results than having only one of them. However, as we can
observe in their respective boxplots, this is not the case. An explanation of
this behavior is that adding too many attention modules that integrate global
context might lead to an over-mixing of information and noise, commonly known
as over-smoothing, a common problem in graph neural networks from which CCAM
takes inspiration.

TABLE II: Aggregated evaluation performance for stage 3 of the challenge. The top section of the table includes the teams from the 2019 challenge. Team scores were taken from [1]. The bottom section includes our three attention models plus the baseline without attention. Team/Algorithm | MI_DSC | MI_NSD | FPS
---|---|---|---
www (Mask R-CNN) | 0.31 | 0.35 | 5*
Uniandes (Mask R-CNN) | 0.26 | 0.29 | 5*
SQUASH (Mask R-CNN) | 0.22 | 0.26 | 5*
|  CASIA_SRL (Dense
---
Pyramid Network)
0.19 | 0.27 | 5*
fisensee (2D U-Net) | 0.17 | 0.16 | 18*
caresyntax (Mask R-CNN) | 0.00 | 0.00 | 5*
VIE (Mask R-CNN) | 0.00 | 0.00 | 5*
CCAM-Backbone | 0.313 | 0.338 | 49
CCAM-Full | 0.308 | 0.333 | 45
CCAM-FPN | 0.000 | 0.000 | 60
Base YOLACT++ | 0.000 | 0.000 | 75
| *Approximated from base method.
---
Original measurement was not reported.
Figure 3: Dot-and-boxplots showing the individual performance of algorithms in
stage 3 of the challenge. Red lines indicate the aggregated metric by 5%
percentile also reported in Table II.

Next, we compare our proposed models and baseline to the top participants of
the 2019 challenge (note that the 2020 edition did not take place). Table II
shows the aggregated MI_DSC and MI_NSD values achieved for each
participant/model, as well as the network architecture they were based on and
their approximated/measured frame rate. Regarding the robustness of our
method, CCAM-Backbone achieves competitive results in both metrics with
respect to the top-performing contestant, reaching 0.313 MI_DSC and 0.338
MI_NSD compared to 0.31 and 0.35 respectively by team _www_.

It is important to note, though, that this team used data from the EndoVis
2017 challenge to improve their generalization performance on additional data.
However, our best model outperforms the second-best contestant by a
considerable margin: 0.053 MI_DSC and 0.048 MI_NSD.

An important contribution of our method is its ability to run in real-time.
Inference speed performance was tested on a 10 second video snippet from the
ROBUST-MIS dataset a total of ten times per model. The reported frame rates
were then aggregated by the mean. Inference was tested on a single Tesla P100
GPU from the DGX-1 cluster with video multi-frame enabled.

As seen in Table II, the vast majority of the submitted models were based on
Mask R-CNN, which is inevitably slow as it relies on a two-stage detector that
limits its performance to 5 fps at most. In contrast, our models comfortably
fulfill real-time operation requirements for clinically usable models, running
at $\geq 45$ fps.

Notably, our top model produces high-quality and temporally consistent masks.
Figure 4 shows some examples with varying types and number of instruments
together with their respective ground truth annotations. The model is robust
to occluded instruments and various harsh conditions, like blood, smoke, and
poor lighting. Nevertheless, it struggles with transparent instruments and
small instruments on the edge of the field of view. Figure 5 illustrates some
examples of challenging frames for our proposed algorithm, which we will seek
to address in future work.

Figure 4: Left: CCAM-Backbone evaluation results on stage 3 frames. Right:
Ground truth annotations. All images have the confidence threshold set to 0.3.
Figure 5: Examples of challenging frames, green contours indicate instrument
instances not detected by our model. From left to right and top to bottom: 1.
transparent instrument, 2. large reflection on instrument, 3. small instrument
on the edge of the field of view, and 4. multiple instruments, some partially
occluded and on the edge of the field of view.

##  V Conclusion

We presented a novel approach for multi-instance segmentation of medical
instruments based on the YOLACT architecture extended with embedded criss-
cross attention modules. The addition of attention made it possible to extract
better global context and exploit salient features leading to improved
segmentation results. Our best model yielded competitive results in terms of
robustness compared to the state-of-the-art, reaching 0.313 on area-based
metric MI_DSC and 0.338 on distance-based metric MI_NSD while attaining real-
time performance. Our experiments showed that adding attention modules to
YOLACT boosts the performance of the model and increases robustness. However,
since CCAM is based on graph neural networks, it can potentially create
feature clusters that can cause over-smoothing of the learned features. Thus,
embedding attention modules at each layer may hinder the performance resulting
from the over-mixing of global information and noise.

In future work, we plan to experiment with different types of attention
mechanisms besides CCAM. Moreover, to increase the robustness of future
models, we believe that stronger data augmentation aimed towards challenging
instances could improve the model performances.

## Acknowledgments

The authors wish to thank the AI Hub and the CIIOT at ITESM for their support
for carrying the experiments reported in this paper on their NVIDIA’s DGX
computer.

## References

  * [1] Tobias Ross _et al._ ,  Comparative validation of multi-instance instrument segmentation in endoscopy: Results of the ROBUST-MIS 2019 challenge, 2021;  Medical Image Analysis, Vol. 70, May, 2021
  * [2] Lena Maier-Hein _et al._ ,  Heidelberg Colorectal Data Set for Surgical Data Science in the Sensor Operating Room, 2020;  arXiv:2005.03501.
  * [3] Sebastian Bodenstedt _et al._ ,  Comparative evaluation of instrument segmentation and tracking methods in minimally invasive surgery, 2018;  arXiv:1805.02475.
  * [4] Sharib Ali _et al._ ,  Deep learning for detection and segmentation of artefact and disease instances in gastrointestinal endoscopy, Challenge Report  Medical Image Analysis, Vol. 70, May, 2021
  * [5] Kaiming He, Georgia Gkioxari, Piotr Dollár and Ross Girshick.  Mask R-CNN, 2017;  arXiv:1703.06870.
  * [6] Bolya, Daniel and Zhou, Chong and Xiao, Fanyi and Lee, Yong Jae.  YOLACT++: Better Real-time Instance Segmentation.  _I_ EEE Transactions on Pattern Analysis and Machine Intelligence, pp. 1–1,(2020). Institute of Electrical and Electronics Engineers (IEEE).
  * [7] Zilong Huang, Xinggang Wang, Yunchao Wei, Lichao Huang, Humphrey Shi, Wenyu Liu and Thomas S. Huang.  CCNet: Criss-Cross Attention for Semantic Segmentation, 2018;  arXiv:1811.11721.
  * [8] Lee R. Dice.  Measures of the Amount of Ecologic Association Between Species.  _E_ cology, 26(3), pp. 297–302,(1945). Ecological Society of America.
  * [9] Stanislav Nikolov _et al_ ,  Deep learning to achieve clinically applicable segmentation of head and neck anatomy for radiotherapy, 2018;  arXiv:1809.04430.
  * [10] Xiaolong Wang, Ross Girshick, Abhinav Gupta and Kaiming He.  Non-local Neural Networks, 2017;  arXiv:1711.07971.
  * [11] Tobias Roß and Annika Reinke.  RobustMIS2019, 2019.
https://phabricator.mitk.org/source/rmis2019/.

  * [12] Manuel Wiesenfarth _et al_ ,  Methods and open-source toolkit for analyzing and visualizing challenge results, 2021;  _S_ cientific Reports, 11(1), pp. 1–15,(2021). Nature Publishing Group.


# Non-Newtonian rheology in twist-bend nematic liquid crystals

E. I. Kats  Landau Institute for Theoretical Physics, RAS,
142432, Chernogolovka, Moscow region, Russia.

###### Abstract

In this work we present a simple qualitative model to describe shear
rheological behavior of the twist-bend nematic liquid crystals ($N_{TB}$). We
find that at relatively low shear rate
(${\dot{\gamma}}\leq{\dot{\gamma}}_{c1}$) the stress tensor $\sigma$ created
by this shear strain, scales as $\sigma\propto{\dot{\gamma}}^{1/2}$. Thus the
effective viscosity decreases with the shear rate
($\eta\propto{\dot{\gamma}}^{-1/2}$) manifesting so-called shear-thinning
phenomenon. At intermediate shear rate
${\dot{\gamma}}_{c1}\leq{\dot{\gamma}}\leq{\dot{\gamma}}_{c2}$, $\sigma$ is
almost independent of ${\dot{\gamma}}$ (a sort of plateau), and at large shear
rate (${\dot{\gamma}}\geq{\dot{\gamma}}_{c2}$), $\sigma\propto{\dot{\gamma}}$,
and it looks like as Newtonian rheology. Within our theory the critical values
of the shear rate scales as
${\dot{\gamma}}_{c1}\propto({\tilde{\eta}_{2}^{0}}/{\tilde{\eta}_{3}^{0}})^{2}$,
and
${\dot{\gamma}}_{c2}\propto({\tilde{\eta}_{2}^{0}}/{\tilde{\eta}_{3}^{0}})^{4}$
respectively. Here ${\tilde{\eta}}_{2}^{0}$ and ${\tilde{\eta}}_{3}^{0}$ are
bare coarse grained shear viscosity coefficients of the effective smectics
equivalent to the $N_{TB}$ phase at large scales. The results of our work are
in the agreement with recent experimental studies.

Background.

A number of exciting and relatively recent publications (about ten years ago,
compared to more than 100 years of the discovery of the classical liquid
crystals) report on a discovery of a new type of equilibrium liquid crystals,
termed twist-bend nematics, $N_{TB}$ (see the papers HS09 ; PN10 ; CD11 ; BK13
; MD14 ; SV20 ). The discovery of $N_{TB}$ nematics opened ”Pandora box” with
new kinds of modulated liquid crystals (see very influential pioneering works
and a few review papers KA12  \- SC20 ). Naturally (as it was the case in
great geographical discoveries of 15-th - 17-th centuries) after the first
step devoted mainly to observations and structural identifications of new
liquid crystals, the interest moves to investigations and exploring of
physical properties of these new phases. Since then the $N_{TB}$, and other
modulated nematics are becoming one of the hottest topics in physics of liquid
crystals.

To start a few words about structural features of $N_{TB}$ nematics is in
order here. Especially surprising is the fact that the twist-bend nematics,
$N_{TB}$) exhibit helical (chiral) orientational ordering despite being formed
from achiral molecules. For comparison, there exist also chiral cholesteric
phases locally equivalent to nematics but possessing simple (orthogonal)
helical structures with pitches in a few $\mu m$ range. The cholesteric
structure appears as a result of relatively weak molecular chirality (that is
why it has a relatively large pitch), and the swirl direction of the spiral
(left or right) is determined by the sign of the molecular chirality. Unlike
this situation, the $N_{TB}$ nematics are formed as a result of spontaneous
chirality breaking, they have nanoscale (a few nanometers) pitch. This fact
suggests that description of the twist-bend nematics in terms of an
orientational elastic energy needs a modification related to relatively short
pitch of the helicoidal structure. In the case the Frank moduli for the short-
scale component of the director field are different from those for the long-
scale component of the director. Therefore the components should be treated in
terms of different elastic energies. We keep the notation $\bm{n}$ for the
long-scale component of the director (nematic director in what follows) and
introducing its short-scale component $\bm{\varphi}$. The components have to
be orthogonal, $\bm{n}\cdot\bm{\varphi}=0$. Thus the vector $\bm{\varphi}$ has
two independent components. The quantity $\bm{n}+\bm{\varphi}$ can be
naturally termed as the $N_{TB}$ phase director. It has the helical conic
structure in space. By other words, the short-scale component $\bm{\varphi}$
rotates around $\bm{n}$ at moving along the $\bm{n}$-direction. Therefore the
absolute value of the vector $\bm{\varphi}$ gives the tilt angle $\theta$ for
the conical spiral. Because the conical helical structure has a certain short
pitch periodicity, it is characterized by the wave vector $q_{0}$,
experimentally on the order of a few inverse molecular length.

Our paper is motivated by two very recent works KK20  \- KK21  on rheological
studies of the $N_{TB}$ liquid crystals. The authors of these papers found
nontrivial non-Newtonian behavior of sheared $N_{TB}$ nematics. At relatively
low shear rate (${\dot{\gamma}}\leq{\dot{\gamma}}_{c1}$) the stress tensor
$\sigma$ created by this shear strain, scales as
$\sigma\propto{\dot{\gamma}}^{1/2}$. Thus the effective viscosity decreases
with the shear rate ($\eta\propto{\dot{\gamma}}^{-1/2}$) manifesting so-called
shear-thinning phenomenon. At intermediate shear rate
${\dot{\gamma}}_{c1}\leq{\dot{\gamma}}\leq{\dot{\gamma}}_{c2}$, $\sigma$ is
almost independent of ${\dot{\gamma}}$ (a sort of plateau), and at large shear
rate (${\dot{\gamma}}\geq{\dot{\gamma}}_{c2}$, $\sigma\propto{\dot{\gamma}}$,
and it looks like Newtonian rheology. The critical values of the shear rate
(${\dot{\gamma}}_{c1}\,,\,{\dot{\gamma}}_{c2}$) indicating transitions between
dynamical regimes depend on temperature. Above certain temperature $T^{*}$
(below $N\,-\,N_{TB}$ phase transition point $T_{c}$, where $N$ stands for
conventional nematic state) the behavior becomes pure Newtonian. The aim of
this paper is to present theoretical rationalization for the observed in these
works KK20 , KK21  results. In what follows we integrate the input from recent
works and discussions, however my own contribution to this field will be also
presented.

Although the origin of the observed experimentally rheological behavior of the
$N_{TB}$ nematics still remains to be clarified, the main message of this work
is robust. Namely we claim that coarse grained dynamic description of the
$N_{TB}$ phase is allows to rationalize qualitatively the observed in the
phase different rheological regimes. Such coarse grained description
supplemented by arguments based on the maximum rate of entropy production
principle, can be used to estimate the critical values of the shear rate
separating these rheological regimes. The estimations made in the paper are
applicable to the shearing of well ordered samples of the $N_{TB}$ phase. If
it is not the case, defects (disclinations or domain walls) affect the
rheology. However relying on the maximum entropy production rate principle, we
expect that the rheological transitions (crossover between different
rheological regimes) are determined by the coarse grained effective viscosity
coefficients. The latter ones should be determined for the partially
disordered (i.e., including defects) $N_{TB}$ phase flow.

Basic derivation of the $N_{TB}$ coarse-grained theory. The main feature which
distinguishes the standard nematic $N$ and the twist-bend nematic $N_{TB}$
liquid crystals is a short wavelength modulation of the orientation order
${\bm{\varphi}}$ presented in the $N_{TB}$ phase. This two component vector
${\bm{\varphi}}$, orthogonal to the nematic director ${\bf n}$
(${\bm{\varphi}}\cdot{\bf n}=0$) can be chosen as the order parameter
describing $N$ \- $N_{TB}$ phase transition. With this vector order parameter
in hands one can write the Landau free energy functional.

Taking into account the nature of the short-scale vector field $\bm{\varphi}$,
we obtain (see more details in KL14 , KA17 )

$\displaystyle\int
dV\left\\{\frac{a}{2}\bm{\varphi}^{2}+\frac{b_{3}}{8q_{0}^{2}}\left[\left(n_{i}n_{k}\partial_{i}\partial_{k}+q_{0}^{2}\right)\bm{\varphi}\right]^{2}+\frac{b_{1}}{2}(\nabla\bm{\varphi})^{2}\right.$
$\displaystyle\left.+\frac{b_{\perp}}{2}\delta^{\perp}_{ij}\partial_{i}\bm{\varphi}\partial_{j}\bm{\varphi}+\frac{\lambda}{24}\varphi^{4}-\frac{\lambda_{1}}{16q_{0}^{2}}\left(\epsilon_{ijk}\varphi_{i}\partial_{j}\varphi_{k}\right)^{2}\right\\},\quad$
(1)

where $\delta^{\perp}_{ij}=\delta_{ij}-n_{i}n_{j}$. As usual, $a\propto
T-T_{c}$, where $T_{c}$ is the mean field transition temperature. The
quantities $b$ are analogs of the Frank moduli for the order parameter
$\bm{\varphi}$. The free energy (1) represents the minimal Landau model for
the $N$–$N_{TB}$ phase transition, neglecting fluctuations of the long-scale
director $\bm{n}$, which anyway is suppressed under shear (see e.g., GP93  \-
OP06 ). Then denoting a preferred direction, $\bm{n}_{0}=(0,0,1)$ we represent
the Landau functional (1) in a more compact form by replacing the order
parameter $\bm{\varphi}$ by its complex long-wavelength (!) counterpart
$\bm{\psi}$

$\bm{\varphi}=2\,\mathrm{Re}\ \left[\bm{\psi}\exp(iq_{0}z)\right].$ (2)

If $\lambda_{1}>0$ then below the phase transition (at $a<0$) minimization of
the free energy leads to the conical helical structure with

$\varphi_{x}=2|\psi_{x}|\cos(q_{0}z+\theta),\quad\varphi_{y}=\pm
2|\psi_{x}|\sin(q_{0}z+\theta),$ (3)

where $\theta$ is the phase of $\psi_{x}$ and signs $\pm$ correspond to two
possible rotation directions of the conical structure.

It is worthwhile to mention the known in the literature other coarse grained
descriptions of the $N_{TB}$ phase SK14 , PS16 , MD16  (I sincerely thank the
anonymous reviewer for attracting my attention to these papers). Although the
approaches SK14 , PS16 , MD16  are conceptually similar to the model presented
in this work, there is an essential difference. In our approach the long
wavelength (nematic director) and short wavelength the $N_{TB}$ order
parameter are explicitly separated into the Landau free energy expansion (1).
Technically the short wavelength nature of the $N_{TB}$ order parameter yields
to the specific form of the gradient terms, providing the softening of the
order parameter in the vicinity of a circle in the reciprocal space (not
around a single point (zero wave vector) as it is the case for the nematic
order parameter). In the $N_{TB}$ phase but not too far from the transition
point $T_{c}$ (that is always the case due to $(T-T_{c})/T_{c}\ll 1$) the
entering Landau theory (1) parameters $b_{3}$, $b_{1}$, $b_{\perp}$,
$\lambda$, $\lambda_{1}$ and $q_{0}$ can be considered in the mean-field
approximation as temperature independent, and only the controlling mean-field
behavior coefficient $a$ scales as $T-T_{c}$. Soft near the circle $|{\bf
q}|=q_{0}$ in the reciprocal space fluctuations of the $N_{TB}$ order
parameter changes this mean-field behavior (see more details below and also in
KL14 ).

Since at the scales $r$ larger than $q_{0}^{-1}$ we are interested in this
work, the $N_{TB}$ phase free energy and dynamic equations coincide with those
for the effective smectic $A$, in what follows we will use interchangeably the
both terms. $N_{TB}$ nematics when discussing generic features of the $N\to
N_{TB}$ phase transition, and effective smectic $A$, speaking about rheology.

Fluctuations around this mean-field state include long wavelength fluctuations

${\bf n}={\bf n}_{0}+\delta{\bf n},$ (4)

and short wavelength fluctuations ${\bm{\varphi}}$ which is orthogonal to
${\bf n}$

${\bm{\varphi}_{\perp}}=2Re\left[\bm{\psi}e^{iq_{0}z}\right]\,;\,{\bm{\varphi}_{||}}=-2\delta{\bf
n}Re\left[\bm{\psi}e^{iq_{0}z}\right].$ (5)

Two remarks are in order here. First, as it was shown previously (BR92  -MF02
externally imposed shear flow suppresses long wavelength fluctuations. Second,
renormalization group flow draws the system towards a symmetric state WK74 ,
therefore, the renormalized value of $b_{1}$ coefficient tends to zero. Then,
the proposed (1) model is equivalent to the standard De Gennes model GP93
describing nematic - smectic $A$ phase transition, GP93 , KL93 -CL00 . It is
worth to noting, that there is no density modulation in the liquid-like
$N_{TB}$ phase. However, conical $N_{TB}$ orientation order, on scales much
larger than the orientation modulation period looks like as a periodic in
space layered smectic-like structure.

A wide range of coarse-grained models have been proposed, usually dedicated to
modeling of multiscale systems. Coarse graining allows to decrease a number of
essential degrees of freedom at the expense of microscopic details (e.g,
replacing individual building blocks by their groups). Technically in this
work we use slightly modified to be applicable to the $N_{TB}$ phase coarse-
graining procedure described in GP93  for the static behavior of cholesterics,
and in KL86  \- for dynamics of cholesteric liquid crystals. The properties of
the deformed $N_{TB}$ liquid crystal depends essentially on the ratio of the
inhomogeneity scale (say $r_{in}$) and the $N_{TB}$ heliconical spiral pitch
$q_{0}^{-1}$. At scales $r_{in}\gg q_{0}^{-1}$ to find large-scale static and
dynamic characteristics, we have to eliminate fast degrees of freedom. It is
done GP93 , KL86  by appropriate integration out the fast degrees of freedom.
Performing the averaging we express the coarse grained coefficients (elastic
moduli and viscosity coefficients) in terms of the bare $N_{TB}$ parameters
defined at scales $\ll q_{0}^{-1}$.

The elastic energy of this coarse grained phase reads as

${\tilde{F}}=\frac{1}{2}\int d^{3}r\left[{\tilde{B}}\left(\frac{\partial
u}{\partial
z}\right)^{2}+{\tilde{K}}\left(\nabla_{\perp}^{2}u\right)^{2}\right].$ (6)

Here $u$ is an orientation layer displacement along the normal to the layer
($z$ axis in our notation), and coarse-grained elastic moduli GP93 , KL93  are

${\tilde{B}}=b_{\perp}q_{0}^{2}\,;\,{\tilde{K}}=\frac{3}{8}b_{\perp}.$ (7)

Coarse-grained rheology of the $N_{TB}$ phase.

Figure 1: Two rheological configurations considered in the paper.

Since we are interested in rheological behavior (dynamics) the equilibrium
elastic energy (6) should be supplemented by the coarse grained viscous
(dissipative) stress tensor (see its derivation in KL86 , KL93 ). In what
follows we consider two main rheological configurations (see Figure 1) for the
effective smectic, representing $N_{TB}$ nematic at large scales. Namely,
perpendicular configuration (where the normal to the layer, unit vector ${\bf
l}$, perpendicular to the imposed shear velocity circulation $curl{\bf v}$)
and parallel configuration, where ${\bf l}$ parallel to $curl{\bf v}$.

It is worth to note that in the both configurations (${\bf l}$ parallel to
$curl{\bf v}$, or ${\bf l}$, perpendicular to the $curl{\bf v}$) the shear
velocity ${\bf v}$ is always oriented along the layers of the effective
smectic.

Coarse grained (smectic-like 5 viscosity coefficients [37] are expressed in
terms of the bare (nematic-like) According to the definitions [27], [35] the
uniaxial nematic phase $N$ is characterized 5 independent viscosity
coefficients $\eta_{1}\,-\,\eta_{5}$ (or 6 Leslie’s viscosity coefficients,
which have to satisfied one constraint GP93 ). At small space scales
($r<q_{0}^{-1}$) these viscosity coefficients are certainly different in the
$N$ and $N_{TB}$ phases. However the difference is relatively small (on the
order of $\bm{\varphi}^{2}$), and we neglect the difference in what follows.
What is relevant for us is the fact that for two rheological configurations
(see figure 1) we are interested in this work, only two combinations of the
five viscosity coefficients needed for a full description of dynamical
behavior. The flow is determined by these two combinations of the coarse
grained viscosity coefficients. Namely, ${\tilde{\eta}}_{2}$ coefficient
determines the shear flow rheology for the perpendicular configuration, and
the coarse grained viscosity ${\tilde{\eta}}_{3}$ determines the flow in the
parallel configuration. It is worth to stress that coarse grained free energy
of the $N_{TB}$ phase coincides with the smectic $A$ free energy, provided the
smectic layer displacement $u$ is replaced by the phase of the $N_{TB}$ order
parameter. Similar statements holds for the coarse grained dynamics of the
$N_{TB}$ phase. However we have to keep in mind that the coarse grained
viscosity coefficients have a physical meaning only at the scales $r$ larger
than $q_{0}^{-1}$. At smaller scales rheological behavior is determined by the
bare viscosity coefficients. Therefore when the shearing liquid crystal sample
thickness becomes smaller than $q_{0}^{-1}$ the coarse grained approximation
is meaningless. In particularly no any room for the coarse grained theory in
the $N$ phase, where the $N_{TB}$ order parameter $|\bm{\varphi}|\to 0$ and
hence $q_{0}\equiv 0$.

In the rheological configurations shown in figure 1 these two coarse grained
viscosity coefficients can be expressed KL86  in terms of the bare $N_{TB}$
viscosity coefficients ($\eta_{i}$, with $i=1,...5$) as

${\tilde{\eta}}_{2}=2\eta_{2}-\frac{1}{2}\eta_{3}+\frac{1}{4}\eta_{1}\,;\,{\tilde{\eta}}_{3}=\frac{1}{4}\left[2\eta_{3}+2\eta_{2}-\eta_{1}\right].$
(8)

For typical in nematic liquid crystals values of the bare viscosity
coefficients GP93 , KL06 , OP06 , we estimate from (8) the values of the
needed for us coarse grained bare viscosity coefficients:

${\tilde{\eta}}_{2}^{0}\simeq 0.832\,Poise\,;\,{\tilde{\eta}}_{3}^{0}\simeq
0.2\,Poise.$ (9)

Because in the limit ${\dot{\gamma}}\to 0$,
${\tilde{\eta}}_{2}>{\tilde{\eta}}_{3}$ just the perpendicular configuration
is preferable one (the configuration leads to the maximum rate for the entropy
production).

This principle (the maximum entropy production rate) as well as the apparently
controversial statement (the principle of minimum entropy production
principle) in each particular condition should be reconsidered, based on
statistical mechanics, and hydrodynamics to find a stationary state for non-
equilibrium systems. In this work we rely on the mapping of coarse grained
dynamics of the $N_{TB}$ phase into that for the smectic $A$ liquid crystals.
With this mapping in hands the three regimes observed in the sheared $N_{TB}$
liquid crystals can be rationalized similarly to the known for smectic $A$
results (see e.g., two review articles SB10 , FK14 ), and the results suggest
three rheological regimes which are governed by the maximum entropy production
rate (irrespective to physical mechanisms behind).

To find the rheological relation for such configuration, and then the
rheological phase diagram, one has to analyze the effective smectic order
parameter fluctuations, and how the fluctuations are affected by the imposed
shear ${\dot{\gamma}}$. Our approach is more intuitive and qualitative than
just solving the dynamical equations with imposed shear BR92  \- MF02 .
However we expect that it will be useful, offering a deeper insight into
physics behind the $N_{TB}$ phase rheology.

Our theory predictions are based on the following observations. The shear
viscosity in the perpendicular configuration ${\tilde{\eta}}_{2}$ scales
proportional to the correlation length OP06

${\tilde{\eta}}_{2}\propto\xi.$ (10)

To proceed further with the rheological relation we have to find how $\xi$
scales with the shear rate ${\dot{\gamma}}$. According to the Landau -
Brazovskii weak crystallization theory (see the original paper BR75 , on
thermodynamics and review article KLM93 , which includes also dynamics of the
weak-crystallization transition), the correlation length scales as

$\xi\propto\Delta^{1/2},$ (11)

where $\Delta$ is the gap in the soft mode, describing weak first order
transition with emerging one, two, or three dimensional translational
ordering. Said above leads to the following static correlation function for
the order parameter BR75

$\frac{T}{\Delta+\alpha(q-q_{0})^{2}},$ (12)

where $\alpha$ is a phenomenological coefficient translated into the elastic
modulus below phase transition point. This form (12) of the correlation
function is a specific feature of the weak crystallization transitions, where
the critical mode softening occurs at the finite wave vector $q_{0}$. In
dynamics KLM93  the dynamic response function corresponding to the presented
above (12) correlation function reads as

$\frac{T}{\omega-i[\Delta+\alpha(q-q_{0})^{2}]}.$ (13)

We see from (13) that $\Delta$ scales as the frequency $\omega$, and therefore
in the shear rheology condition as the shear rate ${\dot{\gamma}}$. Finally
combining everything together (11) - (13) we conclude that in a low shear rate

$\tilde{\eta}_{2}\propto({\dot{\gamma}})^{-1/2}$ (14)

in the agreement with experimental data KK20  \- KK21 . Now we are in the
position to estimate the first threshold value ${\dot{\gamma}}_{c1}$ for the
shear rate. This threshold corresponds to the condition when
${\tilde{\eta}}_{2}$ becomes on the order of the bare value of
${\tilde{\eta}}_{3}^{0}$:

${\dot{\gamma}}_{c1}\propto\left(\frac{\tilde{\eta}_{2}^{0}}{\tilde{\eta}_{3}^{0}}\right)^{2}$
(15)

At the shear rate larger than the threshold
${\dot{\gamma}}>{\dot{\gamma}}_{c1}$ a part of the $N_{TB}$ sample converts
into the parallel configuration. Then, similar to the famous Van-der-Waals -
Maxwell construction LL80 , HU87 , the rheological curve corresponds to a sort
of plateau. The plateau holds until when the entire volume of the $N_{TB}$
phase transforms into the parallel configuration. In the parallel
configuration according to OP06  ${\tilde{\eta}}_{3}\propto\xi^{5/2}$, and
therefore the weak crystallization theory KLM93  predicts
${\tilde{\eta}}_{3}\propto({\dot{\gamma}})^{-5/4}$. With this scaling in hands
we estimate the second shear rate threshold ${\dot{\gamma}}_{c2}$ as

${\dot{\gamma}}_{c2}\propto\left(\frac{\tilde{\eta}_{2}^{0}}{\tilde{\eta}_{3}^{0}}\right)^{4}.$
(16)

Finally when the $N_{TB}$ sample transforms completely into the parallel
configuration (i.e., at ${\dot{\gamma}}>{\dot{\gamma}}_{c2}$) we face to the
situation discussed first long ago by De Gennes GE76  (see also more details
concerning smectics in the works BR92  \- MF02 ). Namely, a large shear rate
${\dot{\gamma}}>{\dot{\gamma}}_{c2}$ suppresses the order parameter
fluctuations. Then we expect ${\tilde{\eta}}_{3}$ is shear rate independent
quantity and we have the Newtonian-like rheology. Thus our simple model
qualitatively explains all three rheological regimes observed experimentally
KK20  \- KK21  in the $N_{TB}$ phase.

Outlook and Conclusions.

Recent progress in rheology of the $N_{TB}$ liquid crystals has led to a
number of new and exciting experimental results KK20 , KK21 . In the paper we
propose a simple heuristic approach to rationalize these new experimental
data.

The approach is based on the coarse grained dynamic description of the
$N_{TB}$ phase, valid for the space scales larger than the pitch of the
$N_{TB}$ heliconical structure. Such description supplemented by the semi-
qualitative arguments, relayed on the maximum entropy production rate, is used
to estimate critical shear rate magnitudes separating different rheological
regimes. The corresponding thresholds are determined by the coarse grained
viscosities of the system. In own turn the coarse grained parameters entering
the theory are expressed in terms of local quantities of the bare nematic
liquid crystal. The latter characteristics could be measured by X-ray, NMR or
other basically local probes.

The key starting point of our approach is based on a simple observation that
the anisotropic viscous properties of the liquid crystals introduce a host of
novel phenomena in rheology. We find that at relatively low shear rate
(${\dot{\gamma}}\leq{\dot{\gamma}}_{c1}$) the stress tensor $\sigma$ created
by this shear strain, scales as $\sigma\propto{\dot{\gamma}}^{1/2}$. Thus the
effective viscosity decreases with the shear rate
($\eta\propto{\dot{\gamma}}^{-1/2}$) manifesting so-called shear-thinning
phenomenon. At intermediate shear rate
${\dot{\gamma}}_{c1}\leq{\dot{\gamma}}\leq{\dot{\gamma}}_{c2}$, $\sigma$ is
almost independent of ${\dot{\gamma}}$ (a sort of plateau), and at large shear
rate (${\dot{\gamma}}\geq{\dot{\gamma}}_{c2}$), $\sigma\propto{\dot{\gamma}}$,
and it looks like Newtonian rheology. Within our theory the critical values of
the shear rate scales as
${\dot{\gamma}}_{c1}\propto({\tilde{\eta}_{2}^{0}}/{\tilde{\eta}_{3}^{0}})^{2}$,
and
${\dot{\gamma}}_{c2}\propto({\tilde{\eta}_{2}^{0}}/{\tilde{\eta}_{3}^{0}})^{4}$
respectively. Here ${\tilde{\eta}}_{2}^{0}$ and ${\tilde{\eta}}_{3}^{0}$ are
bare coarse grained shear viscosity coefficients of the effective smectics
equivalent to the $N_{TB}$ phase at large scales. Our mainly qualitative
theory may not have the right numbers for the dynamic shear rate thresholds.
However theory predicts the right scaling laws observed in the experiments.
Our consideration suggests that the described phenomena and mechanisms can
bring about different rheological scenarios worthy of further studies. In this
work we have only scratched the surface of this reach subject, focusing only
on the most simple questions, which can be answered by calculations ”on a back
of the envelope”.

The main message of this work is that coarse grained dynamic description of
the $N_{TB}$ phase allows to rationalized observed in the phase different
rheological regimes. Such coarse grained description supplemented by
qualitative arguments based on the maximum rate of entropy production
principle, can be used to estimate the critical values of the shear rate
separating these rheological regimes. The estimations made in the paper are
applicable to the shearing of well ordered samples of the $N_{TB}$ phase. If
it is not the case, defects (disclinations or domain walls) affect the
rheology. However relying on the maximum entropy production rate principle, we
expect that the rheological transitions (crossover between different
rheological regimes) are determined by the coarse grained effective viscosity
coefficients. The latter ones should be determined for the partially
disordered (i.e., including defects) $N_{TB}$ phase flow. Guided by the modest
aim of this work, we present only scaling analysis of three regimes of a
steady shear viscosity curve which is in qualitative agreement with previously
reported observations KK20 , KK21 , PS16 . Even more not all of the observed
qualitative features of the rheological curves are reproduced in our approach.
For example there is no independent of shear rate plateau in the interval
${\dot{\gamma}}_{c1}\leq{\dot{\gamma}}\leq{\dot{\gamma}}_{c2}$. Instead of
that $\sigma\propto{\dot{\gamma}}^{\alpha}$, where the exponent $\alpha\simeq
0.1$ is small but not zero. The matter is that the estimated thresholds of
rheological curves assume a well-defined orientation of the director
respective to the shear flow. Most probably it is not the case in mentioned
above experimental works, especially in the regions of relatively small shear
rates. However, the ordered, defects-free states can be achieved, e.g., by
orienting liquid crystal external field. Such investigation is beyond the
scope of this work, direct experimental measurements of the ordered $N_{TB}$
phase rheology is still a challenging task. I hope that this paper will
stimulate discussions on the intriguing and important issues of non-Newtonian
rheology in liquid crystals.

###### Acknowledgements.

I am dedicating this work to outstanding theoretical physicist and remarkable
person Mark Azbel (12.05.1932 - 31.03.2020). For me personally it was a
rewarding pleasure to start my work at the Landau Institute in Chernogolovka
at the time when M.Azbel was amongst the permanent staff of the Institute. At
that time my understanding of theoretical physics, benefited tremendously from
listening discussions of the ”grands”. One of the sad consequences of the very
fast developments of physics is that it looses its unity. Mark Azbel was one
of a few Landau Institute scientists who worked in a very broad field,
including soft matter physics, the subject of this manuscript.

## References

  * (1) L.E. Hough, M. Spannuth, M.Nakata, et al., Science, 325, 452 (2009).
  * (2) V.P.Panov, M.Nagaraj, J.K.Vij, et al., Phys. Rev. Lett., 105, 167801 (2010).
  * (3) M.Cestari, S.Diez-Berart, D.A.Dunmur, et al., Phys. Rev. E, 84, 031704 (2011).
  * (4) V.Borshch, Y.K.Kim, J.Xiang, et al., Nature Commun., 4, 2635 (2013).
  * (5) J.Mandle, E.J.Davis, S.A.Lobato, et al., Phys. Chem. Chem. Phys., 16, 6907 (2014).
  * (6) E.T.Samulski, A.G.Vanakaras, D.J.Photinos, arXiv: 2009.11399 (2020).
  * (7) S.Kaur, J.Addis, C.Greco, et al., Phys.Rev. E, 86, 041703 (2012).
  * (8) K.Adlem, M.Copic, G.R.Luckhurst, et al., Phys. Rev. E, 88, 022503 (2013).
  * (9) C.Meyer, G.R.Lukhurst, I.Dozov, Phys. Rev. Lett., 111, 067801 (2013).
  * (10) S.M.Shamid, S.Dhakal, J.V.Selinger, Phys. Rev. E, 87, 052503 (2013).
  * (11) N.Vanpotic, M.Cepic, M.A.Osipov, E.Gorecka, Phys. Rev. E, 89, 030501(R) (2014).
  * (12) E.G.Virga, Phys. Rev. E, 89, 052502 (2014).
  * (13) A. Mertelj, L. Cmok, N. Sebastian, R. J. Mandle, R. R. Parker, A. C. Whitwood, J. W. Goodby, M. Copic, Phys. Rev. X, 8, 041025 (2018).
  * (14) M. Chiappini,T.Drwenski, R.van Roij, M.Dijkstra, Phys. Rev. Lett., 123, 068001 (2019).
  * (15) P. L. M. Connor, R. J. Mandle, Soft Matter 16, 324 (2020).
  * (16) I.Dozov, G.R.Luckhurst, Liquid Cryst., 47, 2098 (2020).
  * (17) Xi Chen, E.Korblova, D.Dong, X.Wei, R.Shao, L.Radzihovsky, M.Glaser, J.E.Maclennan, D.Bedrov, D.M.Walba, N.A.Clark, Proc. Natl. Acad. Sci. U.S.A., 117, 14021 (2020).
  * (18) O.D.Lavrentovich, Proc. Natl. Acad. Sci. U.S.A., 117, 14629 (2020).
  * (19) N.Sebastian, L.Cmok, R.J.Mandle, M.R. de la Fuente, I.D.Olenik, M.Copic, A.Mertelj, Phys.Rev.Lett., 124, 037801 (2020).
  * (20) M.P.Kumar, P.Kula, S.Dhara, Phys. Rev. Materials 4, 115601 (2020).
  * (21) M.Praveen Kumar, J.Karcz, P.Kula, S.Dhara, Phys. Rev. Materials, 5, 115605 (2021).
  * (22) S.M.Salili, C.Kim, S.Sprunt, J.T.Gleeson, O.Parric, A.Jakli, RSC Adv., 4, 57419 (2014).
  * (23) Z.Parsouzi, S.M.Shamid, V.Borshch, P.K.Challa, A.R. Baldwin, M.G. Tamba, C.Welch, G.H.Mehl, J.T.Gleeson, A.Jakli, O.D.Lavrentovich, D.W.Allender, J.V. Selinger, S.Sprunt, Phys. Rev., X 6, 021041 (2016).
  * (24) C.Meyer, I.Dozov, Soft Matter, 12, 574 (2016).
  * (25) E.I. Kats, V.V. Lebedev, JETP Lett., 100, 110 (2014).
  * (26) E.I.Kats, Low Temperature Physics (Fizika Nizkikh Temperatur), 43, 7 (2017).
  * (27) P. G. de Gennes, J. Prost, The physics of liquid crystals, Clarendon Press, Oxford (1993).
  * (28) M.Kleman, O.Lavrentovich Soft Matter Physics: An Introduction, Springer, Berlin (2003).
  * (29) P.Oswald, P.Pieranski, Smectics and columnar liquid crystals, Taylor and Francis, New York (2006).
  * (30) R.Bruinsma, Y.Rabin, Phys. Rev. A, 45, 994 (1992).
  * (31) G.H.Fredrickson, J.Rheology, 38, 1045 (1994).
  * (32) M.Goulian, S.T.Milner, Phys. Rev. Lett., 74, 1775 (1995).
  * (33) A.N.Morozov, J.G.E.M. Fraaije, Phys. Rev. E, 65, 031803 (2002).
  * (34) K.G.Wilson and J.Kogut, Phys. Rep. 12, 75 (1974).
  * (35) E.I.Kats and V.V.Lebedev, Fluctuational Effects in the Dynamics of Liquid Crystals, Springer, 1993.
  * (36) P.M.Chaikin, T.C.Lubensky, Principles of condensed matter physics, Cambridge University Press, Cambridge (2000).
  * (37) E.I.Kats, V.V.Lebedev, JETP, 64, 518 (1986) [ZhETF, 91, 871 (1986)].
  * (38) D.Svensek, H.R.Brand, Advances in Polymer Science, 227, 101 (2010).
  * (39) S.Fujii, S.Komura, Ch.-Yi D.Lu, Materials, 7, 5146 (2014).
  * (40) S.A. Brazovskii, JETP 41, 85 (1975), [ZhETF, 68, 175].
  * (41) E.I. Kats, V.V. Lebedev, A.R. Muratov, Physics Reports 228, 1 (1993).
  * (42) L.D.Landau, E.M.Lifshitz, Course of Theoretical Physics, Statistical Physics, Part 1, Pergamon Press, New York (1980).
  * (43) K.Huang, Statistical Mechanics, 2nd edition, John Wiley and Sons, Montreal (1987).
  * (44) P.G. de Gennes, Mol. Cryst. and Liquid Cryst., 34, 91 (1976).