# Robust Domain Adaptation for Pre-trained Multilingual Neural Machine
Translation Models

Mathieu Grosso, Pirashanth Ratnamogan, Alexis Mathey,
William Vanhuffel, Michael Fotso Fotso
BNP Paribas
(mathieu.grosso, pirashanth.ratnamogan<EMAIL_ADDRESS>
<EMAIL_ADDRESS>
###### Abstract

Recent literature has demonstrated the potential of multilingual Neural
Machine Translation (mNMT) models. However, the most efficient models are not
well suited to specialized industries. In these cases, internal data is scarce
and expensive to find in all language pairs. Therefore, fine-tuning a mNMT
model on a specialized domain is hard. In this context, we decided to focus on
a new task: Domain Adaptation of a pre-trained mNMT model on a single pair of
language while trying to maintain model quality on generic domain data for all
language pairs. The risk of loss on generic domain and on other pairs is high.
This task is key for mNMT model adoption in the industry and is at the border
of many others. We propose a fine-tuning procedure for the generic mNMT that
combines embeddings freezing and adversarial loss. Our experiments
demonstrated that the procedure improves performances on specialized data with
a minimal loss in initial performances on generic domain for all languages
pairs, compared to a naive standard approach (+10.0 BLEU score on specialized
data, -0.01 to -0.5 BLEU on WMT and Tatoeba datasets on the other pairs with
M2M100).

##  1 Introduction

Language
pair i* mNMT model pre-trained Language pair i** mNMT model Adapted Language
pair 1* Language
pair N* *Generic data ** Specialized data  Figure 1: Domain Adaptation of a
Pre-trained mNMT

Building a NMT model supporting multiple language pairs is an active and
emerging area of research (NLLB Team et al., 2022; Fan et al., 2020; Tang et
al., 2020). Multilingual NMT(mNMT) uses a single model that supports
translation in multiple language pairs. Multilingual models have several
advantages over their bilingual counterparts (Arivazhagan et al., 2019b). This
modeling proves to be both efficient and effective as it reduces the
operational cost (a single model is deployed for all language pairs) and
improves translation performances, especially for low-resource languages.

All these advantages make mNMT models interesting for real-world applications.
However, they are not suitable for specialized industries that require domain-
specific translation. Training a model from scratch or fine-tuning all the
pairs of a pre-trained mNMT model is almost impossible for most companies as
it requires access to a large number of resources and specialized data. That
said, fine-tuning a single pair of a pre-trained mNMT model in a specialized
domain seems possible. Ideally this domain adaptation could be learned while
sharing parameters from old ones, without suffering from catastrophic
forgetting (Mccloskey and Cohen, 1989). This is rarely the case. The risk of
degrading performance on old pairs is high due to the limited available data
from the target domain and to the extremely high complexity of the pre-trained
model. In our case, overfitting on fine-tuning data means that the model might
not even be multilingual anymore

In this context, this article focuses on a new real-world oriented task fine-
tuning a pre-trained mNMT model in a single pair of language on a specific
domain without losing initial performances on the other pairs and generic
data. Our research focuses on fine-tuning two state-of-the-art pre-trained
multilingual mNMT freely available: M2M100 (Fan et al., 2020) and mBART50
(Tang et al., 2020) which both provide high performing BLEU scores and
translate up to 100 languages.

We explored multiple approaches for this domain adaptation. Our experiments
were made on English to French data from medical
domain111https://opus.nlpl.eu/EMEA-v3.php. This paper shows that fine-tuning a
pre-trained model with initial layers freezing, for a few steps and with a
small learning rate is the best performing approach.

It is organized as follows : firstly, we introduce standard components of
modern NMT, secondly we describe related works, thirdly we present our
methods. We finally systematically study the impact of some state-of-the-art
fine-tuning methods and present our results.

Our main contributions can be separated into 2 parts:

  * •

Defining a new real-world oriented task that focuses on domain adaptation and
catastrophic forgetting on multilingual NMT models

  * •

Defining a procedure that allows to finetune a pre-trained generic model on a
specific domain

##  2 Background

###  2.1 Neural Machine Translation

Neural Machine Translation (NMT) has become the dominant field of machine
translation. It studies how to automatically translate from one language to
another using neural networks.

Most NMT systems are trained using Seq2Seq architectures (Sutskever et al.,
2014; Cho et al., 2014) by maximizing the prediction of the target sequence
$V_{T}=(v_{1},\ldots,v_{T})$, given the source sentence
$W_{S}=(w_{1},\ldots,w_{S})$:

$P\left(v_{1},\ldots,v_{T}\mid w_{1},\ldots,w_{S}\right)$

Today the best performing Seq2Seq architecture for NMT is based on
Transformers (Vaswani et al., 2017) architecture. They are built on different
layers among which the multi-head attention and the feed-forward layer. These
are applied sequentially and are both followed by a residual connection (He et
al., 2015) and layer normalization (Ba et al., 2016).
Although powerful, traditional NMT only translates from one language to
another with a high computational cost compared to its statistical
predecessor. It has been shown that a simple language token can condition the
network to translate a sentence in any target language from any source
language (Johnson et al., 2017). It allows to create multilingual models that
can translate between multiple languages. Using previous notation the
multilingual model adds the condition on target language in the previous
modeling

$P\left(v_{1},\ldots,v_{T}\mid w_{1},\ldots,w_{S},\ell\right)$

where  $\ell$ is the target language.

###  2.2 Transfer Learning

Transfer learning is a key topic in Natural Language Processing (Devlin et
al., 2018; Liu et al., 2019). It is based on the assumption that pre-training
a model on a large set of data in various tasks will help initialize a network
trained on another task where data is scarce.

It is already a key area of research in NMT where large set of generic data
are freely available (news, common crawl, …). However, real-world applications
require specialized models. In-domain data is rare and more costly to gather
for industries (finance, legal, medical, …) making specialized models harder
to train. It is even more true for multilingual model.

In our work, we study how we can adapt a mNMT model on a specific domain by
fine-tuning on only one language pair, without losing too much generality for
all language pairs.

##  3 Related works

###  3.1 Multilingual Neural Machine Translation

While initial research on NMT started with bilingual translation systems
(Sutskever et al., 2014; Cho et al., 2014; Luong et al., 2015; Yang et al.,
2020), it has been shown that the NMT framework is extendable to multilingual
models (Dong et al., 2015; Firat et al., 2016; Johnson et al., 2017; Dabre et
al., 2020) mNMT has seen a sharp increase in the number of publications, since
it is easily extendable and it allows both end-to-end modeling and cross
lingual language representation (Conneau et al., 2017; Linger and Hajaiej,
2020; Conneau et al., 2019).

Competitive multilingual models have been released and open sourced. mBART
(Liu et al., 2019) first, was trained following the BART (Lewis et al., 2019)
objective before being finetuned on an English-centric multilingual dataset
(Tang et al., 2020). M2M100 (Fan et al., 2020) scaled large transformer layers
(Vaswani et al., 2017) with a lot of mined data in order to create a mNMT
without using English as pivot, that can perform translation between any pairs
among 100 languages. More recently, NLLB was released (NLLB Team et al.,
2022), extending the M2M100 framework to 200 languages. Those models are
extremely competitive as they have similar performance to their bilingual
counterpart while allowing a pooling of training and resources.

Our experiments will rely on M2M100 and mBART but it can be generalized to any
new pre-trained multilingual model (NLLB Team et al., 2022).

###  3.2 Domain Adaptation

Domain Adaptation in the field of NMT is a key real-world oriented task. It
aims at maximizing model performances on a certain in-domain data
distribution. Dominant approaches are based on fine-tuning a generic model
using either in-domain data only or a mixture of out-of-domain and in-domain
data to reduce overfitting (Servan et al., 2016a; Van Der Wees et al., 2017).
Many works have extended domain adaptation to multi-domain, where model is
finetuned on multiple and different domains (Sajjad et al., 2017; Zeng et al.,
2018; Mghabbar and Ratnamogan, 2020).
However, to the best of our knowledge, our work is the first exploring domain
adaptation in the context of recent pre-trained multilingual neural machine
translation systems, while focusing on keeping the model performant in out-of-
domain data in all languages.

###  3.3 Learning without forgetting

Training on a new task or new data without losing past performances is a
generic machine learning task, named Learning without forgetting (Li and
Hoiem, 2016).

Limiting pre-trained weights updates using either trust regions or adversarial
loss is a recent idea that has been used to improve training stability in both
natural language processing and computer vision (Zhu et al., 2019; Jiang et
al., 2020; Aghajanyan et al., 2020). These methods haven’t been explored in
the context of NMT but are key assets that demonstrated their capabilities on
other NLP tasks (Natural Language Inference in particular). Our work will
apply a combination of those methods to our task.

###  3.4 Zero Shot Translation

MNMT has shown the capability of direct translation between language pairs
unseen in training: a mNMT system can automatically translate between unseen
pairs without any direct supervision, as long as both source and target
languages were included in the training data Johnson et al. (2017). However,
prior works (Johnson et al., 2017; Firat et al., 2016; Arivazhagan et al.,
2019a) showed that the quality of zero-shot NMT significantly lags behind
pivot-based translation (Gu et al., 2019). Based on these ideas, some paper
(Liu et al., 2021) have focused on training a mNMT model supporting the
addition of new languages by relaxing the correspondence between input tokens
and encoder representations, therefore improving its zero-shot capacity. We
were interested in using this method as learning less specific input tokens
during the finetuning procedure could help our model not to overfit the
training pairs. Indeed, generalizing to a new domain can be seen as a task
that includes generalizing to an unseen language.

##  4 Methods

Our new real-world oriented task being at the cross-board of many existing
task, we applied ideas from current literature and tried to combine different
approaches to achieve the best results.

###  4.1 Hyperparameters search heuristics for efficient fine-tuning

We seek to adapt generic multilingual model to a specific task or domain.
(Cettolo et al., 2014; Servan et al., 2016b). Recent works in NMT (Domingo et
al., 2019) have proposed methods to adapt incrementally a model to a specific
domain. We continue the training of the generic model on specific data,
through several iterations (see Algorithm 1). This post-training fine-tuning
procedure is done without dropping the previous learning states of the
multilingual model. The resulting model is considered as adapted or
specialized to a specific domain. We want to avoid the model to suffer from
forgetting on generic domain and pairs. To this end, we include different
methods in this fine-tuning, that have been mentioned in the literature. These
methods includes in particular choosing a small learning rate (Howard and
Ruder, 2018), a triangular learning schedule (Houlsby et al., 2019), reducing
the number of steps and freezing some of the layers(Stickland and Murray,
2019).

###  4.2 Smoothness-inducing Adversarial Regularizer

We seek to reduce the loss on generic domain and other pairs. Indeed, due to
limited data resources from downstream tasks and the extremely large capacity
of pre-trained models, aggressive fine-tuning often causes the adapted model
to overfit the data of downstream tasks and forget the knowledge of the pre-
trained model. To this end, we added a Smoothness-inducing Adversarial
Regularization (SMART) term during the fine-tuning (Jiang et al., 2020).
Models fine-tuned on GLUE task with SMART approach outperform even the
strongest pre-trained baseline on all 8 tasks. Comparing with BERT (Devlin et
al., 2018) and RoBERTa (Liu et al., 2019), BERTSMART and RoBERTaSMART are
performing better by a big margin. This approach gives a smoothness-inducing
property to the model $f$. This is helpful to prevent overfitting and to
improve generalization on low resource target domain for a certain task.
Therefore, adding it to our task should avoid overfitting on the new domain.

Given the model $f(.;\theta)$ and $n$ data points of the target task denoted
by $\left\\{\left(x_{i},y_{i}\right)\right\\}_{i=1}^{n}$, where $x_{i}$’s
denote the embeddings of the input sentences, given by the first embedding
layer of the language model and $y_{i}$’s are the associated labels, SMART is
adding a regularization term $\mathcal{R}_{\mathrm{s}}(\theta)$ to the
canonical optimisation loss below:

$\min_{\theta}(\mathcal{F}(\theta))=\mathcal{L}(\theta)+\lambda_{\mathrm{s}}\mathcal{R}_{\mathrm{s}}(\theta)$
(1)

where $\mathcal{L}(\theta)$ is the loss function defined as

$\mathcal{L}(\theta)=\frac{1}{n}\sum_{i=1}^{n}\ell\left(f\left(x_{i};\theta\right),y_{i}\right)$
(2)

and $\ell(\cdot,\cdot)$ is the loss function depending on the target task,
$\lambda_{\mathrm{s}}>0$ is a tuning parameter, and
$\mathcal{R}_{\mathrm{s}}(\theta)$ is the smoothness-inducing adversarial
regularizer. Here we define $\mathcal{R}_{\mathrm{s}}(\theta)$ as

$\mathcal{R}_{\mathrm{s}}(\theta)=\frac{1}{n}\sum_{i=1}^{n}\max_{\left\|\bar{x}_{i}-x_{i}\right\|_{p}\leq\epsilon}\ell_{\mathrm{s}}\left(f\left(\bar{x}_{i};\theta\right),f\left(x_{i};\theta\right)\right)$
(3)

where $\epsilon>0$ is a tuning parameter. Since NMT is a classification tasks,
$f(;\theta)$ outputs a probability simplex and $\ell_{\mathrm{s}}$ is chosen
as the symmetrized KL-divergence, i.e.,

$\ell_{\mathrm{s}}(P,Q)=\mathcal{D}_{\mathrm{KL}}(P\|Q)+\mathcal{D}_{\mathrm{KL}}(Q\|P)$

###  4.3 Enabling the model to learn less aggressive input tokens

We seek at reducing the loss of performances on the pairs learned during the
pre-training of the model. A factor causing a too important language-specific
representation is the positional correspondence to input tokens (Liu et al.,
2021). Relaxing it should help the model learn the new domain while not
focusing too much on the language representation. Recent advances in mNMT
showed that we can reduce the positional correspondence learned from the input
tokens seen during training thanks to Positional Disentangling Encoder (PDE)
(Liu et al., 2021). PDE corresponds to removing some of the residual
connections of the model architecture. PDE is reported to beat by +18.5 BLEU
models that do not use it on zero shot translation pairs while retaining
quality on supervised directions (Liu et al., 2021). Doing this during the
domain adaptation fine-tuning helps to learn less specific input tokens (since
we train only from English to French). Therefore, adapting this method to our
domain adaptation training is straightforward and could bring gain in BLEU on
language pairs seen during pre-training while not sacrificing performances on
the new specific domain.

##  5 Experimental Settings

###  5.1 Pre-trained Generic Models used

We have worked with two pre-trained mNMT models: M2M100 and mBART50 large.

M2M100 is a multilingual encoder-decoder model, based on large Transformer
architecture that can handle 100 languages. It was trained on a non-English-
centric dataset of 7.5B sentences from generic domain, as such it is the first
true many-to-many NMT model. To ease the fine-tuning process and due to
hardware limitations, we worked with the lightest version released (418M
parameters).

mBART50 is a multilingual encoder-decoder model, based on training on an
English-centric dataset and on large Transformer architecture that can handle
50 languages. It was trained following the BART objective (Lewis et al.,
2019). More formally, the model aims to reconstruct a text that has been
previously noised.

We will compare the domain adaptation performance between mBART50 which was
trained on English-centric data and M2M100 which was trained on non English-
centric data.

###  5.2 Datasets and preprocessing

In order to assess the effectiveness of our different domain adaptation
strategies, we focused on the medical domain on the English to French using
data from the EMEA3222https://opus.nlpl.eu/EMEA.php dataset (Tiedemann, 2012).
We used the same preprocessing as the original publications (BPE joint-
tokenization from sentencepiece). We split the dataset into a train and a test
dataset. We chose to use the first 5.000 sentences for the testing set and
350.000 sentences for the training set. For the evaluation data on the generic
domain, we used generic data from different sources including
WMT333https://opus.nlpl.eu/WMT-News.php and
Tatoeba444https://github.com/Helsinki-NLP/Tatoeba-Challenge. For the
evaluation data on the medical domain, we also used EMEA3 dataset in different
languages.

###  5.3 Detailed Procedure

We first define a hyperparameters search heuristics procedure. We chose a
range of learning rate and trained the model with these values. We set prior
threshold between the loss we accept on generic data and the increase we
target on medical data. Then apply the procedure in algorithm 1. Having done
this, we kept best settings (best learning rate and number of steps for given
threshold), and tried freezing first layers to reduce the loss on generic
domain. We define ${\epsilon}_{3}$, a threshold between loss on medical domain
and gain on generic domain. We reproduce the same procedure and reports our
best results. This allows us to find the optimal model $\theta_{opt}$,
representing the best compromise between not losing performances on generic
data and good adaptation to the medical domain.

Algorithm 1 Hyperparameters search heuristic for domain adaptation using
simple fine-tuning Algorithm

1:$T$ : the maximum number of steps; $L$ : the number of layers we have
frozen; $L_{r}$: the learning rate, $\epsilon_{1}$: the threshold for
${\Delta}_{1}$ : the difference of BLEU between baseline and adapted model on
EN-FR generic domain data, $\epsilon_{2}$ threshold for $\Delta_{2}$ : the
mean difference of BLEU between baseline and adapted model on all other
generic data, $\theta_{0}$ is the parameters of the pretrained model,
$\theta_{opt}$: is the parameters of the model that has optimal value of BLEU
on domain and generic.

2:$T\leftarrow 100K$

3:$L\leftarrow 1$

4:for $L_{r}=3e-5,1e-5,...,1e-8$ do

5: $\theta_{s}\leftarrow\theta_{0}$

6: for $s\leftarrow 1$ to $T$ do

7: $\theta_{s+1}\leftarrow$ AdamUpdate ${}_{B}\left({\theta}_{s}\right)$

8: Every 2k steps, evaluate model on validation set and compute $\Delta_{1}$
and $\Delta_{2}$

9: if $\Delta_{1}\leq\epsilon_{1}\cup\Delta_{2}\leq\epsilon_{2}$ is true then

10: $\theta_{opt}\leftarrow\theta_{s}$

11: else

12: $\theta_{opt}\leftarrow\theta_{s}$

13: end For loop

14: end if

15: end for

16:end for

17:$\theta_{opt}$

M2M100 We trained M2M100 on the medical EN-FR dataset. We used the adam
optimizer ($\beta_{1}=0.9,\beta_{2}=0.98$), label smoothing, a dropout of 0.1
and a weight decay of 0. We applied our hyperparameters search heuristic
procedure 1 to find the best model. We set $\epsilon_{1}=2,\epsilon_{2}=1$. On
this configuration, optimal results were reported with a learning rate of
1e-07, freezing the embeddings at the encoder level, and 60K steps.
mBART50 We trained mBART50 large on the medical EN-FR dataset. We used the
adam optimizer ($\beta_{1}=0.9,\beta_{2}=0.98$), label smoothing, a dropout of
0.3 and a weight decay of 0. Again, we applied our hyperparameters search
heuristic procedure to find the best model 1. We had to increase the value of
$\epsilon_{1},\epsilon_{2}$ since mBART50 tends to forget the generic domain
quicker than M2M100. We set $\epsilon_{1}=4,\epsilon_{2}=3$. On this
configuration, optimal results were reported with a learning rate of $6e-07$,
freezing the embeddings at the encoder level and 10K steps.

SMART: We finetuned the model with the SMART procedure and continue
hyperparameters search as in algorithm 1. In Algorithm 2, we note
$\mathcal{R}_{\mathrm{s}}(\theta)=\frac{1}{\lvert B\rvert}\sum_{x_{i}\in
B}\max_{\left\|\bar{x}_{i}-x_{i}\right\|p\leq\epsilon}\ell_{\mathrm{s}}\left(f\left(\bar{x}_{i};\theta\right),f\left(x_{i};\theta\right)\right)$
and $AdamUpdate$ the ADAM update rule for optimizing equation 1 using the
mini-batch $B$. Lastly, we set $T_{\tilde{x}}=1$. For the perturbation, we set
$\epsilon=10^{-5}$ and $\sigma=10^{-5}$. The learning rate $\eta$ is set to
$10^{-3}$.

Algorithm 2 Adding SMART to procedure

1:$T$ : the total number of iterations; $\mathcal{X}$: the dataset;
$\theta_{0}$: the parameter of the pre-trained model; $\sigma^{2}$: the
variance of the random initialization for $\bar{x}_{i}$ ’s; $T_{\tilde{x}}$:
the number of iterations for updating $\bar{x}_{i}$ ’s; $\eta$: the learning
rate for updating $\bar{x}_{i}$ ’s; $\beta$: clipping value.

2:$\theta_{1}\leftarrow\theta_{0}$

3:for $t\leftarrow 1$ to $T$ do

4: $\bar{\theta}_{s}\leftarrow\theta_{t}$

5: Sample a mini-batch $B$ from $\mathcal{X}$

6: For all $x_{i}\in B$, initialize $\bar{x}_{i}\leftarrow x_{i}+v_{i}$ with
$v_{i}\sim\mathcal{N}\left(0,\sigma^{2}I\right)$

7: for $m=1,...,T_{\tilde{x}}$ do

8:
$\bar{x}_{i}\leftarrow\bar{x}_{i}+\eta\mathcal{R}_{\mathrm{s}}(\bar{\theta}_{s})$

9: end for

10: $\bar{\theta}_{s+1}\leftarrow AdamUpdate_{B}\left(\bar{\theta}_{s}\right)$

11: $\theta_{t+1}\leftarrow CLIP(\bar{\theta}_{s+1},1-\beta,1+\beta)$

12:end for

13:$\theta_{T}$

PDE Finally, we define PDE. It consists in applying Algorithm 1 and then
removing first all the residual connection in the penultimate Encoder layers
(Chen et al., 2022), then we try removing only the attention layer residual
connections (figure 2).

Multi-Head Attention Multi-Head Attention Add & NormAdd & Norm Multi-Head
Attention Multi-Head Attention Add & NormAdd & Norm Figure 2: PDE
Illustration: Removing Residual Connections on encoder block

##  6 Results and Analysis

###  6.1 Hyperparameters search heuristic

####  6.1.1 Main Results

M2M100 As shown in table 1 we reached more than 9.00 increase of BLEU score on
the medical dataset without sacrificing performance on generic domain, the
loss is not important on most of the pairs (between 0.01 and 0.2). In figure
3, we see that the mean results is rather stable and that the BLEU on generic
English to French data does not decrease a lot (around -1.5 BLEU). The model
converges after 60K steps so we stop training.

mBART50 Again we reach more than 9.00 BLEU increase (Figure 4). We observe
that after 50K steps mBART50 starts converging around 40.00 BLEU, yet we
decided to stop domain adaptation training sooner than with M2M100 as a trade-
off between good performance on the EN-FR medical domain and loss of
performance on the generic domain. Globally, we achieved better results with
M2M100 than mBART50.

Figure 3: Domain Adaptation (Medical Domain EN-FR) of M2M100 Figure 4: Domain
Adaptation (Medical Domain EN-FR) of mBART50

####  6.1.2 Catastrophic forgetting with a big learning rate

We tested several learning rate values and we report here our results with a
bigger learning rate ($3e-5$). For both models, it led to a catastrophic
forgetting on the non-finetuned pairs along with a huge performance increase
on the EN-FR Medical dataset, reaching a higher BLEU on the Medical dataset.
We decided to focus on a smaller learning rate as a trade-off between loss on
generic domain and gain on the medical domain.

Figure 5: Domain Adaptation of M2M100 with big Learning Rate Table 1: Global
results on domain adaptation of M2M100 and mBART50

| M2M100 | mBART50
---|---|---
| Baseline | Finetuned | Finetuned SMART | Finetuned PDE | Baseline | Finetuned | Finetuned SMART | Finetuned PDE
EN-FR medical data | 26.94 | 36.05 | 35.93 | 29.71 | 23.99 | 33.87 | 30.3 | 26.12
EN-FR (WMT) | 28.63 | 26.90 | 26.41 | 25.01 | 31.25 | 27.10 | 26.10 | 17.56
Mean results | 24.97 | 25.38 | 25.15 | 21.70 | 19.83 | 16.85 | 15.1 | 13.63
DE-EN (WMT) | 22.42 | 22.53 | 22.39 | 21.15 | 26.13 | 21.85 | 20.64 | 19.44
EN-DE (WMT) | 19.48 | 19.52 | 19.21 | 17.96 | 22.72 | 19.84 | 18.61 | 16.83
RU-EN (WMT) | 26.3 | 26.27 | 25.4 | 24.90 | 29.72 | 24.64 | 23.55 | 22.47
FR-DE (WMT) | 17.82 | 17.80 | 17.58 | 14.69 | 10.92 | 8.98 | 7.1 | 4.30
EN-FI (WMT) | 12.51 | 12.72 | 12.51 | 11.74 | 13.39 | 11.23 | 10.27 | 9.74
FI-EN (WMT) | 23.55 | 23.18 | 23.39 | 21.40 | 22.10 | 18.90 | 17.75 | 16.33
BG-IT (Tatoeba) | 26.65 | 27.54 | 27.01 | 26.20 | * | * | * | *
DA-TR (Tatoeba) | 20.22 | 22.27 | 21.75 | 20.23 | * | * | * | *
PL-RU (Tatoeba) | 33.79 | 33.72 | 33.68 | 29.69 | 14.45 | 10.49 | 10.34 | 9.87
PT-ES (Tatoeba) | 51.49 | 51.98 | 52.54 | 50.34 | 21.57 | 18.87 | 16.80 | 12.54
JA-ES (Tatoeba) | 20.55 | 21.66 | 21.58 | 20.30 | 17.55 | 15.42 | 13.27 | 11.12

###  6.2 SMART

Figure 6: Adding SMART to M2M100 Domain Adaptation training

We have reported our fine-tuning results for M2M100 and mBART50 with SMART in
Table 1.

Our goal with SMART was to reach a higher BLEU score on the generic domain
data without sacrificing performances on the medical dataset. In Table 1, we
note a good increase in BLEU score. Moreover, we have noted that the BLEU
change less when moving learning rate in a reasonable range compared to the
other methods that are extremely sensitive to hyperparameters. In this
context, SMART is useful in order to achieve quick adaptation of a mNMT model
to a new domain. It makes domain adaptation procedure more consistent.
Therefore, SMART training procedure allows efficient and robust domain
adaptation. However if exploring a large scale of hyper parameters if feasible
simple fine-tuning procedure like in Algorithm 1 can provide better results as
shown in Table 1.

###  6.3 PDE

We seek at reducing the loss of performances on the pairs learned during pre-
training of the model (and that are not used during the post-training domain
adaptation). Relaxing the correspondence to the input tokens learned during
Domain Adaptation. Fine-tuning was supposed to help learning less specific
input tokens and therefore the model would be less likely to forgot all the
pretrained pairs. As expected, the model learned less aggressive input tokens
and do not overfit on English input tokens. However, in practice this does not
seem to work well. Indeed, the model is also likelier to forget the pretrained
input tokens making this method unfit to our procedure. Using PDE a posteriori
(during fine-tuning) seems to be inefficient, since the model is performing
worse on all pairs and not only on the English pairs.

We report our results in table 1.

###  6.4 Analysis

####  6.4.1 Zero-shot Domain Adaptation on other pairs

We challenged the approach on domain adaptation on languages unseen during the
post-training on the medical domain using EMEA3 dataset available on other
languages. Table 2 shows that for M2M100 all BLEU scores are increasing,
moreover the pairs that implies either English or French are particularly
benefiting from this domain adaptation. On mBART50, we also note improvements,
first the loss is less important than on generic dataset for the pairs that do
not include French as output showing that the model is learning a bit. When
French is the output, the domain adaptation is working really fine and we see
improvements. Domain-specific data are often hard to gather, especially for
low-ressource pairs. That’s why being able to improve the performances on a
new domain for several pairs using a domain-specific dataset from a single
pair is a very interesting propriety from the mNMT models.

| M2M100 | mBART50
---|---|---
| Baseline | Ours | Baseline | Ours
EN-FI medical | 12.93 | 14.83 | 10.83 | 10.1
DE-PL medical | 12.62 | 13.6 | 11.1 | 7.85
FR-IT medical | 23.07 | 24.62 | 10.77 | 8.58
EN-ES medical | 32.38 | 35.15 | 15.82 | 17.5
ES-IT medical | 25.43 | 27.06 | 8.40 | 7.50
ES-FR medical | 24.37 | 30.64 | 19.03 | 25.6
LT-PL medical | 12.49 | 13.8 | 8.5 | 7.9
DE-FR medical | 18.85 | 22.20 | 13.3 | 18.19
LT-PT medical | 17.44 | 19.26 | * | *

Table 2: Zero shot domain adaptation on medical dataset for other pairs

####  6.4.2 Comparison of initial pre-trained mNMT models (mBART 50 vs M2M100)

We investigated why mBART50 was more likely to forget other pairs compared to
M2M100. First, we have worked with the 418M-parameters version of M2M100. This
is not the largest M2M100 version released (and certainly not the most
optimized) and this could possibly explain the differences. Then, another
hypothesis is the different dataset used during training of both models.
Indeed, mBART50 is trained on English-centric data, and M2M100 is not. Non-
English centric models are known to achieve higher BLEU especially on low
resource data (Fan et al., 2020). Extending this study to domain adaptation,
we believe non-English-centric models might be more robust to domain
adaptation. We noted that when fine-tuning mBART50 with a bigger learning
rate, the first pairs to be forgotten are the non-English ones. Testing this
hypothesis on NLLB might be useful.

##  7 Conclusion and Discussion

In this paper, we propose a study of robust domain adaptation approaches on
mNMT models where in-domain data is available only for a single language pair.
Best performing approach combines embedding freezing and simple fine-tuning
with good hyperparameters. This approach shows good improvements with few in-
domain data on all language pairs. The framework effectively avoids
overfitting and aggressive forgetting on out-of-domain generic data while
quickly adapting to in-domain data. We demonstrate that this could be a
solution for incremental adaptation of mNMT models. Finally our work is a call
for more research in domain adaptation for multilingual models as it is key
for real-world applications.

##  8 Limitations

This study was limited by hardware issues. We did not have the possibility to
fine-tune on M2M100 large version (12B parameters) that requires 64 GB of
VRAM.

Testing our results with a larger version of M2M100 might be interesting.

Also, our study focused on two pre-trained multilingual neural machine
translation models. However, many others exist and will be released (NLLB Team
et al., 2022). We think that our work is generic enough to be applied on other
pre-trained models but extensive experiments on these new models should be
carried out.

Finally, the work has been realised on English to French data. We showed
domain adaptation is possible for languages with English morphology and tested
the impact of this training on many different languages morphology (Japanese,
English, Russian, …). Applying domain adaptation training on other morphology
languages and on other domains is also an area to investigate.

##  9 Ethics Statement

The dataset was gathered on OPUS and is largely open-sourced. It was released
by Tiedemann (2012) and we have downloaded it from OPUS website. We have
reviewed the dataset and have not noted any issue with these data. They are
very specific to health domain and therefore are not inappropriate. The
dataset does not deal with demographic or identity characteristics.

Moreover, these experiments were made using only 2 GPUs and training were
relatively short. Given the urgency of addressing climate change, we believe
our domain adaptation procedure could help have high-performing mNMT models at
small carbon and energy costs. Moreover, SMART framework allows for quicker
research of the right hyperparameters, therefore reducing even further the
number of experiments and the carbon costs of our method.

##  10 Acknowledgments

The authors would also like to thank Mr. Baoyang Song, Mr. Laurent Lam, Mr.
Laatiri Seif Eddine from BNP Paribas for their valuable comments and
suggestions. The work is supported by BNP Paribas.

## References

  * Aghajanyan et al. (2020) Armen Aghajanyan, Akshat Shrivastava, Anchit Gupta, Naman Goyal, Luke Zettlemoyer, and Sonal Gupta. 2020.  Better fine-tuning by reducing representational collapse.  _arXiv preprint arXiv:2008.03156_.
  * Arivazhagan et al. (2019a) Naveen Arivazhagan, Ankur Bapna, Orhan Firat, Roee Aharoni, Melvin Johnson, and Wolfgang Macherey. 2019a.  The missing ingredient in zero-shot neural machine translation.
  * Arivazhagan et al. (2019b) Naveen Arivazhagan, Ankur Bapna, Orhan Firat, Dmitry Lepikhin, Melvin Johnson, Maxim Krikun, Mia Xu Chen, Yuan Cao, George Foster, Colin Cherry, Wolfgang Macherey, Zhifeng Chen, and Yonghui Wu. 2019b.  Massively multilingual neural machine translation in the wild: Findings and challenges.
  * Ba et al. (2016) Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E. Hinton. 2016.  Layer normalization.
  * Cettolo et al. (2014) Mauro Cettolo, Nicola Bertoldi, Marcello Federico, Holger Schwenk, Loïc Barrault, and Christophe Servan. 2014.  Translation project adaptation for MT-enhanced computer assisted translation.  _Machine Translation_ , 28:127.
  * Chen et al. (2022) Guanhua Chen, Shuming Ma, Yun Chen, Dongdong Zhang, Jia Pan, Wenping Wang, and Furu Wei. 2022.  Towards making the most of cross-lingual transfer for zero-shot neural machine translation.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 142–157, Dublin, Ireland. Association for Computational Linguistics.
  * Cho et al. (2014) Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014.  Learning phrase representations using rnn encoder-decoder for statistical machine translation.  _arXiv preprint arXiv:1406.1078_.
  * Conneau et al. (2019) Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  Unsupervised cross-lingual representation learning at scale.  _arXiv preprint arXiv:1911.02116_.
  * Conneau et al. (2017) Alexis Conneau, Guillaume Lample, Marc’Aurelio Ranzato, Ludovic Denoyer, and Hervé Jégou. 2017.  Word translation without parallel data.  _arXiv preprint arXiv:1710.04087_.
  * Dabre et al. (2020) Raj Dabre, Chenhui Chu, and Anoop Kunchukuttan. 2020.  A survey of multilingual neural machine translation.  _ACM Computing Surveys (CSUR)_ , 53(5):1–38.
  * Devlin et al. (2018) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018.  Bert: Pre-training of deep bidirectional transformers for language understanding.
  * Domingo et al. (2019) Miguel Domingo, Mercedes García-Martínez, Álvaro Peris, Alexandre Helle, Amando Estela, Laurent Bié, Francisco Casacuberta, and Manuel Herranz. 2019.  Incremental adaptation of NMT for professional post-editors: A user study.  _CoRR_ , abs/1906.08996.
  * Dong et al. (2015) Daxiang Dong, Hua Wu, Wei He, Dianhai Yu, and Haifeng Wang. 2015.  Multi-task learning for multiple language translation.  In _Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 1723–1732, Beijing, China. Association for Computational Linguistics.
  * Fan et al. (2020) Angela Fan, Shruti Bhosale, Holger Schwenk, Zhiyi Ma, Ahmed El-Kishky, Siddharth Goyal, Mandeep Baines, Onur Celebi, Guillaume Wenzek, Vishrav Chaudhary, Naman Goyal, Tom Birch, Vitaliy Liptchinsky, Sergey Edunov, Edouard Grave, Michael Auli, and Armand Joulin. 2020.  Beyond english-centric multilingual machine translation.  arXiv.
  * Firat et al. (2016) Orhan Firat, Kyunghyun Cho, and Yoshua Bengio. 2016.  Multi-way, multilingual neural machine translation with a shared attention mechanism.
  * Gu et al. (2019) Jiatao Gu, Yong Wang, Kyunghyun Cho, and Victor O. K. Li. 2019.  Improved zero-shot neural machine translation via ignoring spurious correlations.
  * He et al. (2015) Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2015.  Deep residual learning for image recognition.
  * Houlsby et al. (2019) Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin de Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019.  Parameter-efficient transfer learning for nlp.
  * Howard and Ruder (2018) Jeremy Howard and Sebastian Ruder. 2018.  Universal language model fine-tuning for text classification.
  * Jiang et al. (2020) Haoming Jiang, Pengcheng He, Weizhu Chen, Xiaodong Liu, Jianfeng Gao, and Tuo Zhao. 2020.  SMART: Robust and efficient fine-tuning for pre-trained natural language models through principled regularized optimization.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_. Association for Computational Linguistics.
  * Johnson et al. (2017) Melvin Johnson, Mike Schuster, Quoc V. Le, Maxim Krikun, Yonghui Wu, Zhifeng Chen, Nikhil Thorat, Fernanda Viégas, Martin Wattenberg, Greg Corrado, Macduff Hughes, and Jeffrey Dean. 2017.  Google’s multilingual neural machine translation system: Enabling zero-shot translation.  _Transactions of the Association for Computational Linguistics_ , 5:339–351.
  * Lewis et al. (2019) Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zettlemoyer. 2019.  Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension.
  * Li and Hoiem (2016) Zhizhong Li and Derek Hoiem. 2016.  Learning without forgetting.
  * Linger and Hajaiej (2020) Mathis Linger and Mhamed Hajaiej. 2020.  Batch clustering for multilingual news streaming.  _arXiv preprint arXiv:2004.08123_.
  * Liu et al. (2021) Danni Liu, Jan Niehues, James Cross, Francisco Guzmán, and Xian Li. 2021.  Improving zero-shot translation by disentangling positional information.
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  Roberta: A robustly optimized bert pretraining approach.
  * Luong et al. (2015) Minh-Thang Luong, Hieu Pham, and Christopher D Manning. 2015.  Effective approaches to attention-based neural machine translation.  _arXiv preprint arXiv:1508.04025_.
  * Mccloskey and Cohen (1989) Michael Mccloskey and Neil J. Cohen. 1989.  Catastrophic interference in connectionist networks: The sequential learning problem.  _The Psychology of Learning and Motivation_ , 24:104–169.
  * Mghabbar and Ratnamogan (2020) Idriss Mghabbar and Pirashanth Ratnamogan. 2020.  Building a multi-domain neural machine translation model using knowledge distillation.  _arXiv preprint arXiv:2004.07324_.
  * NLLB Team et al. (2022) NLLB Team, Marta R. Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang, Guillaume Wenzek, Al Youngblood, Bapi Akula, Loic Barrault, Gabriel Mejia Gonzalez, Prangthip Hansanti, John Hoffman, Semarley Jarrett, Kaushik Ram Sadagopan, Dirk Rowe, Shannon Spruit, Chau Tran, Pierre Andrews, Necip Fazil Ayan, Shruti Bhosale, Sergey Edunov, Angela Fan, Cynthia Gao, Vedanuj Goswami, Francisco Guzmán, Philipp Koehn, Alexandre Mourachko, Christophe Ropers, Safiyyah Saleem, Holger Schwenk, and Jeff Wang. 2022.  No language left behind: Scaling human-centered machine translation.
  * Sajjad et al. (2017) Hassan Sajjad, Nadir Durrani, Fahim Dalvi, Yonatan Belinkov, and Stephan Vogel. 2017\.  Neural machine translation training in a multi-domain scenario.  _arXiv preprint arXiv:1708.08712_.
  * Servan et al. (2016a) Christophe Servan, Josep Crego, and Jean Senellart. 2016a.  Domain specialization: a post-training domain adaptation for neural machine translation.  _arXiv preprint arXiv:1612.06141_.
  * Servan et al. (2016b) Christophe Servan, Josep Maria Crego, and Jean Senellart. 2016b.  Domain specialization: a post-training domain adaptation for neural machine translation.  _CoRR_ , abs/1612.06141.
  * Stickland and Murray (2019) Asa Cooper Stickland and Iain Murray. 2019.  Bert and pals: Projected attention layers for efficient adaptation in multi-task learning.
  * Sutskever et al. (2014) Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014.  Sequence to sequence learning with neural networks.  _Advances in neural information processing systems_ , 27.
  * Tang et al. (2020) Yuqing Tang, Chau Tran, Xian Li, Peng-Jen Chen, Naman Goyal, Vishrav Chaudhary, Jiatao Gu, and Angela Fan. 2020.  Multilingual translation with extensible multilingual pretraining and finetuning.
  * Tiedemann (2012) Jörg Tiedemann. 2012.  Parallel data, tools and interfaces in opus.  In _Proceedings of the Eight International Conference on Language Resources and Evaluation (LREC’12)_ , Istanbul, Turkey. European Language Resources Association (ELRA).
  * Van Der Wees et al. (2017) Marlies Van Der Wees, Arianna Bisazza, and Christof Monz. 2017.  Dynamic data selection for neural machine translation.  _arXiv preprint arXiv:1708.00712_.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.
  * Yang et al. (2020) Shuoheng Yang, Yuxin Wang, and Xiaowen Chu. 2020.  A survey of deep learning techniques for neural machine translation.
  * Zeng et al. (2018) Jiali Zeng, Jinsong Su, Huating Wen, Yang Liu, Jun Xie, Yongjing Yin, and Jianqiang Zhao. 2018.  Multi-domain neural machine translation with word-level domain context discrimination.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 447–457.
  * Zhu et al. (2019) Chen Zhu, Yu Cheng, Zhe Gan, Siqi Sun, Tom Goldstein, and Jingjing Liu. 2019.  Freelb: Enhanced adversarial training for natural language understanding.  _arXiv preprint arXiv:1909.11764_.


††thanks: These two authors contributed equally††thanks: These two authors
contributed equally

# Pressure-sensitive ion conduction in a conical channel: optimal pressure and
geometry

W.Q. Boon  Institute for Theoretical Physics, Utrecht University,
Princetonplein 5, 3584 CC Utrecht, The Netherlands  T.E. Veenstra  Institute
for Theoretical Physics, Utrecht University, Princetonplein 5, 3584 CC
Utrecht, The Netherlands  M. Dijkstra  Soft Condensed Matter, Debye Institute
for Nanomaterials Science, Utrecht University, Princetonplein 1, 3584 CC
Utrecht, The Netherlands  R. van Roij  Institute for Theoretical Physics,
Utrecht University, Princetonplein 5, 3584 CC Utrecht, The Netherlands

###### Abstract

Using both analytic and numerical analyses of the Poisson-Nernst-Planck
equations we theoretically investigate the electric conductivity of a conical
channel, which in accordance with recent experiments exhibits a strong non-
linear pressure dependence. This mechanosensitive diodic behavior stems from
the pressure-sensitive build-up or depletion of salt in the pore. From our
analytic results we find that the optimal geometry for this diodic behavior
strongly depends on the flow rate, the ideal ratio of tip-to-base-radii being
equal to 0.22 at zero flow. With increased flow this optimal ratio becomes
smaller and simultaneously the diodic performance becomes weaker. Consequently
an optimal diode is obtained at zero-flow, which is realized by applying a
pressure drop that is proportional to the applied potential and to the inverse
square of the tip radius thereby countering electro-osmotic flow. When the
applied pressure deviates from this ideal pressure drop the diodic performance
falls sharply, explaining the dramatic mechanosensitivity observed in
experiments.

A fluidic channel allows for the simultaneous transport of solvent, charge,
and dissolved salt when connected to two liquid electrolyte reservoirs at
different pressures, voltages, salt concentrations, and/or temperatures. Such
ionic transport is not only interesting from a fundamental point of view, but
also for energy harvesting Zhang  _et al._ (2021); Kim  _et al._ (2010); Siria
_et al._ (2013, 2017), desalination Elimelech and Phillip (2011); Campione
_et al._ (2018) and microfluidic applications Wang  _et al._ (2009); Squires
and Quake (2005). In all these devices fluidic channels with dimensions in the
nano- and micrometer regime are used Bocquet and Charlaix (2010); Schoch  _et
al._ (2008); Eijkel and Berg (2005), a size range where the influence of
surface charge on transport becomes significant due to the relatively large
surface-to-volume ratio. This surface charge is key to electrokinetic
transduction phenomena such as the flow of electrolyte by an electric
potential drop (electro-osmosis) or the electric (streaming) current induced
by flow due to an applied pressure drop Werkhoven  _et al._ (2018, 2019); Ober
_et al._ (2021). While these electrokinetic transduction phenomena have long
been understoodLevine  _et al._ (1975); Sadeghi  _et al._ (2017a, b, c), at
least in simple channel geometries, in conical pores exotic transport behavior
such as electro-osmotic flow inversion Laohakunakorn  _et al._ (2015); Lan
_et al._ (2016); Bishop  _et al._ (2015), other non-linear flow-effects Asghar
_et al._ (2022a, b) and current rectification White and Bund (2008a); Wen  _et
al._ (2019); Proctor (2021); Woermann (2003, 2004); Kovarik  _et al._ (2009);
Lin  _et al._ (2018) have been observed. Such non-linear transport behavior
makes conical pores uniquely attractive for biochemical sensing Vlassiouk  _et
al._ (2009); Bush  _et al._ (2020); Hou  _et al._ (2011); Piruska  _et al._
(2010); Gholinejad  _et al._ (2022, 2021, 2020); Gong  _et al._ (2022) and
neuromorphic applications Li  _et al._ (2020); Ghosal  _et al._ (2019); Chun
and Chung (2015); Jung  _et al._ (2022). In this letter, we analyse the
intricate case of a micron-sized cone-shaped channel exposed to a simultaneous
pressure and electric potential drop by means of the well-known Poisson-
Nernst-Planck-Stokes (PNPS) equations. We will show that the ionic current in
conical nanopores can be either strongly reduced or enhanced by a pressure
difference and concomitant flow, resulting into an extremely mechanosensitive
ionic diode similar to those present in cell membranes Cox  _et al._ (2019);
Anishkin  _et al._ (2014) and such a pressure-sensitivity can also be used to
optimize power generation in artificial poresQian  _et al._ (2020). Recent
experiments revealed such a non-linear pressure-induced electric transport in
conical pores even at micrometer length scales Jubin  _et al._ (2018). It was
found that the electric current $I(\Delta P,\Delta\psi)$ due to an applied
potential difference $\Delta\psi$ is very sensitively dependent on the applied
pressure drop $\Delta P$ over the channel.

Figure 1: Schematic of an axially symmetric conical channel of length $L$,
base radius $R_{\rm{b}}$ at $x=0$, and tip radius $R_{\rm{t}}\leq R_{\rm{b}}$
at $x=L$. The channel connects two bulk 1:1 electrolytes in the half spaces
$x<0$ and $x>L$. The channel wall carries a negative surface charge density
$e\sigma$ that induces an electric double layer of thickness
$\lambda_{\rm{D}}$, the Debye length. Volume, charge, and salt ions are
transported through the channel by an applied potential drop $\Delta\psi$ and
pressure drop $\Delta P$.

Surprisingly, the observed pressure dependence of the electric conductance
occurred at extremely low rather than high pressures. For conical pores it was
already observed that, for $\Delta P=0$, the response of the current $I$ is
asymmetric with regard to the sign of $\Delta\psi$ and this so-called current
rectification is attributed to concentration polarization White and Bund
(2008a); Wen  _et al._ (2019); Proctor (2021); Kovarik  _et al._ (2009); Lin
_et al._ (2018); Woermann (2003, 2004); Pandey and Bhattacharyya (2021). Here
we show that the flow (and hence pressure)-sensitive conductivity for $\Delta
P\neq 0$ can also be understood by the concentration-polarization, in contrast
to earlier work which suggests that novel mechanisms are needed such as a bulk
space-chargeJubin  _et al._ (2018) or a non-linear streaming currentQian  _et
al._ (2020). Such a flow-sensitive conduction was previously noted in
numerical calculations, which however ignored electro-osmotic flowLan  _et
al._ (2011) that we find to be of great importance. Whether electro-osmotic
flow canWhite and Bund (2008b); Kubeil and Bund (2011); Vlassiouk  _et al._
(2008); Ramirez  _et al._ (2008); Cervera  _et al._ (2006) or cannotLin  _et
al._ (2015); Hsu  _et al._ (2017); Ai  _et al._ (2010); Brown  _et al._ (2021)
be ignored is debated in the literature and we reconcile these two opposing
views by showing that the importance of flow depends on Péclet number; in the
small Péclet regimeWhite and Bund (2008b) flow can be ignored while in the
large Péclet regimeLin  _et al._ (2015) it is important. This large-Pe regime
is natural for large micrometer channelsKovarik  _et al._ (2009); Lin  _et
al._ (2018); Aarts  _et al._ (2022); Choi  _et al._ (2016); Jubin  _et al._
(2018) common in experiments, while theoretical descriptions of such systems
often neglect flowDal Cengio and Pagonabarraga (2019); Poggioli  _et al._
(2019); Xiang  _et al._ (2022); Ramirez  _et al._ (2008). To account for the
effect of flow on rectification we derive for the first time a closed-form
expression for the ion-distribution in the channel. Together with the $2\times
2$ transport matrix accounting for both the pressure-driven flow and the
electro-osmotic flow neglected by Ref.Lan  _et al._ (2011) this concentration
profile naturally accounts for the mechanosensitive cone conductance, without
needing to invoke any coupling between current and the Maxwell stress-tensor
as was done in Ref.Jubin  _et al._ (2018). Such a pressure-sensitivity cannot
be captured by recent analytic theories as they neglect flow entirelyDal
Cengio and Pagonabarraga (2019); Poggioli  _et al._ (2019). Furthermore we
find that the optimal cone geometry for rectification strongly depends on the
flow rate, while studies searching numerically for such a geometry often
neglect exactly this feature Kubeil and Bund (2011); Xiang  _et al._ (2022);
Ramirez  _et al._ (2008).

We consider two reservoirs of an aqueous 1:1 electrolyte in the two half
spaces $x<0$ and $x>L$ connected by an axially-symmetric cone-shaped channel
of length $L$. Here $x$ is the cartesian coordinate along the symmetry axis;
the radial coordinate is $r$. The channel has a wide base radius
$R_{\rm{b}}\ll L$ at $x=0$ and a narrow tip radius $R_{\rm{t}}\leq R_{\rm{b}}$
at $x=L$. The radius of the channel for $x\in[0,L]$ reads
$R(x)=R_{\rm{b}}-(x/L)(R_{\rm{b}}-R_{\rm{t}})$. The channel wall at $r=R(x)$
carries a uniform negative surface charge density $e\sigma$, with $e$ the
proton charge. The two reservoirs both contain an identical aqueous 1:1
electrolyte with viscosity $\eta$, ionic diffusion coefficient $D$, dielectric
permittivity $\epsilon$, and total ionic bulk concentration $2\rho_{\rm{b}}$.
Thus, asymptotically far from the channel, at either side $|x|/L\gg 1$, the
local cation concentration $\rho_{+}(x,r)$ and anion concentration
$\rho_{-}(x,r)$ are both equal to $\rho_{\rm{b}}$.

Inspired by the experiments of Ref. Jubin  _et al._ (2018) we consider an
applied pressure drop $\Delta P$ and a simultaneous electric potential drop
$\Delta\psi$ across the channel. These steady driving forces give rise to a
potential $\psi(x,r)$ and a pressure excess $P(x,r)-P_{0}$ which vanish in the
bulk phase $x\gg L$ and are equal to $\Delta\psi$ and $\Delta P$,
respectively, for $x\ll-L$, where $P_{0}$ is an arbitrary reference pressure.
They drive a fluid flow with velocity $\mathbf{u}(x,r)$ and ionic fluxes ${\bf
j}_{\pm}(x,r)$, leading to nontrivial concentration profiles
$\rho_{\pm}(x,r)$. In the Supplementary Material I (SM I) we present the
standard Poisson-Nernst-Planck-Stokes (PNPS) equations and the blocking and
no-slip boundary conditions. Together with Gauss’ law for the surface charge,
they form a closed set for ${\bf u}$, $\psi$, ${\bf j}_{\pm}$, and
$\rho_{\pm}$. Convenient linear combinations are the total salt concentration
$\rho_{\mathrm{s}}=\rho_{\mathrm{+}}+\rho_{\mathrm{-}}$, the charge density
$\rho_{\mathrm{e}}=\rho_{\mathrm{+}}-\rho_{\mathrm{-}}$, and the associated
fluxes $\mathbf{j}_{\mathrm{s}}=\mathbf{j}_{+}+\mathbf{j}_{-}$ and
$\mathbf{j}_{\mathrm{e}}=\mathbf{j}_{+}-\mathbf{j}_{-}$. In equilibrium, i.e.
for vanishing $\Delta P$ and $\Delta\psi$, all fluxes vanish and the PNPS
equations describe an Electric Double Layer (EDL) with an excess of cations
and a depletion of anions close to $r=R(x)$ such that the negative surface
charge is compensated Adamson and Gast (1967). The thickness of the EDL is
given by the Debye length $\lambda_{\rm{D}}=\sqrt{\epsilon
k_{\mathrm{B}}T/2e^{2}\rho_{\rm{b}}}=10$ nm for the case $\rho_{\rm{b}}=1$mM
that we consider.

Inspired by the experimental conditions of Ref. Jubin  _et al._ (2018), the
focus of this letter will be on the long-channel thin-EDL limit with $L\gg
R_{\rm{b}}\geq R_{\rm{t}}\gg\lambda_{\rm{D}}$ such that EDL-overlap does not
play a role. This is in contrast to a large body of literature on non-linear
transport in cone-shaped channels, where overlap of the EDL is a key
ingredient for current rectification and diodic behavior White and Bund
(2008a); Wen  _et al._ (2019); Proctor (2021). We will show that the conical
geometry combined with simultaneous pressure- and potential-induced transport
leads to an $x$-independent volumetric flow rate $Q=2\pi\hat{{\bf
x}}\cdot\int_{0}^{R(x)}{\bf u}(x,r)rdr$ and electric current $I=2\pi
e\hat{{\bf x}}\cdot\int_{0}^{R(x)}{\bf j}_{\rm{e}}(x,r)rdr$ that satisfy an
Onsager-like relation

$\begin{pmatrix}Q\\\ I\end{pmatrix}=\frac{\pi
R_{\rm{b}}R_{\rm{t}}}{L}\begin{pmatrix}\mathbb{L}_{11}&\mathbb{L}_{12}\\\\[6.0pt]
\mathbb{L}_{21}&\mathbb{L}_{22}(\Delta
P,\Delta\psi)\end{pmatrix}\begin{pmatrix}\Delta P\\\ \Delta\psi\end{pmatrix}.$
(1)

We set out to calculate all elements of the transport matrix $\mathbb{L}$
analytically, not only the permeability $\mathbb{L}_{11}$ and the electro-
osmotic mobility $\mathbb{L}_{12}=\mathbb{L}_{21}$, but also the electric
conductance $\mathbb{L}_{22}$ that, as we will see, strongly depends on the
applied pressure-and voltage drop -in agreement with experiments Jubin  _et
al._ (2018). This pressure sensitivity is due to highly nontrivial ion
concentration profiles that vary on length scales of the channel dimensions,
as follows from our analytic expression obtained from the PNPS equations. From
this we will find that optimal current rectification requires not only a
pressure drop $\Delta P^{*}=-\mathbb{L}_{12}\Delta\psi/\mathbb{L}_{11}$ (such
that $Q=0$) but also a universal optimal geometry with
$R_{\rm{t}}/R_{\rm{b}}\simeq 0.22$.

Figure 2: (a) Cross-sectional averaged salt concentration
$\bar{\rho}_{\rm{s}}(x)$ normalized by the bulk concentration $2\rho_{\rm{b}}$
as a function of the lateral position $x$ for our standard parameter set (see
text). For potential drops $\Delta\psi=+0.4$V (red) and $-0.4$V (blue), for
which $\Delta\rho=\mp 31$mM according to Eq. (5), the solid lines represent
concentration profiles at Péclet numbers that vary between 0 and $\mp 200$ in
steps of 20. The green curve represents the case $\Delta\psi=0$V at any Pe.
(b) The normalized laterally averaged concentration
$\langle\bar{\rho}_{\rm{s}}\rangle/2\rho_{\rm{b}}$ as a function of the
pressure drop $\Delta P$ at potential drops $\Delta\psi=+$0.4V (red), $-0.4$V
(blue), and $0$V (green). Solid lines represent Eq. (4), data points are from
numerical solutions to the full PNPS equations, which for $\Delta\psi=\pm
0.4$V show an extremum very close to $\Delta P^{*}=\mp 13$mbar from Eq. (6)
where Pe=0, denoted by the vertical dashed lines.

We solve the PNPS equations for a wide variety of system parameters and show
illustrative examples for the standard parameter set inspired by Ref. Jubin
_et al._ (2018) with tip radius $R_{\rm{t}}=0.17\mu$m, channel length
$L=10\mu$m, base radius $R_{\rm{b}}=1.04\mu$m, viscosity $\eta=1$ mPas,
dielectric constant 80 times vacuum permittivity, ionic diffusion coefficient
$D=1$ nm2/ns, and surface charge $e\sigma=-0.02e/$ nm2, which gives at
$\rho_{\rm{b}}=1$mM a zeta potential of $\psi_{0}=-40$mV corresponding to a
silica surface in contact with an aqueous 1:1 electrolyte Iler (1955). In line
with the Stokes equation we find ${\bf u}(x,r)$ to contain essentially two
contributions. (i) A pressure drop on its own induces a Poiseuille-like flow
that is directed towards the (virtual) vertex of the cone for $\Delta P>0$, or
away from it for $\Delta P<0$ Chun  _et al._ (2003); Happel and Brenner
(2012); Veenstra (2020). Its contribution $Q_{\rm{P}}\equiv(\pi
R_{\rm{b}}R_{\rm{t}}/L)\mathbb{L}_{11}\Delta P$ to $Q$ is independent of $x$
and can be obtained analytically Chun  _et al._ (2003); Happel and Brenner
(2012); Veenstra (2020) to yield
$\mathbb{L}_{11}=R_{\rm{b}}^{2}R_{\rm{t}}^{2}/8\eta\langle R^{2}\rangle$,
where the angular brackets denote a lateral average $\langle
R^{2}\rangle=\int_{0}^{L}R^{2}(x)\mathrm{d}x/L=(R_{\rm{b}}^{2}+R_{\rm{t}}^{2}+R_{\rm{b}}R_{\rm{t}})/3$.
The excellent agreement between the pressure-drop dependence of our linear
expression for $Q_{\rm{P}}$ and our numerically obtained value of $Q$ at
$\Delta\psi=0$ is shown in Fig.S1(a) of SM II. (ii) For our negative surface
charge the potential drop $\Delta\psi$ on its own induces an electro-osmotic
plug-like flow towards the tip of the cone for $\Delta\psi>0$, or away from
the tip for $\Delta\psi<0$. We are not aware of an explicit expression in the
literature for $\mathbb{L}_{12}$ that characterizes the electro-osmotic flow
rate $Q_{\rm{\psi}}\equiv(\pi
R_{\rm{b}}R_{\rm{t}}/L)\mathbb{L}_{12}\Delta\psi$ in a conical pore. Here we
derive an explicit expression for $\mathbb{L}_{12}$, which first requires an
expression for the cross-sectional averaged electric field
$-\partial_{x}\bar{\psi}(x)$, see Eq. (S1) of SM II, where
$\bar{\psi}(x)=2\pi\int_{0}^{R(x)}\psi(x,r)r\mathrm{d}r/\pi R^{2}(x)$. This
averaged electric field has to be proportional to the inverse of the cross
section $\pi R^{2}(x)$ in order to be divergence free. The proportionality
constant follows, in the long-channel limit, from the condition that
$\int_{0}^{L}\partial_{x}\bar{\psi}(x)\mathrm{d}x=-\Delta\psi$. This yields

$\partial_{x}\bar{\psi}(x)=-\frac{\Delta\psi}{L}\frac{R_{\rm{b}}R_{\rm{t}}}{R^{2}(x)},$
(2)

which compares well to the numerical results, as illustrated in Fig. S2 in SM
II. Using the standard electro-osmotic mobility
$\mathbb{L}_{12}=-\epsilon\psi_{0}/\eta$ for a cylinder Werkhoven and van Roij
(2020), but now with our laterally varying electric field and radius, we
obtain $Q_{\psi}=\pi
R^{2}(x)(-\epsilon\psi_{0}/\eta)\partial_{x}\bar{\psi}(x)$ which with Eq. (2)
is independent of $x$ and hence represents a valid divergence-free solution
for the stationary state. In Fig. S1(b) of SM II we compare this expression
for $Q_{\psi}$ as a function of $\Delta\psi$ with numerical calculations. The
agreement is good, although minor deviations on the order of $\sim 10\%$ are
visible which we attribute to the approximate nature of our $\mathbb{L}_{12}$.

With $\mathbb{L}_{11}$ and $\mathbb{L}_{12}$ established, we continue with
$\mathbb{L}_{22}$, for which the total ion concentration $\rho_{\rm{s}}(x,r)$
is expected to play a major role. In our numerical calculations we find weak
radial variation of $\rho_{\rm{s}}(x,r)$ outside the EDL-vicinity $r\simeq
R(x)$, in agreement with Ref. Lan  _et al._ (2011). Hence within the thin-EDL
limit this implies that the cross-sectional averaged concentration
$\bar{\rho}_{\rm{s}}(x)$ is a good proxy for the salt concentration at axial
position $x$. If we now define the total salt flux as $J(x)=2\pi\hat{{\bf
x}}\cdot\int_{0}^{R(x)}{\bf j}_{\rm{s}}(x,r)r\mathrm{d}r$ we can insert the
diffusive, conductive, and advective contributions of ${\bf j}_{\rm{s}}$ as
given by the PNPS equations in SM I to rewrite the stationarity condition
$\partial_{x}J(x)=0$ for $x\in[0,L]$ as

$\\!\\!\\!D\partial_{x}\left(\\!\\!\pi
R^{2}(x)\partial_{x}\bar{\rho}_{\rm{s}}(x)\\!-\\!2\pi
R(x)\sigma\frac{e\partial_{x}\bar{\psi}}{k_{\mathrm{B}}T}\\!\\!\right)\\!-\\!Q\partial_{x}\bar{\rho}_{\rm{s}}(x)=0.$
(3)

Here we use the radial independence of $\rho_{\rm{s}}(x,r)$ and $\psi(x,r)$ in
the thin-EDL limit as well as the slab-neutrality condition
$2\pi\int_{0}^{R(x)}\rho_{\rm{e}}(x,r)r\mathrm{d}r=-2\pi R(x)\sigma$ as
derived in SM II. The slab neutrality condition is an important difference
with the analysis presented in Ref.Jubin  _et al._ (2018), where it was
suggested that a bulk space charge is of key importance for understanding the
observed mechano-sensitivity of conical pores. For a given $\Delta P$ and
$\Delta\psi$ we consider $Q$ and $\partial_{x}\bar{\psi}(x)$ known from Eqs.
(1) and (2), respectively, such that Eq. (3) is an ordinary second-order
differential equation for $\bar{\rho}_{\rm{s}}(x)$; together with its
solutions presented below it constitutes the key result of this letter. An
important role will be played by the conductive contribution
$J_{\rm{cond}}(x)$ to $J$ given by $J_{\rm{cond}}(x)=-2\pi
D\sigma(e\Delta\psi/k_{\rm{B}}T)R_{\rm{b}}R_{\rm{t}}/R(x)L$, which varies with
$x$ in a conical channel and thus acts as a source or sink term in Eq. (3)
that sucks ions into the channel for $\Delta\psi<0$ and pushes them out for
$\Delta\psi>0$.

Given the long-channel limit of interest and the equal salinity of both
reservoirs, we solve Eq. (3) with boundary conditions
$\bar{\rho}_{\rm{s}}(0)=\bar{\rho}_{\rm{s}}(L)=2\rho_{\rm{b}}$, resulting in

$\displaystyle\bar{\rho}_{\rm{s}}(x)-2\rho_{\rm{b}}=\frac{\Delta\rho}{\text{Pe}}\Bigg{[}\frac{x}{L}\frac{R_{\rm{t}}}{R(x)}-\frac{\exp\left({\displaystyle\frac{x}{L}\frac{R_{\rm{t}}^{2}}{R_{\rm{b}}R(x)}\text{Pe}}\right)-1}{\exp\left({\displaystyle\frac{R_{\rm{t}}}{R_{\rm{b}}}\text{Pe}}\right)-1}\Bigg{]}$
(4)
$\displaystyle=\begin{dcases}\frac{\Delta\rho}{2}\frac{x}{L}\big{(}1-\frac{x}{L}\big{)}\frac{R_{\rm{t}}^{2}}{R^{2}(x)}&\text{if
\ \ |Pe|}\ \\!\ll\bigg{(}\frac{R_{\rm{b}}}{R_{\rm{t}}}\bigg{)}^{2};\\\
\dfrac{\Delta\rho}{2|\text{Pe}|}\bigg{(}\frac{R_{\rm{b}}}{R(x)}\big{(}1-\frac{x}{L}(1+\frac{R_{\rm{t}}}{R_{\rm{b}}})\big{)}\mp
1\bigg{)}&\text{if \
$\pm$Pe}\gg\bigg{(}\frac{R_{\rm{b}}}{R_{\rm{t}}}\bigg{)}^{2}.\end{dcases}$

Here we not only introduced the tip Péclet number $\text{Pe}\equiv QL/D\pi
R_{\rm{t}}^{2}$ with $Q(\Delta P,\Delta\psi)$ given by Eq. (1) but also a
measure for the concentration inhomogeneity

$\Delta\rho\equiv\frac{2(R_{\rm{b}}-R_{\rm{t}})\sigma}{R_{\rm{t}}^{2}}\frac{e\Delta\psi}{k_{\rm{B}}T},$
(5)

thus $\Delta\rho=0$ if $R_{\rm{b}}=R_{\rm{t}}$ and hence
$\bar{\rho}_{\rm{s}}(x)=2\rho_{\rm{b}}$ in this case. Note that both Pe and
$\Delta\rho$ have a sign, and that the dependence on the potential drop is not
only accounted for by $\Delta\rho$ but also by Pe through the electro-osmotic
contribution to $Q$, see Eq. (1). Clearly, Eq. (4) reveals concentration
variations on length scales on the order of the full channel length $0\leq
x\leq L$, most prominently for smaller $|\text{Pe}|$. Since the Péclet number
quantifies the importance of flow, we can now reconcile the discrepancy
between works which find electro-osmotic flow to be negligibleLan  _et al._
(2011) and others which find it to be importantLin  _et al._ (2015), as the
former concerns a parameter set with small Pe$\simeq
10^{-2}(R_{\rm{b}}/R_{\rm{t}})^{2}$ and the latter with large Pe$\simeq
3(R_{\rm{b}}/R_{\rm{t}})^{2}$. For $\Delta\psi=\pm 0.4$V, which for our
standard parameter set gives $\Delta\rho=\mp 31$mM from Eq. (5), we plot the
concentration profile $\bar{\rho}_{\rm{s}}(x)$ of Eq. (4) in Fig. 2(a) for
Péclet numbers between 0 and $\mp 200$. In Fig. 2(b) we plot the salt
concentration $\langle\bar{\rho}_{\rm{s}}\rangle$ laterally averaged over the
interval $x\in[0,L]$, which will play a key role in the electric conductivity
$\mathbb{L}_{22}$, as a function of the imposed pressure drop $\Delta P$ for
the three voltage drops $\Delta\psi=+0.4$V (red), 0V (green), and $-0.4$V
(blue), as obtained numerically from solutions of the PNPS equations (symbols)
and on the basis of a straightforward numerical integration of Eq. (4)
(lines). The agreement, although not perfect, is very good especially for
$\Delta P>0$. Our Eq. (4) not only correctly predicts the increase/decrease
compared to $2\rho_{\rm{b}}$ for a negative/positive potential drop but also
the non-monotonic dependence on $\Delta P$; the absolute difference with
$2\rho_{\rm{b}}$ is largest (and on the order of 30%) for $\Delta P\simeq\mp
10$mbar, which corresponds in both cases to $\text{Pe}\simeq 0$. The two
vertical dashed lines represent the pressure drop $\Delta
P^{*}=-(\mathbb{L}_{12}/\mathbb{L}_{11})\Delta\psi$, where $Q=0$ and hence
$\text{Pe}=0$ on the basis of Eq. (1), such that the optimal concentration
polarisation is to be expected. Collecting our earlier results we find the
optimal pressure drop per voltage drop

$\frac{\Delta
P^{*}}{\Delta\psi}=\epsilon\psi_{0}\frac{8(R_{\rm{b}}^{2}+R_{\rm{b}}R_{\rm{t}}+R_{\rm{t}}^{2})}{3R_{\rm{b}}^{2}R_{\rm{t}}^{2}}\stackrel{{\scriptstyle
R_{\rm{b}}\gg
R_{\rm{t}}}}{{\simeq}}\frac{8\epsilon\psi_{0}}{3R_{\rm{t}}^{2}},$ (6)

which yields about $-32\text{mbar}/\text{V}$ for our standard parameter set,
and about -27 mbar/V for the extremely large tip-base ratios $R_{\rm{b}}\gg
R_{\rm{t}}$ generated by the extrusion of a pipette in the experiments of Ref.
Jubin  _et al._ (2018) (if we assume $\psi_{0}=-40$mV common for silicaIler
(1955)). Clearly, the inverse square scaling of $\Delta P^{*}$ with
$R_{\rm{t}}$ is key to explaining the dramatic pressure sensitivity observed
in the experiments Jubin  _et al._ (2018). In fact, our results suggest even
more pressure sensitivity for larger conical channels, e.g. for $R_{\rm
t}=10\mu$m we have $\Delta P^{*}$ in the microbar regime, which can already be
exerted by the sound of passing traffic Khan  _et al._ (2010); Bies  _et al._
(2017). For cases where $\Delta P\gg\Delta P^{*}$, concentration polarization
is washed out by the flow; variation of current with both pressure and voltage
then closely follows Ohmic conduction. As flow suppresses diodic performance,
at a static pressure drop $\Delta P\neq\Delta P^{*}$ rectification can also be
increased by lowering the dielectric constant $\epsilon$ or increasing the
viscosity $\eta$ thereby lowering the electro-osmotic flow rate while keeping
$\Delta\rho$ unchanged.

Figure 3: (a) Heat map of the laterally averaged salt concentration
$\langle\bar{\rho_{\rm{s}}}\rangle$ for the standard parameter set (see text)
in the potential drop $\Delta\psi$ \- pressure drop $\Delta P$ plane. (b)
Current-pressure ($I-\Delta P$) relation for three fixed potential drops
showing a minimum close to $\Delta P=\Delta P^{*}$ of Eq. (6). Symbols
represent numerical solutions to the full PNPS equations at parameter
combinations shown in (a), solid lines represent our analytic solution based
on Eq. (1). (c) Current-voltage ($I-\Delta\psi$) relation for $\Delta P=0$
(pink) and the optimal pressure drop $\Delta P=\Delta P^{*}$ (black) which
shows increased current rectification ICR compared to $\Delta P=0$. The inset
shows the pressure drop dependence of the ICR=$-I(-1$V)/$I(1$V), which
exhibits two maxima at $\Delta P=\pm\Delta P^{*}$.

Now that we have established that Eq. (4) gives a fair account of the salt
concentration profile in the channel, we will use it to approximate
$\mathbb{L}_{22}$. In the thin-EDL limit the total current $I$ is dominated by
the conductive component
$-(De/k_{\rm{B}}T)\rho_{\rm{s}}(x,r)\partial_{x}\psi(x,r)$ of $\hat{{\bf
x}}\cdot e{\bf j}_{\rm{e}}(x,r)$ and cross-sectional integration of this
current with Eq. (2) and the same thin-EDL limit as before yields
$I_{\rm{cond}}(x)=\pi
R_{\rm{b}}R_{\rm{t}}eD\bar{\rho}_{\rm{s}}(x)(e\Delta\psi/k_{\rm{B}}TL)$, which
manifestly depends on $x$ on the basis of Eq. (4). In steady state this
lateral variation of the conductive current must be compensated by diffusive
and advective currents and the resulting laterally-invariant current $I$ can
be obtained by treating the concentration profile $\bar{\rho}_{\rm{s}}(x)$ as
a collection of resistors in series Werkhoven and van Roij (2020), such that
$\mathbb{L}_{22}=(De^{2}/k_{\rm{B}}T)\langle\bar{\rho}_{\rm{s}}\rangle$ which
reveals that conductance is proportional to the laterally averaged salt
concentration.

For our standard parameter set we plot $\langle\bar{\rho}_{\rm s}\rangle$ in
Fig. 3(a) as a heat map in the $(\Delta\psi,\Delta P)$ plane, including a few
iso-concentration contours. We clearly see the largest concentration
variations, and hence the largest variations of $\mathbb{L}_{22}$, along the
black line that represents $\Delta P=\Delta P^{*}$ of Eq. (6). In Fig. 3(b) we
plot the $\Delta P$-dependence of the electric current $I$ (lines) as
predicted from Eq. (1) for three voltage drops ($\pm 0.4$ V and zero),
together with full numerical calculations (symbols) at the state points
indicated by the color-matching symbols in (a) and Fig. 2(b). The overall
agreement is quantitative at $\Delta\psi=0$, which is fully in the linear-
response regime, while the nonlinear gross features at $\Delta\psi=\pm 0.4$V,
especially at $\Delta P\simeq\Delta P^{*}\simeq\mp 13$mbar, are accounted for
with reasonable accuracy, the more so at the positive potential drop. In Fig.
3(c) we plot current-voltage relations at pressure drops $\Delta P=0$ and
$\Delta P=\Delta P^{*}$, using the same color coding as in (a). The degree of
non-Ohmic behavior, characterised by the ionic current rectification
$\text{ICR}=-I(-1V)/I(1V)$ is clearly larger at the optimal pressure drop
$\Delta P^{*}$, which is indeed borne out by the inset which shows the full
$\Delta P$ dependence of ICR, revealing peaks at $\pm\Delta P^{*}$.

Finally, using our explicit knowledge of $\langle\bar{\rho}_{\rm{s}}\rangle$
and the full transport matrix of Eq. (1) we can explicitly search for an
optimal cone geometry at which the deviation from Ohmic conductance is
largest. Naively Eq.(5) suggests that for a large concentration profile the
ideal tip-to-base ratio should be small ($R_{\rm{t}}/R_{\rm{b}}\ll 1$),
however Eq.(4) shows that in this limit the concentration profile becomes
localized near the tip resulting in a small channel-averaged concentration
change. The ideal pore geometry balances the magnitude and spread of the
concentration profile and in SM III we show that this optimum occurs at a
universal tip-to-base ratio $R_{\rm{t}}/R_{\rm{b}}\simeq 0.22$ for
$|\mbox{Pe}|\lesssim 1$. The optimum ratio for concentration polarisation
decreases as the power law $b|\mbox{Pe}|^{-\nu}$ with $b=2.5$ and $\nu=0.9$
for $\mbox{Pe}\geq 10^{2}$, and $b=0.9$ and $\nu=0.55$ for
$\mbox{Pe}\leq-10^{2}$; for all flow rates the ideal tip-to-base ratio is less
than $0.22$. Interestingly, the ideal pore geometry is independent of the
channel length $L$, which follows from our Eq.(4) for $\bar{\rho}_{\rm{s}}(x)$
that only depends on $x/L$, such that $\mathbb{L}_{22}$ (and in fact the whole
matrix $\mathbb{L}$) is independent of the channel length. Hence the
concentration polarization does not depend on the cone opening angle, which is
surprising as most authors identify it as the key geometric parameter
controlling pressure-sensitivityJubin  _et al._ (2018) and current
rectificationKubeil and Bund (2011); Perry  _et al._ (2010).

In conclusion, we provide a full microscopic understanding of the ultra-
sensitive pressure- and voltage dependence of the electric conductivity of
cone-shaped channels. We identify, and quantify, concentration polarisation
due to geometric frustration which leads to a source term in Eq. (3), even in
the thin-EDL case considered here. Moreover, we found an optimal channel
geometry $R_{\rm{t}}/R_{\rm{b}}\simeq 0.22$ and an optimal operation condition
Eq. (6) for current rectification. These insights are important for further
developments of mechanotronic Cox  _et al._ (2019); Anishkin  _et al._ (2014);
Marcotte  _et al._ (2020) and biochemical Vlassiouk  _et al._ (2009); Bush
_et al._ (2020); Hou  _et al._ (2011) sensing as well as microfluidic Wang
_et al._ (2009); Squires and Quake (2005) and neuromorphic applications Li
_et al._ (2020); Robin  _et al._ (2021).

###### Acknowledgements.

This work is part of the D-ITP consortium, a program of the Netherlands
Organisation for Scientific Research (NWO) that is funded by the Dutch
Ministry of Education, Culture and Science (OCW).

## References

  * Zhang  _et al._ (2021) Z. Zhang, L. Wen, and L. Jiang, Nature Reviews Materials 6, 622 (2021).
  * Kim  _et al._ (2010) D.-K. Kim, C. Duan, Y.-F. Chen, and A. Majumdar, Microfluidics and Nanofluidics 9, 1215 (2010).
  * Siria  _et al._ (2013) A. Siria, P. Poncharal, A.-L. Biance, R. Fulcrand, X. Blase, S. T. Purcell, and L. Bocquet, Nature 494, 455 (2013).
  * Siria  _et al._ (2017) A. Siria, M.-L. Bocquet, and L. Bocquet, Nature Reviews Chemistry 1, 1 (2017).
  * Elimelech and Phillip (2011) M. Elimelech and W. A. Phillip, Science 333, 712 (2011).
  * Campione  _et al._ (2018) A. Campione, L. Gurreri, M. Ciofalo, G. Micale, A. Tamburini, and A. Cipollina, Desalination 434, 121 (2018).
  * Wang  _et al._ (2009) X. Wang, C. Cheng, S. Wang, and S. Liu, Microfluidics and Nanofluidics 6, 145 (2009).
  * Squires and Quake (2005) T. M. Squires and S. R. Quake, Reviews of Modern Physics 77, 977 (2005).
  * Bocquet and Charlaix (2010) L. Bocquet and E. Charlaix, Chemical Society Reviews 39, 1073 (2010).
  * Schoch  _et al._ (2008) R. B. Schoch, J. Han, and P. Renaud, Reviews of Modern Physics 80, 839 (2008).
  * Eijkel and Berg (2005) J. C. Eijkel and A. v. d. Berg, Microfluidics and Nanofluidics 1, 249 (2005).
  * Werkhoven  _et al._ (2018) B. L. Werkhoven, J. C. Everts, S. Samin, and R. van Roij, Physical Review Letters 120, 264502 (2018).
  * Werkhoven  _et al._ (2019) B. L. Werkhoven, S. Samin, and R. van Roij, The European Physical Journal Special Topics 227, 2539 (2019).
  * Ober  _et al._ (2021) P. Ober, W. Q. Boon, M. Dijkstra, E. H. Backus, R. van Roij, and M. Bonn, Nature Communications 12, 1 (2021).
  * Levine  _et al._ (1975) S. Levine, J. Marriott, G. Neale, and N. Epstein, Journal of Colloid and Interface Science 52, 136 (1975).
  * Sadeghi  _et al._ (2017a) M. Sadeghi, M. H. Saidi, and A. Sadeghi, Physics of Fluids 29, 062002 (2017a).
  * Sadeghi  _et al._ (2017b) M. Sadeghi, M. H. Saidi, A. Moosavi, and A. Sadeghi, Physics of Fluids 29, 122006 (2017b).
  * Sadeghi  _et al._ (2017c) M. Sadeghi, M. H. Saidi, and A. Sadeghi, Physics of Fluids 29, 062002 (2017c).
  * Laohakunakorn  _et al._ (2015) N. Laohakunakorn, V. V. Thacker, M. Muthukumar, and U. F. Keyser, Nano Letters 15, 695 (2015).
  * Lan  _et al._ (2016) W.-J. Lan, M. A. Edwards, L. Luo, R. T. Perera, X. Wu, C. R. Martin, and H. S. White, Accounts of Chemical Research 49, 2605 (2016).
  * Bishop  _et al._ (2015) G. W. Bishop, M. M. Lopez Jr, P. Ramiah Rajasekaran, X. Wu, and C. R. Martin, The Journal of Physical Chemistry C 119, 16633 (2015).
  * Asghar  _et al._ (2022a) Z. Asghar, M. Waqas, M. A. Gondal, and W. A. Khan, Alexandria Engineering Journal 61, 4519 (2022a).
  * Asghar  _et al._ (2022b) Z. Asghar, M. W. Saeed Khan, M. A. Gondal, and A. Ghaffari, Proceedings of the Institution of Mechanical Engineers, Part E: Journal of Process Mechanical Engineering , 09544089221097693 (2022b).
  * White and Bund (2008a) H. S. White and A. Bund, Langmuir 24, 2212 (2008a).
  * Wen  _et al._ (2019) C. Wen, S. Zeng, S. Li, Z. Zhang, and S.-L. Zhang, Analytical Chemistry 91, 14597 (2019).
  * Proctor (2021) J. E. Proctor,  _Theory of Ion Transport and Ion Current Rectification in Nanofluidic Diodes_ , thesis (2021).
  * Woermann (2003) D. Woermann, Physical Chemistry Chemical Physics 5, 1853 (2003).
  * Woermann (2004) D. Woermann, Physical Chemistry Chemical Physics 6, 3130 (2004).
  * Kovarik  _et al._ (2009) M. L. Kovarik, K. Zhou, and S. C. Jacobson, The Journal of Physical Chemistry B 113, 15960 (2009).
  * Lin  _et al._ (2018) C.-Y. Lin, L.-H. Yeh, and Z. S. Siwy, The Journal of Physical Chemistry Letters 9, 393 (2018).
  * Vlassiouk  _et al._ (2009) I. Vlassiouk, T. R. Kozel, and Z. S. Siwy, Journal of the American Chemical Society 131, 8211 (2009).
  * Bush  _et al._ (2020) S. N. Bush, T. T. Volta, and C. R. Martin, Nanomaterials 10, 571 (2020).
  * Hou  _et al._ (2011) X. Hou, W. Guo, and L. Jiang, Chemical Society Reviews 40, 2385 (2011).
  * Piruska  _et al._ (2010) A. Piruska, M. Gong, J. V. Sweedler, and P. W. Bohn, Chemical Society Reviews 39, 1060 (2010).
  * Gholinejad  _et al._ (2022) M. Gholinejad, A. Jabari Moghadam, and S. A. Mousavi Shaegh, Physics of Fluids 34, 012014 (2022).
  * Gholinejad  _et al._ (2021) M. Gholinejad, A. Jabari Moghadam, D.-T. Phan, A. K. Miri, and S. A. Mousavi Shaegh, Physics of Fluids 33, 051301 (2021).
  * Gholinejad  _et al._ (2020) M. Gholinejad, A. Jabari Moghadam, S. A. Mousavi Shaegh, and A. K. Miri, Physics of Fluids 32, 072012 (2020).
  * Gong  _et al._ (2022) Y. Gong, C. Zhang, X. Weng, B. Peng, and H. Jiang, Physics of Fluids 34, 022006 (2022).
  * Li  _et al._ (2020) C. Li, T. Xiong, P. Yu, J. Fei, and L. Mao, ACS Applied Bio Materials 4, 71 (2020).
  * Ghosal  _et al._ (2019) S. Ghosal, J. D. Sherwood, and H.-C. Chang, Biomicrofluidics 13, 011301 (2019).
  * Chun and Chung (2015) H. Chun and T. D. Chung, Annual Review of Analytical Chemistry 8, 441 (2015).
  * Jung  _et al._ (2022) J. Jung, J. Kim, J. Lee, Y.-W. Oh, S. Jung, I.-S. Kang, and K. Choi, AIP Advances 12, 035141 (2022).
  * Cox  _et al._ (2019) C. D. Cox, N. Bavi, and B. Martinac, Cell Reports 29, 1 (2019).
  * Anishkin  _et al._ (2014) A. Anishkin, S. H. Loukin, J. Teng, and C. Kung, Proceedings of the National Academy of Sciences 111, 7898 (2014).
  * Qian  _et al._ (2020) F. Qian, W. Zhang, D. Huang, W. Li, Q. Wang, and C. Zhao, Physical Chemistry Chemical Physics 22, 2386 (2020).
  * Jubin  _et al._ (2018) L. Jubin, A. Poggioli, A. Siria, and L. Bocquet, Proceedings of the National Academy of Sciences 115, 4063 (2018).
  * Pandey and Bhattacharyya (2021) D. Pandey and S. Bhattacharyya, Physics of Fluids 33, 062006 (2021).
  * Lan  _et al._ (2011) W.-J. Lan, D. A. Holden, and H. S. White, Journal of the American Chemical Society 133, 13300 (2011).
  * White and Bund (2008b) H. S. White and A. Bund, Langmuir 24, 2212 (2008b).
  * Kubeil and Bund (2011) C. Kubeil and A. Bund, The Journal of Physical Chemistry C 115, 7866 (2011).
  * Vlassiouk  _et al._ (2008) I. Vlassiouk, S. Smirnov, and Z. Siwy, ACS Nano 2, 1589 (2008).
  * Ramirez  _et al._ (2008) P. Ramirez, P. Y. Apel, J. Cervera, and S. Mafé, Nanotechnology 19, 315707 (2008).
  * Cervera  _et al._ (2006) J. Cervera, B. Schiedt, R. Neumann, S. Mafé, and P. Ramírez, The Journal of Chemical Physics 124, 104706 (2006).
  * Lin  _et al._ (2015) D.-H. Lin, C.-Y. Lin, S. Tseng, and J.-P. Hsu, Nanoscale 7, 14023 (2015).
  * Hsu  _et al._ (2017) J.-P. Hsu, S.-T. Yang, C.-Y. Lin, and S. Tseng, The Journal of Physical Chemistry C 121, 4576 (2017).
  * Ai  _et al._ (2010) Y. Ai, M. Zhang, S. W. Joo, M. A. Cheney, and S. Qian, The Journal of Physical Chemistry C 114, 3883 (2010).
  * Brown  _et al._ (2021) W. Brown, M. Kvetny, R. Yang, and G. Wang, The Journal of Physical Chemistry C 125, 3269 (2021).
  * Aarts  _et al._ (2022) M. Aarts, W. Boon, B. Cuénod, M. Dijkstra, R. van Roij, and E. Alarcon-Llado, arXiv preprint arXiv:2206.14594 (2022).
  * Choi  _et al._ (2016) E. Choi, C. Wang, G. T. Chang, and J. Park, Nano letters 16, 2189 (2016).
  * Dal Cengio and Pagonabarraga (2019) S. Dal Cengio and I. Pagonabarraga, The Journal of Chemical Physics 151, 044707 (2019).
  * Poggioli  _et al._ (2019) A. R. Poggioli, A. Siria, and L. Bocquet, The Journal of Physical Chemistry B 123, 1171 (2019).
  * Xiang  _et al._ (2022) F. Xiang, W. Zhang, X. Yang, and S. Liang, The Journal of Physical Chemistry B 126, 1779 (2022).
  * Adamson and Gast (1967) A. W. Adamson and A. P. Gast,  _Physical Chemistry of Surfaces_ , Vol. 150 (Interscience publishers New York, 1967).
  * Iler (1955) R. K. Iler,  _The colloid chemistry of silica and silicates_ , Vol. 80 (LWW, 1955).
  * Chun  _et al._ (2003) M.-S. Chun, S.-Y. Lee, and S.-M. Yang, Journal of Colloid and Interface Science 266, 120 (2003).
  * Happel and Brenner (2012) J. Happel and H. Brenner,  _Low Reynolds number hydrodynamics: with special applications to particulate media_ , Vol. 1 (Springer Science & Business Media, 2012).
  * Veenstra (2020) T. E. Veenstra,  _Strongly Non-Linear Pressure-Induced Ion Currents in Conical Nanopores_ , BSc thesis (2020).
  * Werkhoven and van Roij (2020) B. L. Werkhoven and R. van Roij, Soft Matter 16, 1527 (2020).
  * Khan  _et al._ (2010) M. W. Khan, M. A. Memon, M. N. Khan, and M. M. Khan, JLUMHS 9, 114 (2010).
  * Bies  _et al._ (2017) D. A. Bies, C. H. Hansen, and C. Q. Howard,  _Engineering noise control_ (CRC press, 2017).
  * Perry  _et al._ (2010) J. M. Perry, K. Zhou, Z. D. Harms, and S. C. Jacobson, ACS Nano 4, 3897 (2010).
  * Marcotte  _et al._ (2020) A. Marcotte, T. Mouterde, A. Niguès, A. Siria, and L. Bocquet, Nature Materials 19, 1057 (2020).
  * Robin  _et al._ (2021) P. Robin, N. Kavokine, and L. Bocquet, Science 373, 687 (2021).