[Appendix]tocatoc [toc] [atoc]

# L0onie: Compressing COINs with L0-constraints

Juan Ramirez Jose Gallego-Posada
{juan.ramirez<EMAIL_ADDRESS>
Mila and DIRO, Université de Montréal, Canada

###### Abstract

Advances in Implicit Neural Representations (INR) have motivated research on
domain-agnostic compression techniques. These methods train a neural network
to approximate an object, and then store the weights of the trained model. For
example, given an image, a network is trained to learn the mapping from pixel
locations to RGB values. In this paper, we propose L0onie, a sparsity-
constrained extension of the COIN compression method. Sparsity allows to
leverage the faster learning of overparameterized networks, while retaining
the desirable compression rate of smaller models. Moreover, our constrained
formulation ensures that the final model respects a pre-determined compression
rate, dispensing of the need for expensive architecture search.

##  1 Introduction

Implicit Neural Representations (INRs) train neural networks mapping
coordinates (e.g. pixel locations) to features (e.g. RGB values) in order to
approximate a given object. INRs have been applied to a wide range of data
modalities including audio [23], images [24], video [16], 3D scenes [19] and
temperature fields [7]. INRs provide a new perspective on data compression:
rather than dealing with the “raw” features of the object, train an INR to
approximate the object and store the parameters of the learned model. COIN [5]
pioneered this approach for image compression.

INR-based compression is a nascent technology. COIN exhibits sub-par
performance and more intensive computational cost compared to domain-specific
codecs. Moreover, it commonly requires architecture search to balance the
relationship between the model capacity and reconstruction quality. Although
these techniques are not yet competitive with established codecs for well-
studied domains like audio or image compression, the greater promise of
methods like COIN lies on their ability to provide compression standards that
can be applicable to virtually any data modality.

Recent works [6, 14, 22] focus on improving COIN via a meta-learning approach
in which a base network is (pre-)trained over a large collection of
datapoints, so that instance-dependent INRs can be found quickly. The task of
encoding of individual instances is casted as a search for modulations [21] on
the base architecture.

In this work we concentrate on the interplay between the model size, its
representational capacity and the required training time. In L0onie***Loonie
is a colloquial name for the Canadian one dollar coin., we combine
“overparameterized” COIN models with the sparse L0-reparametrization of
Louizos et al. [17]. This allows us to exploit the power of larger models to
achieve better reconstructions, faster; without having to commit to their
undesirable compression rate. We take advantage of the inherent redundancies
in these larger models, and sparsify them during training in order to achieve
a pre-specified compression rate. Our constrained formulation provides direct
control over the resulting compression rate and removes the need for costly
hyper-parameter tuning or architecture search. Fig. 1 provides an overview of
the compression behavior of of L0onie, COIN and JPEG at different BPP budgets.

Our code is available at: https://github.com/juan43ramirez/l0onie.

|  | 1.2 BPP | 0.6 BPP | 0.3 BPP
---|---|---|---|---
Original |  L0onie |  |  |
|  COIN |  |  |
|  JPEG |  |  |

Figure 1: Qualitative comparison between  L0onie,  COIN and  JPEG on a picture
of a loonie.

Figure 2: PSNR achieved by different techniques over the entire Kodak dataset.

##  2 Implicit Neural Representations

For simplicity, we present a description of INRs using terminology from image
processing. However, we highlight that INRs are in principle agnostic†††Note
that a choice of model architecture and activation functions might not be
suitable for all data modalities. to the “data type” at hand. Consider the
image $\mathbf{I}$ to be compressed as a collection of pixel coordinates
$\mathbf{x}_{p}\in[-1,1]^{2}$ and corresponding RGB pixel intensities
$\mathbf{y}_{p}\in[0,1]^{3}$, indexed over a discrete set of pixels
$\mathcal{P}$.

Consider a family of neural networks
$f_{\bm{\theta}}:\mathbb{R}^{2}\rightarrow\mathbb{R}^{3}$ with parameters
$\bm{\theta}$. An implicit neural representation (INR) of image $\mathbf{I}$
is a solution to the following supervised learning problem:

$\min_{\bm{\theta}}\mathcal{L}_{\mathbf{I}}(\bm{\theta})\triangleq\sum_{p\in\mathcal{P}}||f_{\bm{\theta}}(\mathbf{x}_{p})-\mathbf{y}_{p}||^{2}_{2}.$
(1)

Upon solving the optimization problem, the original image can be
(approximately) reconstructed by evaluating the function $f_{\bm{\theta}}$ at
the pixel locations $\mathcal{P}$. The choice of a family of models can have a
large influence in the reconstruction performance, and is an active area of
research. Recent proposals combine fully-connected models with positional
embeddings [25] or sine activation functions [23].

Dupont et al. [5] propose COIN as a creative approach for image compression:
when compressing an image $\mathbf{I}$, rather than storing RGB values, store
the weights of an INR of $\mathbf{I}$. We follow COIN’s choice of MLPs with
sinusoidal activation functions in this work.

The distortion-rate tradeoff in COIN involves balancing the higher expressive
capacity of large models with the detrimental effect of more model parameters
towards the compression rate. Achieving specific compression rates while
retaining low distortion may involve time-consuming architecture search.

###  2.1 Sparsifying INRs

Given that compressing images using COIN already requires more time than
standard methods like JPEG (by several orders of magnitude), dispensing of the
need for architecture search is an important step towards a wider adoption of
INR-based compression. Sparsity techniques are a natural approach to address
this issue.

Magnitude pruning can be used to attain specific compression rates, but
results in sub-optimal performance even after fine-tuning (see Section 3). The
assumptions made by heuristic-based sparsity methods popular in other machine
learning tasks might not transfer well to the context of INRs.‡‡‡E.g.
magnitude pruning relies on the idea that low-norm parameters should have a
small influence in the final prediction. This has been shown empirically for
image classification tasks where the model predicts a label among a discrete
number of classes [15, 8]. We hypothesize that the sub-par performance of
magnitude pruning in our experiments may be due to the use of sine activations
in the model and the continuous nature of the targets.

Louizos et al. [17] propose a framework for learning sparse models by means of
a differentiable reparametrization of the model weights
$\bm{\theta}=\bm{\tilde{\theta}}\odot\mathbf{z}$, where $\bm{\tilde{\theta}}$
are free (signed) parameter magnitudes, and $\mathbf{z}$ are stochastic gates
indicating whether a parameter is active. The gates follow a hard-concrete
distribution [17] with parameters $\bm{\phi}$. The authors then augment the
usual training objective with an additive penalty given by the expected
L0-norm of the gates $\mathbf{z}$, to encourage sparsity in the model.

We propose to combine the MLPs with sinusoidal activations of COIN with this
L0 reparametrization. This sparsity perspective opens the door for training
“overparameterized” models which can achieve better performance, faster [1].
Moreover, learning the sparsity pattern during training avoids having to
commit to the undesirably low compression rate of the fully dense model.

The same reparametrization is used in the concurrent work of Schwarz and Teh
[22], although stemming from a different motivation. While the authors provide
valuable insights regarding the integration of sparsity within a meta-learning
pipeline, they overlook the challenge of tuning the hyper-parameter
$\lambda_{\text{pen}}$, which dictates the relative importance of the sparsity
term. Adjusting $\lambda_{\text{pen}}$ to achieve a specific compression rate
can be as prohibitive as performing architecture search [2, 10].

Note that the stochastic re-parametrization induces a distribution over the
models. For practical reasons, it is convenient to have a single model at
decoding time. We use the gate medians $\hat{\mathbf{z}}(\bm{\phi})\in[0,1]$
for constructing said model. Due to the “stretching” in the hard-concrete
distribution, it is possible for the medians to be exactly 0 or 1 and not just
fractional. See Appendix A for more details.

###  2.2 L0-constrained formulation

Gallego-Posada et al. [10] argue that constrained formulations can provide
greater hyper-parameter interpretability and controllability when learning
sparse neural networks, compared to the commonly used penalized approach. The
authors consider constraints on the expected L0-norm of the parameters using
the reparametrization of Louizos et al. [17]. We extend their constrained
formulation by using proxy-constraints [4] to directly control the compression
rate of the resulting model (see Appendix B).

We consider the problem of finding a sparse INR for an image $\mathbf{I}$
given a constraint on the compression rate expressed in terms of a budget of
$\tau_{\texttt{BPP}}$ bits-per-pixel. We observed degraded performance when
training using Monte Carlo samples for the stochastic gates, as done in
Louizos et al. [17]. For this reason, we train a deterministic model using the
gate medians. This setting coincides with the decoding time model described
above.

Formally, we consider the following constrained optimization problem:

$\displaystyle\underset{\bm{\tilde{\theta}},\bm{\phi}}{\text{min}}\,\mathcal{L}_{\mathbf{I}}(\bm{\tilde{\theta}}\odot\hat{\mathbf{z}}(\bm{\phi}))\,\,\text{s.t.}\hskip
5.69054pt\texttt{BPP}(\texttt{cast}(\bm{\tilde{\theta}}\odot\hat{\mathbf{z}}(\bm{\phi}))\leq\tau_{\texttt{BPP}},$
(2)

where
$\texttt{BPP}(\texttt{cast}(\mathbf{u}))=\sum_{i}\mathbb{I}[\texttt{cast}(\mathbf{u}_{i})\neq
0]\cdot\texttt{bits}(\texttt{cast}(\mathbf{u}_{i}))$, and
$\texttt{cast}(\cdot)$ quantizes its input to a specified data type, such as
float16.

In practice, we optimize the Lagrangian associated with the problem in Eq. 2.
Let $\lambda_{\text{co}}\geq 0$ be the Lagrange multiplier corresponding to
the constraint. The min-max Lagrangian problem is given by:

$\bm{\tilde{\theta}}^{*},\bm{\phi}^{*},\lambda_{\text{co}}^{*}\triangleq\underset{\bm{\tilde{\theta}},\bm{\phi}}{\text{argmin}}\
\underset{\lambda_{\text{co}}\geq
0}{\text{argmax}}\,\,\mathcal{L}_{\mathbf{I}}(\bm{\tilde{\theta}}\odot\hat{\mathbf{z}}(\bm{\phi}))+\lambda_{\text{co}}\left(\texttt{BPP}(\texttt{cast}(\bm{\tilde{\theta}}\odot\hat{\mathbf{z}}(\bm{\phi}))-\tau_{\texttt{BPP}}\right).$
(3)

We apply simultaneous gradient descent on ($\bm{\tilde{\theta}},\bm{\phi}$)
and projected (to $\mathbb{R}^{+})$ gradient ascent on $\lambda_{\text{co}}$.
Note that unlike the penalized formulation in which the multiplicative factor
for the constraint is fixed, here it is dynamically adjusted throughout the
optimization. As detailed in Appendix B, the medians face differentiability
issues when they saturate at 0 or 1. Thus we employ the expected
$\texttt{BPP}(\bm{\tilde{\theta}}\odot\bar{\mathbf{z}}(\bm{\phi}))$ as a
proxy-constraint [4] for computing the gradient for the update of the primal
parameters.

Our constrained formulation grants a direct control over the compression rate
of the final model. This provides an algorithmic approach to retain the best
possible performance and achieve a specific target BPP, without having to
perform searches over the hyper-parameter $\lambda_{\text{pen}}$ or the model
architecture.

##  3 Experiments

We perform experiments on the Kodak image dataset [13] consisting of 24 images
of size 768 $\times$ 512\. We compare our approach with various image
compression techniques: COIN [5], unstructured magnitude pruning with fine-
tuning and JPEG [26]. Details on the implementation and hyper-parameter
configurations can be found in Appendix C. Comprehensive experiments and
qualitative comparisons are presented in Appendix D.

We make the difficulty of the task equal across techniques. For example, for a
BPP budget of 0.3, we train a COIN model of dimensions 2-10x[28]-3. Thus, COIN
models are fully dense and have a fixed BPP throughout training. Since L0onie
and magnitude pruning remove some of the parameters, we initialize them from
larger models and require that they deliver a final model with a BPP of 0.3.
The experimental settings used for each budget are provided in Appendix C.

Benchmark performance. Fig. 2 displays the performance of all methods at
various BPP budgets. Magnitude pruning consistently under-performs all other
approaches, both immediately after pruning (see Fig. 3) and after fine-tuning.
This is remarkable considering that we initialize magnitude pruning from a
larger, fully trained COIN model. L0onie slightly surpasses the performance
obtained by COIN. On the other hand, although JPEG struggles at high
compression rates it clearly dominates at larger BPPs; and its compression
time is negligible in comparison to the other techniques.

Higher performance, faster. Fig. 3 shows the PSNR of the compressed model as a
function of the training time. As expected, the larger L0onie model achieves
better reconstructions much faster. However, recall that the original L0onie
model has a worse compression rate. To ensure a fair comparison, we refresh
the PSNR meter for the L0onie experiments as soon as they satisfy the BPP
constraint – this is marked by a drop in the maximum PSNR metric.

Image 8 |  Image 14 |  Image 22
---|---|---
|  |
Figure 3: Performance comparison during runtime for  L0onie,  COIN and
magnitude pruning with a target of 0.3 BPP. All methods are trained for 50k
steps. Whenever the  L0onie model achieves the desired BPP, we reset its best
PSNR meter.

Once the L0onie models become feasible, they rapidly match and even surpass
the performance of COIN. Despite each individual gradient step taking more
time for the large L0onie models, the L0onie approach reaches a higher PSNR,
faster than COIN. Note that for the same wall-time budget as COIN, L0onie
achieves comparable or higher PSNR, while respecting the BPP constraint.

Training dynamics. Fig. 4 illustrates the training dynamics of all the
methods. We include COIN and magnitude pruning experiments as baselines, whose
BPP is fixed during training. These techniques exhibit a similar behavior,
with fast initial growth, followed by a slow saturation period.

Figure 4: Training dynamics for  L0onie,  COIN and  magnitude pruning on image
7 of the Kodak dataset with a target BPP of 0.3.

L0onie is initialized from a model with an unfeasible BPP. This causes the
Lagrange multiplier $\lambda_{\text{co}}$ to grow at the beginning of
training. In turn, the growing multiplier exerts a force that leads the
optimization to trade-off reconstruction performance in order to reduce the
model’s BPP. Note that this feasibility is obtained relatively early during
training. After the initial drop in PSNR, the now feasible model can focus on
optimizing the reconstruction performance. We observe that once feasible, the
PSNR for L0onie model quickly recovers and surpasses that of COIN.

Note that after reaching feasibility, the BPP statistics barely change for the
L0onie model. Thus optimization is effectively taking place over a particular
subnetwork. Pruning the network at this stage and “fusing” the gates and
weights could allow for more efficient training. These gains could be
particularly substantial when using structured sparsity, which was beyond the
scope of our work.

##  4 Limitations and Future work

Recent work has improved the efficiency of COIN by considering a meta-learning
framework [6, 14]. As demonstrated by Schwarz and Teh [22], sparsity and meta-
learning are complementary avenues for improving INR-based compression.
Despite the controllability advantages of our constrained approach, further
research is required for understanding the optimization dynamics of INRs, both
in the constrained and unconstrained settings.

For simplicity we concentrated on the unstructured sparsity case in this work.
However, the compression gains resulting from structured sparsity (for
example, grouping all weights associated with an input neuron under a shared
gate) could be more notorious and more easily materialized. This is
particularly important during training: once the INR becomes sparse, its
unused parameters can be removed to enable faster training.

Finally, our sparsity-constrained formulation can support progressive
sparsification of the model to reach different compression rates. Once a model
has achieved a desired BPP, this very same model can be used in a
straightforward manner as a starting point to obtain INRs with higher
compression rates. It is not immediately evident how to achieve this
progressive sparsification using vanilla COIN models or penalized L0
formulations, while reliably controling the resulting compression rate.

##  5 Conclusion

We propose L0onie, a sparse extension of COIN models trained using a
constrained formulation. Our method allows to leverage the faster learning of
overparameterized networks, while respecting a desired compression rate,
without requiring costly hyper-parameter tuning or architecture search.

## Acknowledgements

JGP was funded by the Canada CIFAR AI Chair Program and by an IVADO Excellence
Scholarship.

Fig. 1 depicts a circulating unit of currency. Its use in this work contends
as fair use for it is limited to a commentary relating to the image of the
currency itself. The image of all Canadian coins remains the copyright of the
Royal Canadian Mint.

## References

  * Arora et al. [2018] S. Arora, N. Cohen, and E. Hazan.  On the Optimization of Deep Networks: Implicit Acceleration by Overparameterization.  In _ICML_ , 2018.
  * Boyd and Vandenberghe [2004] S. Boyd and L. Vandenberghe.  _Convex Optimization_.  Cambridge University Press, 2004.
  * Bégaint et al. [2020] J. Bégaint, F. Racapé, S. Feltman, and A. Pushparaja.  CompressAI: a PyTorch library and evaluation platform for end-to-end compression research.  _arXiv:2201.12904_ , 2020.
  * Cotter et al. [2019] A. Cotter, H. Jiang, M. Gupta, S. Wang, T. Narayan, S. You, and K. Sridharan.  Optimization with Non-Differentiable Constraints with Applications to Fairness, Recall, Churn, and Other Goals.  In _JMLR_ , 2019.
  * Dupont et al. [2021] E. Dupont, A. Golinski, M. Alizadeh, Y. W. Teh, and A. Doucet.  COIN: COmpression with Implicit Neural representations.  In _ICLR - Neural Compression Workshop_ , 2021.
  * Dupont et al. [2022a] E. Dupont, H. Loya, M. Alizadeh, A. Goliński, Y. W. Teh, and A. Doucet.  COIN++: Data Agnostic Neural Compression.  _arXiv:2201.12904_ , 2022a.
  * Dupont et al. [2022b] E. Dupont, Y. W. Teh, and A. Doucet.  Generative Models as Distributions of Functions.  In _AISTATS_ , 2022b.
  * Gale et al. [2019] T. Gale, E. Elsen, and S. Hooker.  The State of Sparsity in Deep Neural Networks.  In _NeurIPS - Workshop in ODML-CDNNR_ , 2019.
  * Gallego-Posada and Ramirez [2022] J. Gallego-Posada and J. Ramirez.  Cooper: a toolkit for Lagrangian-based constrained optimization.  https://github.com/cooper-org/cooper, 2022.
  * Gallego-Posada et al. [2021] J. Gallego-Posada, J. Ramirez, and A. Erraqabi.  Flexible Learning of Sparse Neural Networks via Constrained $L_{0}$ Regularization.  In _NeurIPS - LatinX in AI Workshop_ , 2021.
  * Jang et al. [2017] E. Jang, S. Gu, and B. Poole.  Categorical Reparameterization with Gumbel-Softmax.  In _ICLR_ , 2017.
  * Kingma and Ba [2015] D. P. Kingma and J. Ba.  Adam: A Method for Stochastic Optimization.  In _ICLR_ , 2015.
  * Kodak [1991] Kodak.  Kodak dataset.  http://r0k.us/graphics/kodak/, 1991.
  * Lee et al. [2021] J. Lee, J. Tack, N. Lee, and J. Shin.  Meta-Learning Sparse Implicit Neural Representations.  In _NeurIPS_ , 2021.
  * Li et al. [2017] H. Li, A. Kadav, I. Durdanovic, H. Samet, and H. P. Graf.  Pruning Filters for Efficient ConvNets.  In _ICLR_ , 2017.
  * Li et al. [2021] Z. Li, S. Niklaus, N. Snavely, and O. Wang.  Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes.  In _CVPR_ , 2021.
  * Louizos et al. [2018] C. Louizos, M. Welling, and D. P. Kingma.  Learning Sparse Neural Networks through $L_{0}$ Regularization.  In _ICLR_ , 2018.
  * Maddison et al. [2017] C. J. Maddison, A. Mnih, and Y. W. Teh.  The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables.  In _ICLR_ , 2017.
  * Mescheder et al. [2019] L. Mescheder, M. Oechsle, M. Niemeyer, S. Nowozin, and A. Geiger.  Occupancy Networks: Learning 3d Reconstruction in Function Space.  In _CVPR_ , 2019.
  * Paszke et al. [2019] A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala.  PyTorch: An Imperative Style, High-Performance Deep Learning Library, 2019.
  * Perez et al. [2017] E. Perez, F. Strub, H. de Vries, V. Dumoulin, and A. Courville.  FiLM: Visual Reasoning with a General Conditioning Layer.  In _AAAI_ , 2017.
  * Schwarz and Teh [2022] J. R. Schwarz and Y. W. Teh.  Meta-Learning Sparse Compression Networks.  _arXiv:2205.08957_ , 2022.
  * Sitzmann et al. [2020] V. Sitzmann, J. Martel, A. Bergman, D. Lindell, and G. Wetzstein.  Implicit Neural Representations with Periodic Activation Functions.  In _NeurIPS_ , 2020.
  * Stanley [2007] K. O. Stanley.  Compositional Pattern Producing Networks: A Novel Abstraction of Development.  _Genetic Programming and Evolvable Machines_ , 8(2):131–162, 2007.
  * Tancik et al. [2020] M. Tancik, P. P. Srinivasan, B. Mildenhall, S. Fridovich-Keil, N. Raghavan, U. Singhal, R. Ramamoorthi, J. T. Barron, and R. Ng.  Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains.  In _NeurIPS_ , 2020.
  * Wallace [1992] G. Wallace.  The JPEG still picture compression standard.  _IEEE Transactions on Consumer Electronics_ , 38(1):xviii–xxxiv, 1992.

##  Appendix A Stochastic Gates

Let $U_{j}\sim\text{Unif}(0,1)$ and $0<\beta<1$. A concrete random variable
[18, 11] $s_{j}\sim q(\cdot\ |\ (\phi_{j},\beta))$ can be obtained by
transforming the uniform random variable as:

$s_{j}=\text{Sigmoid}\left(\frac{1}{\beta}\log\left(\frac{\phi_{j}\,U_{j}}{1-U_{j}}\right)\right).$
(4)

Given hyper-parameters $\gamma<0<1<\zeta$, the hard concrete distribution [17]
corresponds to a stretching and clamping of a concrete random variable. The
hard concrete distribution is a mixed distribution with point masses at $0$
and $1$, and a continuous density over $(0,1)$.

$\hskip
14.22636pt\mathbf{z}=\text{clamp}_{[0,1]}(\mathbf{s}(\zeta-\gamma)+\gamma))$
(5)

Table 1 specifies the values of the fixed parameters associated with the gates
employed throughout this work, following Louizos et al. [17].

Table 1: Fixed parameters of the hard concrete distribution. Parameter | $\gamma$ | $\zeta$ | $\beta$
---|---|---|---
Value | -0.1 | 1.1 | 2/3

We use $\mathbf{z}$ for modeling the stochastic gates of L0onie models. The
stochastic nature of hard concrete variables entails a model which is itself
_stochastic_. For computing forwards and BPP measurements, we replace each
gate by its median $\hat{\mathbf{z}}(\bm{\phi})$:

$\hat{\mathbf{z}}(\bm{\phi})=\text{min}\left(1,\text{ max}\left(0,\
\text{Sigmoid}\left(\frac{\text{log}(\bm{\phi})}{\beta}\right)(\zeta-\gamma)+\gamma\right)\right)$
(6)

Gate medians are a deterministic function of the trainable parameter
$\bm{\phi}$. Moreover, the stretching and clamping enables the medians to
attain the values 0 or 1, thus producing a sparse network.

However, gate medians are poorly suited for setting up the BPP constraint as
they do not allow for change once a gate is “fully turned off or on”; once the
median is zero, the gradient with respect to $\phi_{j}$ is zero. Hence, we use
expected BPP of the model as a surrogate for computing gradients. Note that
this quantity is differentiable with respect to $\phi_{j}$.

$\mathbb{E}_{\mathbf{z}|\bm{\phi}}\left[\
\texttt{BPP}(\bm{\tilde{\theta}}\odot\mathbf{z})\right]=\sum_{i}\texttt{bits}(\bm{\tilde{\theta}}_{i}\odot\mathbf{z}_{j})\
\mathbb{P}[z_{i}\neq
0]=\sum_{i}\texttt{bits}(\bm{\tilde{\theta}}_{i})\,\text{Sigmoid}\left(\log(\phi_{i})-\beta\log\frac{-\gamma}{\zeta}\right)$

##  Appendix B Proxy-constraints

Let us recall the min-max Lagrangian optimization problem from (3)

$\bm{\tilde{\theta}}^{*},\bm{\phi}^{*},\lambda_{\text{co}}^{*}\triangleq\underset{\bm{\tilde{\theta}},\bm{\phi}}{\text{argmin}}\
\underset{\lambda_{\text{co}}\geq
0}{\text{argmax}}\,\,\mathcal{L}_{\mathbf{I}}(\bm{\tilde{\theta}}\odot\hat{\mathbf{z}}(\bm{\phi}))+\lambda_{\text{co}}\left(\texttt{BPP}(\texttt{cast}(\bm{\tilde{\theta}}\odot\hat{\mathbf{z}}(\bm{\phi}))-\tau_{\texttt{BPP}}\right).$

Applying simultaneous gradient descent-ascent on this problem corresponds to
the update scheme:

$\displaystyle[\bm{\tilde{\theta}}^{t+1},\bm{\phi}^{t+1}]$
$\displaystyle\triangleq[\bm{\tilde{\theta}}^{t},\bm{\phi}^{t}]\
{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}-}\
\eta_{\text{primal}}\,\nabla_{[\bm{\tilde{\theta}},\bm{\phi}]}\left[\mathcal{L}_{\mathbf{I}}(\bm{\tilde{\theta}}\odot\hat{\mathbf{z}}(\bm{\phi}))+\lambda_{\text{co}}\texttt{BPP}(\texttt{cast}(\bm{\tilde{\theta}}\odot\hat{\mathbf{z}}(\bm{\phi}))\right]$
$\displaystyle\lambda_{\text{co}}^{t+1}$
$\displaystyle\triangleq\max\left(0,\lambda_{\text{co}}^{t}\
{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}+}\
\eta_{\text{dual}}\,\left(\texttt{BPP}(\texttt{cast}(\bm{\tilde{\theta}}\odot\hat{\mathbf{z}}(\bm{\phi}))-\tau_{\texttt{BPP}}\right)\right)$

The gradient update for $\lambda_{\text{co}}$ matches the value of constraint
violation. Whenever the constraint is not satisfied, the Lagrange multiplier
increases. We employ the dual restarts technique of Gallego-Posada et al. [10]
when the constraint is satisfied.

Note that primal update involves computing the gradient of the
$\texttt{BPP}(\texttt{cast}(\bm{\tilde{\theta}}\odot\hat{\mathbf{z}}(\bm{\phi}))$
with respect to the parameters $\bm{\tilde{\theta}}$ and $\bm{\phi}$. As
mentioned in Appendix A, this quantity is not differentiable when the gate
medians attain the values 0 or 1. To overcome this issue we employ the
expected BPP as a surrogate or proxy-constraint [4, §4.2] when computing the
gradient for the primal update. The original (non-proxy) constraint is used
for updating the value of the Lagrange multiplier.

##  Appendix C Experimental Details

Our implementation is developed in Python 3.8, using Pytorch 1.11 [20] and the
Cooper constrained optimization library [9]. We provide scripts to replicate
our experiments at: https://github.com/juan43ramirez/l0onie.

###  C.1 Codec Baselines

The baseline results for the JPEG [26] standard were obtained using the
CompressAI library [3].

###  C.2 Model Architectures

We considered the same setup as Dupont et al. [5]: MLPs with 2 input
dimensions corresponding to the position of input pixels normalized to lie in
$[-1,1]$ and 3 outputs corresponding to RGB values normalized to lie in
$[0,1]$. These models are trained at single (float32) precision.

All our models use Sine activations at every layer except for the output
layer. These are setup with a fixed angular frequency of $\omega_{0}=30$.

When replicating the experiments of Dupont et al. [5], we consider specific
architectures so as to yield bits-per-pixel of $0.07$, $0.15$, $0.3$ and $0.6$
at half (float16) precision. Table 2 contains the hidden layers associated
with each of these architectures.

Hidden layers | Width of Layers | BPP @ float32 | BPP @ float16
---|---|---|---
5 | 20 | 0.15 | 0.07
5 | 30 | 0.3 | 0.15
10 | 28 | 0.6 | 0.3
10 | 40 | 1.2 | 0.6
10 | 40 | 1.62 | 0.81
Table 2: Architectures used throughout this paper, along with their respective
bits-per-pixel when used to compress an image of 768 $\times$ 512 pixels. BPPs
are calculated for each architecture at single (float32) and half (float16)
precision.

For L0onie and magnitude pruning experiments, we consider unstructured
sparsity, where each parameter can be pruned individually. We also consider
target BPPs of compressed models at the same levels of $0.07$, $0.15$, $0.3$
and $0.6$ used to evaluate the COIN approach. Furthermore, the initial (dense)
architectures which are sparsified by L0onie and magnitude pruning to meet
these target BPPs are also based on the architectures presented in Table 2. We
consider the next larger architecture from the one whose BPP matches the one
targeted. This is illustrated in Table 3.

Target BPP (@ float16) | Initial Architecture
---|---
BPP (@ float16) | Hidden Layers | Width of Layers
0.07 | 0.15 | 5 | 30
0.15 | 0.3 | 10 | 28
0.3 | 0.6 | 10 | 40
0.6 | 0.81 | 13 | 40
Table 3: Initial architectures considered when applying L0onie and magnitude
pruning to meet various target BPPs. Note that the initial architecture has a
larger BPP than the desired target.

###  C.3 Magnitude Pruning

All magnitude pruning experiments follow the procedure presented in this
section. First, we train a COIN model at a certain (larger) BPP. This serves
as a baseline which can then be pruned to achieve a lower BPP. The selection
of the initial architecture depends on the provided target BPP. The details
about this choice are presented in Table 3.

Thereafter, we loop over the layers of the baseline model, sorting their
individual parameters based on magnitude. A suitable proportion of the
parameters with the smaller magnitude is set to 0. We perform the sorting and
pruning separately for weight matrices and biases.

We tried two variants for maginitude pruning: \raisebox{-.8pt}{{\hspace{-1mm}
\small1}}⃝ applying the same level of pruning for all the layers, or
\raisebox{-.8pt}{{\hspace{-1mm} \small2}}⃝ keeping the first and last layers
fully dense. This last technique is commonly used in conjunction with
magnitude pruning, and is particularly important in the setting of this work
since the number of input dimensions is very low.

The results of method \raisebox{-.8pt}{{\hspace{-1mm} \small1}}⃝ were
significantly and consistently worse than those of method
\raisebox{-.8pt}{{\hspace{-1mm} \small2}}⃝. The results reported in this paper
correspond to method \raisebox{-.8pt}{{\hspace{-1mm} \small2}}⃝.

Immediately after pruning we evaluate the performance of the model in terms of
PSNR. We identified a very significant degradation in performance and thus
perform fine-tuning on the remaining parameters using the same number of
iterations as the original training for the COIN baseline. The optimization
hyper-parameters employed during this stage are presented in C.5.

###  C.4 Parameter Initialization

We use the same method proposed by [17] for initializing the learnable
parameters $\bm{\phi}$. We leverage this scheme to ensure that the expected
value of stochastic gates is centered around $0.5$ at initialization.

The network’s weights and biases are initialized in accordance to their use of
sine activation functions, following the procedure described in [23]. This
involves setting them based on a uniform distribution in $[-a,a]$. As
mentioned previously, for L0onie experiments, the gate distributions are
initialized to be symmetric around 0.5, thus shrinking the effective value of
parameters to lie in $[-a/2,a/2]$. We counter this by adjusting the
initialization of weights and biases of L0onie models to lie in $[-2a,2a]$,
thus having effective values of parameters in the desired range.

###  C.5 Optimization Hyper-Parameters

This section presents the optimization hyper-parameters used throughout our
work. Table 4 indicates those considered when training COIN models and for the
fine-tuning of models after magnitude pruning. These are the same across
different images and architectures. Both cases employ the same configuration
for the optimizer: Adam [12] with $(\beta_{1},\ \beta_{2})=(0.9,\ 0.99)$, as
is default in Pytorch.

Table 4: Optimization hyper-parameters employed when training COIN baselines and when fine-tuning models which have been magnitude pruned. Approach | Train Precision | Decode Precision | Steps | Optim. | $\eta_{\text{primal}}^{\bm{\tilde{\theta}}}$
---|---|---|---|---|---
COIN | float32 | float16 | 50.000 | Adam | $2\cdot 10^{-4}$
MP fine-tuning

Note that the constraint functions considered for L0onies involve expectations
but can be computed in closed-form based on the parameters of the gates, as
presented in Appendix A. Therefore, the computation of constraint violations
is deterministic. We employ gradient ascent on the Lagrange multipliers. We
initialize all Lagrange multipliers at zero. Details on the chosen dual
learning rate, along with the use of dual restarts for L0onie experiments are
presented in Table 5.

We use the same configurations across all Kodak images for a given target BPP.
Since our goal is to “overfit” the model to the image, we do not use weight
decay or other regularization techniques.

Table 5: Optimization hyper-parameters employed when training L0onie models.

Target | Initial | Weights | Gates | Lagrange Multipliers
---|---|---|---|---
BPP | Architecture | Optim. | $\eta_{\text{primal}}^{\bm{\tilde{\theta}}}$ | Optim. | $\eta_{\text{primal}}^{\bm{\phi}}$ | Optim. | $\eta_{\text{dual}}$ | Restarts
0.07 | 2 - 5$\times$[30] - 3 | Adam | $10^{-3}$ | Adam | $7\cdot 10^{-4}$ | Grad. Ascent | $7\cdot 10^{-3}$ | Yes
0.15 | 2 - 10$\times$[28] - 3 | $3\cdot 10^{-3}$
0.3 | 2 - 10$\times$[40] - 3 | $1\cdot 10^{-3}$
0.6 | 2 - 13$\times$[40] - 3 | $8\cdot 10^{-4}$

##  Appendix D Additional Results

We evaluate the performance of our approach when compressing each image in the
Kodak dataset at 0.07, 0.15, 0.3 and 0.6 BPPs. In addition, we did the same
for COIN and magnitude pruning plus fine-tuning. Fig. 5 presents the best
PSNRs we obtained.

Magnitude pruning always underperforms as opposed to L0onie and COIN. This gap
is more pronounced at 0.07 BPP, but is consistent across images. Moreover, the
performance of L0onie is generally competitive to that of COIN, whilst being
slightly superior in the 0.15 and 0.3 BPP cases.

0.07 BPP |
---|---
0.15 BPP |
0.3 BPP |
0.6 BPP |
Figure 5: Histogram of best PSNR for all images in the Kodak dataset for
L0onie,  COIN and  magnitude pruning at different compression rates. Images
sorted in decreasing order of maximum PSNR for  L0onieat the end of trainig.

##  Appendix E Qualitative Results

We provide qualitative comparisons for some images of the Kodak dataset of
different levels of compression difficulty, according to the histograms in
Fig. 5. The image grids below are generated from reconstructions at various
BPP budgets for  L0onie,  COIN,  magnitude pruning and  JPEG. The experimental
configuration for these experiments is provided in Appendix C.

|  |  Original |
---|---|---|---
L0onie |  |  |  |
COIN |  |  |  |
MP |  |  |  |
JPEG |  |  |  |
| 0.07 BPP | 0.15 BPP | 0.3 BPP | 0.6 BPP
Figure 6: Reconstruction of Kodak image 23 for all methods at different target BPP. |  |  Original |
---|---|---|---
L0onie |  |  |  |
COIN |  |  |  |
MP |  |  |  |
JPEG |  |  |  |
| 0.07 BPP | 0.15 BPP | 0.3 BPP | 0.6 BPP
Figure 7: Reconstruction of Kodak image 15 for all methods at different target BPP. |  |  Original |
---|---|---|---
L0onie |  |  |  |
COIN |  |  |  |
MP |  |  |  |
JPEG |  |  |  |
| 0.07 BPP | 0.15 BPP | 0.3 BPP | 0.6 BPP
Figure 8: Reconstruction of Kodak image 8 for all methods at different target BPP. |  |  Original |
---|---|---|---
L0onie |  |  |  |
COIN |  |  |  |
MP |  |  |  |
JPEG |  |  |  |
| 0.07 BPP | 0.15 BPP | 0.3 BPP | 0.6 BPP
Figure 9: Reconstruction of Kodak image 2 for all methods at different target
BPP.


# A Direct Method of Moving Planes for Logarithmic Schrödinger Operator

Rong Zhang, Vishvesh Kumar and Michael Ruzhansky  School of Mathematical
Sciences, Nanjing Normal University, Nanjing, 210023, China
and
Department of Mathematics: Analysis, Logic and Discrete Mathematics, Ghent
University, Ghent, Belgium<EMAIL_ADDRESS>/<EMAIL_ADDRESS>Department of Mathematics: Analysis, Logic and Discrete Mathematics, Ghent
University, Ghent, Belgium<EMAIL_ADDRESS>/<EMAIL_ADDRESS>Department of Mathematics: Analysis, Logic and Discrete Mathematics, Ghent
University, Ghent, Belgium
and
School of Mathematical Sciences, Queen Mary University of London, United
Kingdom<EMAIL_ADDRESS>
###### Abstract.

In this paper, we study the radial symmetry and monotonicity of nonnegative
solutions to nonlinear equations involving the logarithmic
Schr$\ddot{\text{o}}$dinger operator $(\mathcal{I}-\Delta)^{\log}$
corresponding to the logarithmic symbol $\log(1+|\xi|^{2})$, which is a
singular integral operator given by

$(\mathcal{I}-\Delta)^{\log}u(x)=c_{N}P.V.\int_{\mathbb{R}^{N}}\frac{u(x)-u(y)}{|x-y|^{N}}\kappa(|x-y|)dy,$

where $c_{N}=\pi^{-\frac{N}{2}}$,
$\kappa(r)=2^{1-\frac{N}{2}}r^{\frac{N}{2}}\mathcal{K}_{\frac{N}{2}}(r)$ and
$\mathcal{K}_{\nu}$ is the modified Bessel function of second kind with index
$\nu$. The proof hinges on a direct method of moving planes for the
logarithmic Schr$\ddot{\text{o}}$dinger operator.

###### Key words and phrases:

Logarithmic Schr$\ddot{\text{o}}$dinger operator; Symmetry and monotonicity;
The direct method of moving planes; Logarithmic symbol.

###### 1991 Mathematics Subject Classification:

35R11, 35B06, 35B50, 35B51, 35D30.

##  1\. Introduction.

The study of Schrödinger equations received a great deal of attention from
researchers in the past decades because of its vast applications in several
areas of mathematics and mathematical physics. In particular, Schrödinger
equations arise in quantum field theory and in the Hartree-Fock theory (see
[5, 15, 16, 19]). Recently, there is a surge of interest to investigate
integrodifferential operators of order close to zero and associated linear and
nonlinear integrodifferential equations (see [1, 4, 12, 14, 17, 18]). In
particular, the logarithmic Laplacian and the logarithmic
Schr$\ddot{\text{o}}$dinger operator are two interesting examples of such
class of operators. The logarithmic Schr$\ddot{\text{o}}$dinger operator
$(\mathcal{I}-\Delta)^{\log}$ (see [10]) and the logarithmic Laplacian
$L_{\Delta}$ (see [1]) have the similar behavior locally concerning to the
singularity of kernels but the logarithmic Schr$\ddot{\text{o}}$dinger
operator eliminates the integrability problem of the logarithmic Laplacian at
infinity. To define the logarithmic Schr$\ddot{\text{o}}$dinger operator, let
us begin with the following observation

$\lim_{s\rightarrow 0^{+}}(\mathcal{I}-\Delta)^{s}u(x)=u(x),\ \text{for}\ u\in
C^{2}(\mathbb{R}^{N}),$ (1.1)

where for $s\in(0,1)$, the operator $(\mathcal{I}-\Delta)^{s}$ stands for the
relativistic Schr$\ddot{\text{o}}$dinger operator, for sufficiently regular
function $u:\mathbb{R}^{N}\rightarrow\mathbb{R}$, which can be represented via
hypersingular integral (1.1) (see [8]),

$(\mathcal{I}-\Delta)^{s}u(x)=u(x)+c_{N,s}\lim_{\epsilon\rightarrow
0^{+}}\int_{\mathbb{R}^{N}\setminus
B_{\epsilon}(0)}\frac{u(x)-u(y)}{|x-y|^{N+2s}}\varpi_{s}(|x-y|)dy,$ (1.2)

where $c_{N,s}=\frac{\pi^{-\frac{N}{2}}4^{s}}{\Gamma(-s)}$ is a normalization
constant and the function $\varpi_{s}$ is given by

$\displaystyle\varpi_{s}(r)$
$\displaystyle=2^{1-\frac{N+2s}{2}}r^{\frac{N+2s}{2}}\mathcal{K}_{\frac{N+2s}{2}}(r)=\int_{0}^{+\infty}t^{-1+\frac{N+2s}{2}}e^{-t-\frac{r^{2}}{4t}}dt.$
(1.3)

Furthermore, if $u\in C^{2}(\mathbb{R}^{N})$, then
$(\mathcal{I}-\Delta)^{s}u(x)$ is well defined by (1.2) for every
$x\in\mathbb{R}^{N}$. Here the function $\mathcal{K}_{\nu}$ is the modified
Bessel function of the second kind with index $\nu>0$ and it is given by

$\mathcal{K}_{\nu}(r)=\frac{(\frac{\pi}{2})^{\frac{1}{2}}r^{\nu}e^{-r}}{\Gamma(\frac{2\nu+1}{2})}\int_{0}^{\infty}\big{(}1+\frac{t}{2}\big{)}^{\nu-\frac{1}{2}}e^{-rt}t^{\nu-\frac{1}{2}}dt,$

for more properties of $\mathcal{K}_{\nu}$, see e.g. [10, 6, 7, 9, 11] and the
references therein.

It is well known that $\mathcal{K}_{\nu}$ is a real and positive function
satisfying

$\mathcal{K}^{\prime}_{\nu}(r)=-\frac{\nu}{r}\mathcal{K}_{\nu}(r)-\mathcal{K}_{\nu-1}(r)<0,$
(1.4)

for all $r>0$, $\mathcal{K}_{\nu}=\mathcal{K}_{-\nu}$ for $\nu>0$.
Furthermore, for $\nu>0$

$\mathcal{K}_{\nu}(r)\sim\begin{cases}\frac{\Gamma(\nu)}{2}(\frac{r}{2})^{\nu},&r\rightarrow
0,\\\\[5.69054pt]
\frac{\sqrt{\pi}}{\sqrt{2}}r^{-\frac{1}{2}}e^{-r},&r\rightarrow\infty.\end{cases}$
(1.5)

It follows from (1.1) that one may expect a Taylor expansion with respect to
parameter $s$ of the operator $(\mathcal{I}-\Delta)^{s}$ near zero for $u\in
C^{2}(\mathbb{R}^{N})$ and $x\in\mathbb{R}^{N}$ as

$\displaystyle(\mathcal{I}-\Delta)^{s}u(x)=u(x)+s(\mathcal{I}-\Delta)^{\log}u(x)+o(s),\quad\text{as}\
s\rightarrow 0^{+}.$ (1.6)

The logarithmic Schr$\ddot{\text{o}}$dinger operator
$(\mathcal{I}-\Delta)^{\log}$ appears as the first order term in the above
expansion.

In this paper, we study the integrodifferential operator
$(\mathcal{I}-\Delta)^{\log}$ corresponding to the logarithmic symbol
$\log(1+|\xi|^{2})$, which is a singular integral operator given by

$(\mathcal{I}-\Delta)^{\log}u(x)=c_{N}P.V.\int_{\mathbb{R}^{N}}\frac{u(x)-u(y)}{|x-y|^{N}}\kappa(|x-y|)dy,$
(1.7)

where $c_{N}=\pi^{-\frac{N}{2}}$, $P.V.$ stands for the Cauchy principal value
of the integral,
$\kappa(r)=2^{1-\frac{N}{2}}r^{\frac{N}{2}}\mathcal{K}_{\frac{N}{2}}(r)$ and
$\mathcal{K}_{\nu}$ is the modified Bessel function of second kind with index
$\nu$. One can also easily deduce from (1.4) that $\kappa^{\prime}(r)<0$ for
$r>0$. Using the expression (1.7), one can define
$(\mathcal{I}-\Delta)^{\log}$ for a quite large class of functions $u$. To
illustrate this, define the space $\mathcal{L}_{0}(\mathbb{R}^{N})$ as the
space of locally integrable functions $u:\mathbb{R}^{N}\rightarrow\mathbb{R}$
such that

$\|u\|_{\mathcal{L}_{0}(\mathbb{R}^{N})}:=\int_{\mathbb{R}^{N}}\frac{|u(x)|e^{-|x|}}{(1+|x|)^{\frac{N+1}{2}}}dx<+\infty.$

Then, it was proved by [10, Proposition 2.1] that for
$u\in\mathcal{L}_{0}(\mathbb{R}^{N})\cap L^{\infty}(\mathbb{R}^{N})$ which is
also Dini continuous at some $x\in\mathbb{R}^{N}$, the quantity
$[(\mathcal{I}-\Delta)^{\log}u](x)$ is well defined by the formula (1.7). Let
us recall the definition of Dini continuity. Let $U$ be a measurable subset of
$\mathbb{R}^{N}$ and let $u:U\rightarrow\mathbb{R}$ be a measurable function.
The modulus of continuity $\Psi_{u,x,U}:(0,+\infty)\rightarrow[0,+\infty)$ of
$u$ at a point $x\in U$ is defined by

$\Psi_{u,x,U}(r):=\sup_{y\in U,|x-y|\leq r}|u(x)-u(y)|.$

We call the function $u$ Dini continuous at $x$ if

$\int_{0}^{1}\frac{\Psi_{u,x,U}(r)}{r}dr<\infty.$

Using the generalized direct method of moving planes, in this note we obtain
the radial symmetry and monotonicity of nonnegative solutions for the
nonlinear equations involving the logarithmic Schr$\ddot{\text{o}}$dinger
operator (see Theorem 1.1), namely, we consider the nonlinear
Schr$\ddot{\text{o}}$dinger equation

$(\mathcal{I}-\Delta)^{\log}u(x)+mu(x)=u^{p}(x),\quad x\in\mathbb{R}^{N},$
(1.8)

with $u(x)\geq 0$ for all $x\in\mathbb{R}^{N}$.

The following result present symmetry and monotonicity properties of
Schr$\ddot{\text{o}}$dinger equation (1.8).

######  Theorem 1.1.

Let $u\in\mathcal{L}_{0}(\mathbb{R}^{N})$ be a nonnegative Dini continuous
solution of (1.8) with $m>0$ and $1<p<\infty$. If

$\lim_{|x|\rightarrow\infty}u(x)=a<\left(\frac{m}{p}\right)^{\frac{1}{p-1}},$
(1.9)

then $u$ must be radially symmetric and monotone decreasing about some point
in $\mathbb{R}^{N}$.

The paper is organized as follows: In Section 2, we prove some results for the
logarithmic Schr$\ddot{\text{o}}$dinger operator. By the direct method of
moving planes, we obtain the symmetry and monotonicity of nonnegative
solutions for the nonlinear equations involving logarithmic
Schr$\ddot{\text{o}}$dinger operator in Section 3.

##  2\. Key ingredients for the method of moving planes

This section is devoted to developing basic and key results needed to apply
the method of moving planes for establishing the proof of our main result in
the next section. We first present some basic notation and nomenclatures which
will be beneficial for rest of the paper.

Choose an arbitrary direction, say, the $x_{1}$-direction. For arbitrary
$\lambda\in\mathbb{R}$, let

$T_{\lambda}=\\{x\in\mathbb{R}^{N}\mid x_{1}=\lambda\\}$

be the moving plane, and let

$\Sigma_{\lambda}=\\{x\in\mathbb{R}^{N}\mid x_{1}<\lambda\\}$

be the region to the left of the plane $T_{\lambda}$, and

$x^{\lambda}=(2\lambda-x_{1},x_{2},\cdot\cdot\cdot,x_{N})$

be the reflection of $x$ about the plane $T_{\lambda}$.

By denoting $u(x^{\lambda}):=u_{\lambda}(x)$, we define

$\omega_{\lambda}(x):=u_{\lambda}(x)-u(x),\quad x\in\Sigma_{\lambda},$

to compare the values of $u(x)$ and $u_{\lambda}(x)$.

The following results on the strong maximum principle for the operator
$(\mathcal{I}-\Delta)^{\log}$ can be deduced from [13, Theorem 1.1] (see also
[10, Theorem 6.1]).

######  Lemma 2.1.

(Strong maximum principle) Let $\Omega\subset\mathbb{R}^{N}$ be a domain, and
let $u\in\mathcal{L}_{0}(\mathbb{R}^{N})$ be a continuous function on
$\bar{\Omega}$ satisfying

$\begin{cases}\ (\mathcal{I}-\Delta)^{\log}u(x)\geq 0,&\text{in}\ \Omega,\\\ \
\qquad u(x)\geq 0,&\text{in}\ \mathbb{R}^{N}\backslash\Omega,\end{cases}$
(2.1)

then $u>0$ in $\Omega$ or $u=0$ a.e. in $\mathbb{R}^{N}$.

Now, we will prove the following maximum principles for the logarithmic
Schr$\ddot{\text{o}}$dinger operator.

######  Theorem 2.2.

(Maximum principle for antisymmetric functions) Let $\Omega$ be a bounded
domain in $\Sigma_{\lambda}$. Assume that
$\omega_{\lambda}\in\mathcal{L}_{0}(\mathbb{R}^{N})\cap
L^{\infty}(\mathbb{R}^{N})$ is Dini continuous on $\Omega$ and is lower semi-
continuous on $\bar{\Omega}$. If

$\begin{cases}\ (\mathcal{I}-\Delta)^{\log}\omega_{\lambda}(x)\geq 0,&in\
\Omega,\\\ \ \qquad\omega_{\lambda}(x)\geq 0,&in\
\Sigma_{\lambda}\backslash\Omega,\\\ \
\quad\omega_{\lambda}(x^{\lambda})=-\omega(x),&in\
\Sigma_{\lambda},\end{cases}$ (2.2)

then

$\omega_{\lambda}\geq 0\quad in\ \Omega.$ (2.3)

Furthermore, if $\omega_{\lambda}(x)=0$ at some point in $\Omega$, then we
have

$\omega_{\lambda}=0,\ a.e.\ \text{in}\ \mathbb{R}^{N}.$ (2.4)

These conclusions hold for unbounded region $\Omega$ if we further assume that

$\liminf_{|x|\rightarrow\infty}\omega_{\lambda}(x)\geq 0.$

###### Proof.

If $\omega_{\lambda}$ is not nonnegative on $\Omega$, then the lower semi-
continuity of $\omega_{\lambda}$ on $\bar{\Omega}$ implies that there exists a
$x^{o}\in\bar{\Omega}$ such that

$\omega_{\lambda}(x^{o}):=\min_{\bar{\Omega}}\omega_{\lambda}(x)<0.$

One can further deduce from (2.2) that $x^{o}$ is in the interior of $\Omega$.
It follows that

$\displaystyle(\mathcal{I}-\Delta)^{\log}\omega_{\lambda}(x^{o})=$
$\displaystyle
c_{N}P.V.\int_{\mathbb{R}^{N}}\frac{\omega_{\lambda}(x^{o})-\omega_{\lambda}(y)}{|x^{o}-y|^{N}}\kappa(|x^{o}-y|)dy$
(2.5) $\displaystyle=$ $\displaystyle
c_{N}P.V.\Bigg{(}\int_{\Sigma_{\lambda}}\frac{\omega_{\lambda}(x^{o})-\omega_{\lambda}(y)}{|x^{o}-y|^{N}}\kappa(|x^{o}-y|)dy+\int_{\Sigma_{\lambda}}\frac{\omega_{\lambda}(x^{o})-\omega_{\lambda}(y^{\lambda})}{|x^{o}-y^{\lambda}|^{N}}\kappa(|x^{o}-y^{\lambda}|)dy\Bigg{)}.$

Since $|x^{o}-y|\leq|x^{o}-y^{\lambda}|$ we have
$\frac{1}{|x^{o}-y|}\geq\frac{1}{|x^{o}-y^{\lambda}|}$ and
$\kappa(|x^{o}-y|)\geq\kappa(|x^{o}-y^{\lambda}|)$ as $\kappa$ is a decreasing
function, and, therefore,

$\frac{\omega_{\lambda}(x^{o})-\omega_{\lambda}(y)}{|x^{o}-y|^{N}}\kappa(|x^{o}-y|)\leq\frac{\omega_{\lambda}(x^{o})-\omega_{\lambda}(y)}{|x^{o}-y^{\lambda}|^{N}}\kappa(|x^{o}-y^{\lambda}|),$

since $\omega_{\lambda}(x^{o})-\omega_{\lambda}(y)\leq 0$.

Thus, we obtain from (2.5) that

$\displaystyle(\mathcal{I}-\Delta)^{\log}\omega_{\lambda}(x^{o})\leq$
$\displaystyle
c_{N}P.V.\int_{\Sigma_{\lambda}}\bigg{(}\frac{\omega_{\lambda}(x^{o})-\omega_{\lambda}(y)}{|x^{o}-y^{\lambda}|^{N}}+\frac{\omega_{\lambda}(x^{o})+\omega_{\lambda}(y)}{|x^{o}-y^{\lambda}|^{N}}\bigg{)}\kappa(|x^{o}-y^{\lambda}|)dy$
(2.6) $\displaystyle=$ $\displaystyle
c_{N}P.V.\int_{\Sigma_{\lambda}}\frac{2\omega_{\lambda}(x^{o})}{|x^{o}-y^{\lambda}|^{N}}\kappa(|x^{o}-y^{\lambda}|)dy<0,$

which contradicts (2.2). Therefore, our assumption is wrong and consequently,
we have $\omega_{\lambda}(x)\geq 0$ in $\Omega$.

Now we have proved that $\omega_{\lambda}(x)\geq 0$ in $\Omega$. If there is
some point $\tilde{x}\in\Omega$ such that $\omega_{\lambda}(\tilde{x})=0$,
then from Lemma 2.1, we derive immediately $\omega_{\lambda}=0$ a.e. in
$\mathbb{R}^{N}$.

For unbounded domain $\Omega$, the condition

$\liminf_{|x|\rightarrow\infty}\omega_{\lambda}(x)\geq 0,$

ensures that the negative minimum of $\omega_{\lambda}$ must be attained at
some point $x^{o}$, then we can derive the same contradiction as above.

This completes the proof of Theorem 2.2. ∎

The following decay at infinity will also be necessary for proving subsequent
results.

######  Theorem 2.3.

(Decay at infinity) Let $\Omega$ be an unbounded domain in $\Sigma_{\lambda}$.
Suppose that a Dini continuous
$\omega_{\lambda}\in\mathcal{L}_{0}(\mathbb{R}^{N})\cap
L^{\infty}(\mathbb{R}^{N})$ is a solution to

$\begin{cases}\
(\mathcal{I}-\Delta)^{\log}\omega_{\lambda}(x)+c(x)\omega_{\lambda}(x)\geq
0,&in\ \Omega,\\\ \ \qquad\omega_{\lambda}(x)\geq 0,&in\
\Sigma_{\lambda}\backslash\Omega,\\\ \
\quad\omega_{\lambda}(x^{\lambda})=-\omega(x),&in\
\Sigma_{\lambda},\end{cases}$ (2.7)

with the measurable function $c(x)$ such that

$\limsup_{|x|\rightarrow\infty}c(x)\geq 0.$ (2.8)

Then there exists a constant $R_{o}>0$, such that if

$\omega_{\lambda}(x^{o})=\min_{\Omega}\omega_{\lambda}(x)<0,$ (2.9)

then

$|x^{o}|\leq R_{o}.$ (2.10)

###### Proof.

We prove the assertion by contradiction. Suppose that (2.10) is false, then by
(2.7) and (2.9), we have

$\omega_{\lambda}(x^{o})=\min_{\Sigma_{\lambda}}\omega_{\lambda}(x)<0.$

After a direct calculation, we obtain

$\displaystyle(\mathcal{I}-\Delta)^{\log}\omega_{\lambda}(x^{o})=$
$\displaystyle
c_{N}P.V.\int_{\mathbb{R}^{N}}\frac{\omega_{\lambda}(x^{o})-\omega_{\lambda}(y)}{|x^{o}-y|^{N}}\kappa(|x^{o}-y|)dy$
(2.11) $\displaystyle=$ $\displaystyle
c_{N}P.V.\int_{\Sigma_{\lambda}}\bigg{(}\frac{\omega_{\lambda}(x^{o})-\omega_{\lambda}(y)}{|x^{o}-y|^{N}}\kappa(|x^{o}-y|)+\frac{\omega_{\lambda}(x^{o})-\omega_{\lambda}(y^{\lambda})}{|x^{o}-y^{\lambda}|^{N}}\kappa(|x^{o}-y^{\lambda}|)\bigg{)}dy$
$\displaystyle\leq$ $\displaystyle
c_{N}P.V.\int_{\Sigma_{\lambda}}\bigg{(}\frac{\omega_{\lambda}(x^{o})-\omega_{\lambda}(y)}{|x^{o}-y^{\lambda}|^{N}}+\frac{\omega_{\lambda}(x^{o})+\omega_{\lambda}(y)}{|x^{o}-y^{\lambda}|^{N}}\bigg{)}\kappa(|x^{o}-y^{\lambda}|)dy$
$\displaystyle=$ $\displaystyle
c_{N}P.V.\int_{\Sigma_{\lambda}}\frac{2\omega_{\lambda}(x^{o})}{|x^{o}-y^{\lambda}|^{N}}\kappa(|x^{o}-y^{\lambda}|)dy<0.$

Now, we fix $\lambda$ and when $|x^{o}|\geq\lambda$, we have
$B_{|x^{o}|}(\breve{x})\subset\tilde{\Sigma}_{\lambda}:=\mathbb{R}^{N}\backslash\Sigma_{\lambda}$
with $\breve{x}=(3|x^{o}|+x_{1}^{o},(x^{o})^{\prime})$. So, for
$y\in\Sigma_{\lambda}$, we can deduce that
$|x^{o}-y|\leq|x^{o}-\breve{x}|+|\breve{x}-y|\leq|x^{o}|+3|x^{o}|=|4x^{o}|$
which together with the fact that $\kappa$ is a decreasing function implies
that

$\frac{\kappa(|x^{o}-y|)}{|x^{o}-y|}\geq\frac{\kappa(|4x^{o}|)}{|4x^{o}|}.$

Thus, if $|x^{o}|\geq\frac{R_{\infty}}{4}$ with sufficiently large
$R_{\infty}$ then from (1.5) and
$\kappa(r)=2^{1-\frac{N}{2}}r^{\frac{N}{2}}\mathcal{K}_{\frac{N}{2}}(r)$, we
have

$\displaystyle\int_{\Sigma_{\lambda}}\frac{1}{|x^{o}-y^{\lambda}|^{N}}\kappa(|x^{o}-y^{\lambda}|)dy$
$\displaystyle=\int_{\tilde{\Sigma}_{\lambda}}\frac{\kappa(|x^{o}-y|)}{|x^{o}-y|^{N}}dy\geq\int_{B_{|x^{o}|}(\breve{x})}\frac{\kappa(|4x^{o}|)}{|4x^{o}|^{N}}dy$
(2.12)
$\displaystyle\geq\int_{B_{|x^{o}|}(\breve{x})}\frac{2^{1-\frac{N}{2}}\mathcal{K}_{\frac{N}{2}}(|4x^{o}|)}{|4x^{o}|^{\frac{N}{2}}}dy$
$\displaystyle\geq\frac{c_{\infty}\omega_{N}}{2^{\frac{3N}{2}}|x^{o}|^{\frac{1+N}{2}}e^{4|x^{o}|}}:=\frac{C}{|x^{o}|^{\frac{1+N}{2}}e^{4|x^{o}|}},$

where $C=c_{\infty}\omega_{N}2^{-\frac{3N}{2}}$ is a positive constant.

It follows that

$0\leq(\mathcal{I}-\Delta)^{\log}\omega_{\lambda}(x^{o})+c(x^{o})\omega_{\lambda}(x^{o})\leq\bigg{(}\frac{C}{|x^{o}|^{\frac{1+N}{2}}e^{4|x^{o}|}}+c(x^{o})\bigg{)}\omega_{\lambda}(x^{o}),$

or equivalently,

$\frac{C}{|x^{o}|^{\frac{1+N}{2}}e^{4|x^{o}|}}+c(x^{o})\leq 0.$

Now, if $|x^{o}|$ is sufficiently large, this would contradict (2.8).
Therefore, (2.10) holds.

This completes the proof of Theorem 2.3. ∎

##  3\. Proof of the main theorem

###### Proof of Theorem 1.1.

Let $T_{\lambda},\Sigma_{\lambda},x^{\lambda}$ and $\omega_{\lambda}$ be
defined as in the previous section. Then at the points where
$\omega_{\lambda}(x)<0$, it is easy to verify that, for
$\xi_{\lambda}(x)\in(u_{\lambda}(x),u(x)),$ we have

$\displaystyle(\mathcal{I}-\Delta)^{\log}\omega_{\lambda}(x)+m\omega_{\lambda}(x)=u_{\lambda}^{p}(x)-u^{p}(x)=p\xi_{\lambda}^{p-1}(x)\omega_{\lambda}(x)\geq
pu^{p-1}(x)\omega_{\lambda}(x),$ (3.1)

because $\omega_{\lambda}(x)<0$, and $\xi_{\lambda}(x)<u(x)$.

$\mathbf{Step\ 1}$. We will show that for sufficiently negative $\lambda$,

$\omega_{\lambda}(x)\geq 0,\quad x\in\Sigma_{\lambda}.$ (3.2)

First, from the assumption (1.9), for each fixed $\lambda$,
$\lim_{|x|\rightarrow\infty}\omega_{\lambda}(x)=0.$ In fact, by (1.9), we have
$\lim_{|x|\rightarrow\infty}u(x)=a$, and
$\lim_{|x|\rightarrow\infty}u_{\lambda}(x)=a$ implying that
$\lim_{|x|\rightarrow\infty}\omega_{\lambda}(x)=0.$

Thus, if (3.2) is false, then the negative minimum of $\omega_{\lambda}$ can
be obtained at some point, say, $x^{o}$ in $\Sigma_{\lambda}$, that is,

$\omega_{\lambda}(x^{o})=\min_{\Sigma_{\lambda}}\omega_{\lambda}(x)<0.$

Set $c(x):=m-pu^{p-1}(x)$ in (3.1) and then, the assumption (1.9) implies that
$c\in L^{\infty}(\mathbb{R}^{N})$ and

$\lim_{|x|\rightarrow\infty}c(x)\geq 0.$

Consequently, from Theorem 2.3 it follows that there exists $R_{o}>0$
(independent of $\lambda$), such that

$|x^{o}|\leq R_{o}.$ (3.3)

Therefore, by choosing $\lambda<-R_{o}$ and consequently,
$|x^{\lambda}|>R_{0}$ for $x\in\Sigma_{\lambda},$ we obtain by (3.3) that

$\omega_{\lambda}(x)\geq 0,\quad x\in\Sigma_{\lambda}.$ (3.4)

$\mathbf{Step\ 2}$. Step 1 provides a starting point, from which we can now
move the plane $T_{\lambda}$ to the right as long as (3.2) holds to its
limiting position. Define

$\lambda_{o}:=\sup\\{\lambda\mid\omega_{\mu}(x)\geq 0,\ \forall\
x\in\Sigma_{\mu},\ \forall\mu\leq\lambda\\}.$

By (3.3), we know $\lambda_{o}<\infty$.

Next, we will show via a contradiction argument that

$\omega_{\lambda_{o}}(x)\equiv 0,\quad\forall\ x\in\Sigma_{\lambda_{o}}.$
(3.5)

Suppose, on the contrary, that

$\omega_{\lambda_{o}}(x)\geq 0,\ \text{and}\ \
\omega_{\lambda_{o}}(x)\not\equiv 0,\ \text{in}\ \Sigma_{\lambda_{o}},$ (3.6)

then we must have

$\omega_{\lambda_{o}}(x)>0,\quad\forall\ x\in\Sigma_{\lambda_{o}}.$ (3.7)

In fact, if (3.7) is violated, then there exists a point
$\hat{x}\in\Sigma_{\lambda_{o}}$ such that

$\omega_{\lambda_{o}}(\hat{x})=\min_{\Sigma_{\lambda_{o}}}\omega_{\lambda_{o}}(x)=0.$

It means that $u_{\lambda_{o}}(\hat{x})=u(\hat{x})$. Then it follows from
(3.1) that

$(\mathcal{I}-\Delta)^{\log}\omega_{\lambda_{o}}(\hat{x})=u_{\lambda_{o}}^{p}(\hat{x})-u^{p}(\hat{x})=u^{p}(\hat{x})-u^{p}(\hat{x})=0.$

Hence, Theorem 2.2 implies that $\omega_{\lambda_{o}}(\hat{x})\equiv 0$ in
$\Sigma_{\lambda_{o}}$, which contradicts (3.6). Thus (3.7) holds.

Now, we will show that the plane $T_{\lambda}$ can be moved further right.
More precisely, there exists an $\epsilon>0$ such that, for any
$\lambda\in[\lambda_{o},\lambda_{o}+\epsilon)$, we have

$\omega_{\lambda}(x)\geq 0,\quad x\in\Sigma_{\lambda}.$ (3.8)

Once it is proved this will contradict the definition of $\lambda_{o}$.
Therefore, (3.5) must be valid.

Let us now prove (3.8). In fact, by (3.7), we have
$\omega_{\lambda_{o}}(x)>0,\,\,\,x\in\Sigma_{\lambda_{o}},$ which in turn
implies that there is a constant $c_{o}>0$ and $\delta>0$ such that

$\omega_{\lambda_{o}}(x)\geq c_{o}>0,\quad
x\in\overline{\Sigma_{\lambda_{o}-\delta}\cap B_{R_{o}}(0)}.$

Since $\omega_{\lambda}$ is continuous with respect to $\lambda$ there exists
an $\epsilon>0$ such that for $\lambda\in[\lambda_{o},\lambda_{o}+\epsilon)$
we have

$\omega_{\lambda}(x)\geq 0,\quad x\in\Sigma_{\lambda_{o}-\delta}\cap
B_{R_{o}}(0).$ (3.9)

Moreover, combining (3.3) with (3.9), we deduce that $w_{\lambda}(x)\geq 0$ on
$\Sigma_{\lambda_{0}-\delta}.$

To proceed with the proof, we need the following small volume maximum
principle ([10, Theorem 6.1 (iii)] and [13, Theorem 1.3]).

######  Lemma 3.1.

Let $\Omega$ be a open set of $\mathbb{R}^{N}.$ Consider the following problem
on $\Omega$:

$\begin{cases}(\mathcal{I}-\Delta)^{\log}u(x)\geq c(x)u,&in\ \Omega,\\\
\qquad\qquad u\geq 0,&in\ \mathbb{R}^{N}\backslash\Omega,\end{cases}$ (3.10)

with $c\in L^{\infty}(\mathbb{R}^{N})$.

Then, there exists $\delta>0$ such that for every open set
$\Omega\subset\mathbb{R}^{N}$ with $|\Omega|\leq\delta$ and any solution
$u\in\mathcal{V}_{\omega}(\Omega)$ of (3.10) in $\Omega$, where the space
$\mathcal{V}_{\omega}(\Omega)$ is given in [10, Section 6], we have $u\geq 0$
in $\mathbb{R}^{N}$.

Consequently, according to Lemma 3.1 (by taking
($\Omega=(\Sigma_{\lambda}\setminus\Sigma_{\lambda_{o}-\delta})\cap
B_{R_{o}}(0)$)), we obtain that (3.8) holds.

The arbitrariness of the $x_{1}$-direction leads to the radial symmetry of
$u(x)$ about some point in $\mathbb{R}^{N}$, and the monotonicity is a
consequence of the fact that (3.4) holds.

This completes the proof of the Theorem 1.1. ∎

## Acknowledgments

RZ is supported by the China Scholarship Council (No.202106860018), the
National Natural Science Foundation of China (Grant No. 11871278) and by the
National Natural Science Foundation of China (Grant No. 11571093). VK and MR
are supported by the FWO Odysseus 1 grant G.0H94.18N: Analysis and Partial
Differential Equations, the Methusalem programme of the Ghent University
Special Research Fund (BOF) (Grant number 01M01021) and by FWO Senior Research
Grant G011522N. MR is also supported by EPSRC grant EP/R003025/2.

## References

  * [1] H. Chen, T. Weth, The Dirichlet problem for the logarithmic Laplacian, Communications in Partial Differential Equations, 44, 11 (2019): 1100-1139.
  * [2] W. Chen, C. Li, Classification of solutions of some nonlinear elliptic equations, Duke Math. J., 63 (1991): 615-622.
  * [3] W. Chen, C. Li, Y. Li, A direct method of moving planes for the fractional Laplacian. Adv. Math., 308 (2017): 404-437.
  * [4] E. Correa, A. de Pablo, Nonlocal operators of order near zero, Journal of Mathematical Analysis and Applications, 461 (2018): 837-867.
  * [5] T. Cazenave, Semilinear Schrödinger Equations, American Mathematical Soc., 2003.
  * [6] A. Elgart, B. Schlein, Mean field dynamics of boson stars. Commun. Pure Appl. Math., 60 (2007): 500-545.
  * [7] J. Fröhlich, B. Jonsson, E. Lenzmann, Boson stars as solitary waves, Commun. Math. Phys., 274 (2007): 1-30.
  * [8] M. Fall, V. Felli, Sharp essential self-adjointness of relativistic Schrödinger operators with a singular potential. Journal of Functional Analysis, 267 (2014): 1851-1877.
  * [9] M. Fall, V. Felli, Unique continuation properties for relativistic Schrödinger operators with a singular potential, Discrete Contin. Dyn. Syst. A, 35 (2015): 5827-5867.
  * [10] P. Feulefack, The Logarithmic Schrödinger operator and Associated Dirichlet Problems, arXiv preprint arXiv:2112.08783 (2021).
  * [11] Y. Guo, S. Peng, Symmetry and monotonicity of nonnegative solutions to pseudo-relativistic Choquard equations, Z. Angew. Math. Phys., 72, 120 (2021).
  * [12] V. Hernández-Santamará, A. Saldaña, Small order asymptotics for nonlinear fractional problems, arXiv preprint arXiv:2108.00448 (2021).
  * [13] S. Jarohs, T. Weth, On the maximum principle for nonlocal operators, Mathematische Zeitschrift, 293 (1) (2019): 81-111.
  * [14] M. Kassmann, A. Mimica, Intrinsic scaling properties for nonlocal operators, Journal of the European Mathematical Society 19, 4 (2017): 983-1011.
  * [15] G. Lieberman, Second Order Parabolic Differential Equations, World Scientific, 1996.
  * [16] W. Liu, E. Kengne, Schrödinger Equations in Nonlinear Systems, Springer Nature Singapore Pte Ltd., 2019.
  * [17] A. Laptev, T. Weth, Spectral properties of the logarithmic Laplacian, Anal. Math. Phys., 133, 24, (2021).
  * [18] A. Mimica, On harmonic functions of symmetric Lévy processes, In Annales de l’IHP Probabilités et statistiques, 50 (2014): pp. 214–235.
  * [19] C. Sulem, P. Sulem, The Nonlinear Schrödinger Equation: Self-Focusing and Wave Collapse, Springer Science and Business Media, 2007.
  * [20] G. Wang, X. Ren, Z. Bai, W. Hou, Radial symmetry of standing waves for nonlinear fractional Laplacian Hardy-Schrödinger systems, Applied Mathematics Letters, 96 (2019): 131-137.
  * [21] L. Zhang, X. Nie, A direct method of moving planes for the Logarithmic Laplacian, Applied Mathematics Letters, 118 (2021): 107141.