# VReBERT: A Simple and Flexible Transformer for Visual Relationship Detection

Yu Cui1∗, Moshiur Farazi2,1<EMAIL_ADDRESS>1Research School of Computer
Science, The Australian National University (ANU)
2Data61–Commonwealth Scientific and Industrial Research Organisation (CSIRO)
Canberra, Australia

###### Abstract

Visual Relationship Detection (VRD) impels a computer vision model to _see_
beyond an individual object instance and _understand_ how different objects in
a scene are related. The traditional way of VRD is first to detect objects in
an image and then separately predict the relationship between the detected
object instances. Such a disjoint approach is prone to predict redundant
relationship tags (i.e., predicate) between the same object pair with similar
semantic meaning, or incorrect ones that have a similar meaning to the ground
truth but are semantically incorrect. To remedy this, we propose to jointly
train a VRD model with visual object features and semantic relationship
features. To this end, we propose VReBERT, a BERT-like transformer model for
Visual Relationship Detection with a multi-stage training strategy to jointly
process visual and semantic features. We show that our simple BERT-like model
is able to outperform the state-of-the-art VRD models in predicate prediction.
Furthermore, we show that by using the pre-trained VReBERT model, our model
pushes the state-of-the-art zero-shot predicate prediction by a significant
margin (+8.49 R@50 and +8.99 R@100).

##  I Introduction

Understanding how different objects are related in an image is a higher level
of visual perceptions, and computer vision models tasked with this need to
perform far more complex vision tasks compared to simply detecting objects or
their attributes. Such a higher level of image understanding has broad
application. For example, a system that understands visual relationships can
aid visually impaired people by describing a scene and answering intelligent
questions about it [1, 2, 3]. Lu et al.[4] formalized the Visual Relationship
Detection (VRD) task to aid the research for addressing this challenging
problem. They defined the visual relationship as a pair of localized object
instances connected by a predicate, i.e., object1 – relationship tag –
object2. For simplicity, in the context of this paper we refer the object1 and
object2 as subject and object respectively, and the relationship tag as a
predicate. Further, Lu et al. proposed the VRD dataset which contained
involving $100$ object categories and $70$ predicates. The predicates include
verb (e.g., drive, follow), spatial relation (e.g., over, on), comparative
relations (e.g., taller than), and many other categories. This dataset
provided an effective evaluation protocol to benchmark VRD models.

Figure 1: Traditional project-based methods use only one embedding space for
predicate prediction, which will limit the models’ capability of learning the
context of the images. Differently, VReBERT utilizes multi-head attention to
jointly learn multiple semantic meanings in different contexts.

VRD models [4, 5, 6, 7, 8, 9] approach relationship detection by decomposing
the task into two tasks. They first identify and localize two object instances
as subject and object pair, and then try to predict the predicate between
these object instances. There are a couple of fundamental challenges that an
AI model will need to overcome to succeed in visual relationship detection
following this approach. First, the multi-modal aspect of the task. The
objects in the images provide visual cues for a Visual Relationship Detection
(VRD) model to generate both semantic (i.e., relationship tags) and visual
(i.e., a localized pair of objects) output. Second, the relationship tags
(i.e., predicates) generated by a VRD model are often ambiguous as the model
tries to learn visual relationships from looking and relationships between the
same object pairs. It’s easy for the model to forget the context and predict a
synonymous relationship tag that is contextually incorrect (e.g., holding vs
carrying) or fail to predict multiple relationships that may exist between the
same pair of object instances (e.g., <person-on-bicycle> and <person-riding-
bicycle>. Furthermore, predicates may have very different semantic meanings in
different contexts, <picture-on-wall> and <bottle-on-table> share the same
predicate on while the semantic meanings are different. To address the
challenges of the multi-modal aspect of VRD and to reduce the ambiguity in
predicate prediction, one would need to bridge the semantic gap between the
two tasks that VRD is decomposed into. We propose to jointly process the
semantic relationship tags and visual object instance features to bridge this
gap.

Recently, some transformer-based cross-modal methods achieved great success in
vision and language tasks [10, 11, 12]. These methods take advantage of large-
scale mono-modal (e.g., vision or language) or multi-modal (e.g., vision and
language) pre-training. The large-scale pre-training not only reduces the
training cost but also improves the performance on a very small task-specified
dataset as the training does not start from scratch. More importantly, the
transformer performs well in many sequence-to-sequence problems. Inspired by
the success of transformers in cross-modal vision and language tasks, in this
paper we endeavour to find a sequence-to-sequence function that projects the
image patch sequence to a relationship sequence. In this paper, we propose
VReBERT, a simple and flexible transformer-based model designed for visual
relationship detection. The flexible nature of VReBERT allows any typical
object detector module (e.g., Faster R-CNN [13], UperNet [14] \+ Swin-
transformer [15]) to be used with it jointly.

Our proposed VReBERT is similar to BERT [16] at the conceptual level but
redesigned to solve cross-modal visual relation detection. Just like BERT, and
other transformer-based vision and language models VReBERT also benefits from
pre-training on large-scale datasets. We show that our model achieved a new
state-of-the-art performance in the visual relationship task. Furthermore,
with VReBERT, we were able to improve zero-shot visual relationship detection
performance significantly ($\sim$ +9 on R@50 and R@100) leveraging large-scale
pre-training. This further strengthens the rigidity of our approach of jointly
training the relationship detector with visual and semantic features using
transformer architecture. Such strong generalization ability would help our
proposed VRD model to be useful in other downstream tasks.

The main contributions of our paper are as follows:

  * •

We propose a novel visual relationship detection framework VReBERT that can
jointly learn the representation of visual and semantic contents by utilizing
the extracted object location, object label, image feature, and language
priors to detect relationships between an object pair in an image.

  * •

We introduce a multi-stage training method for VReBERT to reduce the training
cost and improve the generalization ability.

  * •

We reported state-of-the-art performance with VReBERT in Visual Relationship
Detection [4] task and a significant improvement in zero-shot visual
relationship detection.

Figure 2: The architecture of our proposed VReBERT model. Visual features from
the image are extracted by a Faster R-CNN object detector. The visual features
and their labels are combined as the input and added with segment and relative
positional embedding. They are then passed to the VReBERT block that jointly
learns the context of the image and language. Such joint training allows our
proposed model to understand the alignment and semantic meanings and make
predicate prediction in a way similar to the task of masked language modeling.

##  II Methodology

We designed the VReBERT as a combination of modular blocks to provide greater
flexibility and further development. First is the object proposal module. For
this work, we use Faster-RCNN[13] as the object detector, but as mentioned
before this module can be replaced with other typical object detectors. This
module takes an RGB image as input and proposes regions of interest (ROI)
where it thinks there might be an object, provides a prediction of an object
class for each ROI, and extracts visual features by its backbone network
(e.g., ResNet[17]). VReBERT utilizes bounding boxes, class labels, and the
extracted features for visual and semantic reasoning. VReBERT models the
relationship prediction as a masked language model prediction task similar to
a vanilla BERT [16]. The key differences are multi-modal inputs and redesigned
embeddings for image tokens. VReBERT can jointly learn the representation of
image and language to capture the interaction between the proposed objects,
which allows VReBERT to solve cross-modal problems. We propose a multi-stage
training strategy for the architecture mentioned above. Pre-training on the
large-scale image and text dataset can help our model get a better result on
the task-specified dataset. The architecture of our model is illustrated in
Fig. 2.

###  II-A Object Detector

As the foundation of visual relationship detection, the object detection
module sits at the beginning of the pipeline. Considering that this paper
focuses on visual and semantic reasoning of the relationships, we would like
the object detection module to be as simple and easy-use as possible. We
employ a Faster R-CNN [13] as the object detection module. The two main
reasons for our choice are as follow. First, Faster R-CNN is fast and easy to
use with lots of open-access pre-trained models which is beneficial for large-
scale pre-training. Compared to other object detection frameworks, Faster
R-CNN is also very competitive in terms of effectiveness. Second, Faster R-CNN
[13] uses a modular design. It contains an RPN [13] and a Fast R-CNN [18]. We
can take advantage of this design and make the training and evaluation faster.
The details will be covered in Sec. II-E. The object detector module proposes
ROIs and corresponding object category labels. ROIs include the bounding box
and the representation extracted from the image. The labels come with
classification probabilities. We extracted the object bounding box, region
representation, category label, and category confidence for relationship
reasoning.

###  II-B VReBERT

Many of the previous works [4, 5, 19, 20] tried to project the subject-
predicate-object sequence to the semantic embedding space to understand the
visual relationships, which can be generically expressed as:

$S_{text}=\textrm{F}_{seq2seq}(S_{image})$ (1)

Furthermore, Yun et al. [21] mathematically showed that transformers are
universal approximators of sequence-to-sequence functions. Inspired by this,
we want to find a sequence-to-sequence function that projects the image
feature sequence to a relationship sequence. As mentioned earlier, previous
works struggled with two challenges: one predicate with multiple semantic
meanings and multiple correct predicates between one subject-object pair. To
conquer the first challenge our method has to be able to model the polysemy
(i.e., a word or phrase having multiple related meanings) and understand the
different meanings of a predicate. For the second challenge, we need to
utilize both visual and linguistic information. Our model needs to have an
excellent multi-modal reasoning capability. The multi-head attention mechanism
proposed with the transformer is designed to jointly understand different
representation sub-spaces, which can efficiently deal with the polysemy.
Different attention heads can learn the different semantic meanings of a
predicate. When looking at a predicate, the transformer can leverage all
attention heads and consider all possible meanings. Inspired by [16, 22, 23],
we propose VReBERT, a BERT-like transformer model for visual and semantic
reasoning.

###  II-C Multi-modal Embedding

We follow similar token nomenclature of vanilla BERT and use special tokens
like [CLS] and [SEP] as the task indicator and sentence divider. The word
embedding is kept basically the same, while we redesign the image embedding
for visual relationship reasoning.

####  II-C1 Word embedding

The sentence input contains the word-based representation of object and
subject. Then we use WordPiece [24] method to tokenize the word-based object
labels from the object detection module into sub-word tokens. We add a special
token [MASK] between the token of subject and object for masked language
modeling. After we get the token representation of the relationship sequence,
we add the segmentation embedding and position embedding to the tokens like
the vanilla BERT.

One thing worth noting is that we utilize a relative positional embedding
method instead of the absolute position embedding used in BERT. The attention
mechanism cannot comprehend the sequence order from the word tokens. So [25]
and [16] use a fixed sinusoid function to encode the sequence order of the
tokens. The absolute positional embedding does not perform well in some tasks
(e.g., question answering [26]). Furthermore, Yun et al. [21] showed that
trainable positional encoding is important for transformer-based models to be
a universal approximator of a sequence-to-sequence function. Recently,
transformer-based methods [27, 28, 29] adopt relative position encoding as
trainable tokens. In our paper, we employ a relative positional encoding
method similar to [26] as follow:

$\mathbf{a}_{ij}=\mathbf{w}_{j-i}$ (2)

where $\mathbf{a}_{ij}$ represents the relative position embedding of position
$j$ with respect to position $i$, $\mathbf{w}_{j-i}$ is a trainable vector.
The position embedding will interact with both the queries and the keys during
self-attention. The modified scale attention $Attn_{ij}$ can be represented
as:

$Attn_{ij}=\frac{(\mathbf{x}_{i}\mathbf{W}^{Q})\cdot(\mathbf{x}_{j}\mathbf{W}^{K})+\mathbf{W}^{Q}\cdot\mathbf{a}_{ij}+\mathbf{W}^{K}\cdot\mathbf{a}_{ij}}{\sqrt{d_{z}}}$
(3)

We expect this trainable relative positional embedding to learn unique
position tokens to capture the impact of the position on attention. More
specifically, besides the sequence order, we want it to learn the alignment of
image and text.

####  II-C2 Image Embedding

Image embedding consists of visual features and image positional embedding.

Visual Features: The image is represented as a combination of visual features
and positional embeddings. The visual features of the bounding box containing
the $subject$, $object$, and $predicate$ (i.e., the bounding box containing
both subject and object) are represented as $\mathbf{i}_{sub}$,
$\mathbf{i}_{pred}$, $\textbf{i}_{obj}$ respectively which are all
$\in\mathbb{R}^{D}$.

Image positional embedding: Further to the visual features, the spatial
location of the relationship triplet is parameterized by normalized bounding
box locations. Each bounding box is represented by
$[X_{min},Y_{min},X_{max},Y_{max}]$ and is normalized by the corresponding
height $H$, or weight $W$ dimension of the image. We further add another
spatial area normalization term for the bounding boxes to normalize the
bounding box area with respect to the total image area. The resultant
5-dimension image position embedding vector is projected to $D$-dimension
using a fully connected layer $\mathbf{P}$. The image position embedding can
be represented as

$\small{\begin{split}\mathbf{p}_{i}=\textrm{P}(\frac{X_{min}}{W},\frac{Y_{min}}{H},\frac{X_{max}}{W},\frac{Y_{max}}{H},\\\
\frac{(X_{max}-X_{min})*(Y_{max}-Y_{min})}{W*H})\end{split}}$ (4)

####  II-C3 Special embedding

Additional two kinds of special embedding are used to train our model in a
BERT-like manner. We use the same segment embedding to delineate between image
embedding and relationship sequence by setting the former to $0$ and the
latter to $1$. We also use the relative position embedding mentioned earlier
for image sequence position embedding to force the relationship patches to
have the triplet order <sub-pred-obj>. This relative positional embedding is
learned by the model by understanding the alignment between the image tokens
and the word tokens.

###  II-D Relationship likelihood

For relationship detection, we need to consider both object likelihood and
predicate likelihood. We use the following likelihood function for
relationship:

$\begin{split}P_{i,j,k}(pred,obj,sub)=P_{i}(pred\ |\ obj,\
sub)P_{j}(obj)P_{k}(sub)\hfill\end{split}$ (5)

where $P_{j}(obj)$ and $P_{k}(sub)$ are the likelihood from the object
detection module for object and subject being in a specific category and
$P_{i}(pred\ |\ obj,sub)$ is the likelihood from the VReBERT, which indicates
the predicate belongs to category $i$.

###  II-E Multi-stage Training Strategy

Having a flexible and lightweight training strategy, the generalization
capability of transformer-based models increase significantly. With the large-
scale pre-training and task-specified fine-tuning, BERT-like models [16, 22,
23] achieved great success. Since our model jointly learns the representation
of image and language, our pre-training stage is more complex than only
training on language or image. We implement the following multi-stage training
strategy to reduce the training and evaluation cost and to be able to run some
task-specified benchmarks to gauge the performance of the model undergoing the
full training. This strategy is designed for the VReBERT model with a typical
object detector (Faster R-CNN in our case).

  * S1

Pre-training on large-scale datasets: The object detector used pre-trained
weights learned from PASCAL VOC 2012, and VReBERT is initialized from a pre-
trained BERTbase [16] model.

  * S2

Separate training of object detector and the transformer core: The object
detector is re-trained on the target Visual Relationship Dataset [4]. The BERT
part of our model is separately trained on masked relationship prediction
where the input to the models is <sub-[MASK]-obj> and the model tries to
predict the relationship only from language cues without looking at the image.
Such separate mono-modal training allows the models to ‘learn’ task-specific
capabilities which they can build during the multi-modal training.

  * S3

Training VReBERT on the multi-modal VRD dataset: After the training on the
mono-modal task, VReBERT can handle linguistic reasoning and object detection.
This stage aims to combine these two abilities for visual relationship
detection. Now, the VReBERT model trained image representations (i.e., visual
features, image positional embedding as discussed in II-C2) extracted using
the object detector, word embeddings similar to that of the masked
relationship prediction stage, segment and relative positional embedding. One
thing worth noting is that the predicate benchmark in [4] uses the same input
and output setup. Thus, we can evaluate VReBERT’s ability to predict the
predicate after this stage of training, and we report our model performance on
predicate prediction from this stage for fair comparison.

  * S4

Fine-tuning: We fine-tune the region proposal network (RPN) for the object
detector, to make the pipeline end-to-end. This stage helps the VReBERT to get
better object proposals for predicate prediction with a combined loss function
during fine-tuning our model on the Visual Relationship Dataset.

##  III Experiments

###  III-A Implementation details

Our implementation is on the PyTorch platform. We employ a Faster R-CNN with
ResNet101 [17] pre-trained on ImageNet as the backbone. The image feature
output is set to $2048$-dimensional. The image feature is projected to the
same dimension as the VReBERT linguistic embeddings. The VReBERT is configured
with $768$ hidden dimensions, 12 self-attention heads, and 12 hidden layers.
The dropout probability is set to 0.1 to improve the generalization
capability. We employ the AdamW optimizer with an initial learning rate of
$1\times 10^{-5}$ and set $\beta_{1}=0.9$, $\beta_{2}=0.999$.

###  III-B Dataset

In this paper, we train and evaluate our model on the VRD dataset [4]. This
dataset contains $5000$ images with $100$ object categories and 70 predicate
categories. It also has $37,993$ relationships in total, $6672$ relationship
types are collected resulting in $7.6$ relationships per image. Each object
category is related to $24.25$ predicates on average. We follow the dataset
split in [4]. We use $4000$ images to train our model and $1000$ images to
test. The test set contains $1877$ relationships that never appear in the
training set. The unseen relationships can help us evaluate the performance
gain from large-scale pre-trained and the model’s generalization ability. Even
though VRD is a relatively small dataset, we choose this to benchmark our
model can be trained with such dataset size leveraging large-scale pre-
training.

###  III-C Evaluations metrics

Following [4], we report Recall@50 (R@50) and Recall@100 (R@100) as our
evaluation metric for image relationship detection. Recall@N indicates how
many true relationships are detected by the model in the top-N results. Recall
is defined as a weighted representation of True Positive (TP), considering
False Negatives (FN), and is expressed as:

$Recall=\frac{TP}{TP+FN}$ (6)

###  III-D Predicate Prediction

Predicate prediction is the task of predicting the relationship tag (i.e.,
predicate) based on the ground truth object position and label. For this task,
we want to test the predicate reasoning ability of our method. So, we use the
ground truth visual information to avoid the performance impact from the
object detection module. This task can directly compare VReBERT with the
previous visual relationship detection methods on the predicate prediction
ability. With the multi-stage training strategy, we can analyze the predicate
prediction performance after the third stage of training as previously
discussed in Sec. II-E. By doing so, even without fully training the entire
model, we can have a general idea about the performance of our model.
Considering that predicate prediction is the key to visual relationship
detection, the experiment will mainly focus on this task.

####  III-D1 Comparable methods

We compare our method with the following comparable methods in the literature
utilizing a diverse and comprehensive set of approaches. First, with the VRD
[4] where the projection-based word2vector method was proposed together with
the Visual Relationship Detection problem. The LKD [9] used knowledge
distillation [30] utilizing both student and teacher networks to distill
internal and external linguistic knowledge for visual relationship detection.
Further, DR-Net[31] is an RNN-based joint recognition method utilizing both
image feature and linguistic information. Furthermore, DSR[7] proposed a
visual relationship detection method based on deep structure ranking. Finally
with BLOCK[6] fusion model used linear superdiagonal fusion to combine visual
and semantic features for visual relationship detection.

####  III-D2 Experimental setup

We perform ablation with different combinations of our VReBERT models report
their performance on Tab. I. The different combinations are

  * ES1

BERT from scratch: In this setting, we only employ a vanilla BERT. We train
this model from scratch using the relationship labels from the VRD dataset to
show the performance of the pure language model.

  * ES2

Pre-trained BERT: This model initialized from a BERT model pre-trained on
large-scale datasets. We fine-tuned this model on the VRD dataset.

  * ES3

VReBERT with pre-trained visual features: We use image features from a pre-
trained Fast R-CNN (After S2 pre-training in Sec. II-E) features for predicate
reasoning. And we use absolute sequence position encoding.

  * ES4

VReBERT with fine-tuned visual features: We fine-tune the Fast R-CNN with
VReBERT and use the fine-tuned image feature for prediction. And we use
absolute sequence position encoding.

  * ES5

VReBERT with image positional embedding: We add image positional embedding
(IP) to compare the performance gain from image positional embedding.

  * ES6

VReBERT full model: We replace the absolute sequence positional encoding with
relative positional encoding.

TABLE I: Predicate Prediction Performances of Different Methods.

Method |  R@50 (%) |  R@100 (%)
---|---|---
VRD-L only [4] |  48.64 |  61.95
VRD-Full [4] |  70.97 |  84.34
LKD [9] |  85.64 |  94.65
DR-Net [31] |  80.78 |  81.90
DSR [7] |  86.01 |  93.18
BLOCK [6] |  86.58 |  92.58
BERT from scratch |  54.50 |  71.29
Pre-trained BERT |  62.35 |  79.34
VReBERT with PF + AP |  61.40 |  85.11
VReBERT with FF + AP |  83.24 |  92.77
VReBERT with FF + AP + IP |  88.48 |  94.90
VReBERT with FF + RP + IP |  $\mathbf{88.57}$ |  $\mathbf{95.26}$
Improvement over SOTA |  2.93 $\uparrow$ |  0.61 $\uparrow$

  * •

The first set of models are from literature, the middle set is our blind
VReBERT (no visual input), and the last set of models is multimodal VReBERT.

  * •

‘PF’ is the visual feature generated by the pre-trained Faster R-CNN.

  * •

‘FF’ is the visual feature generated by the fine-tuned Faster R-CNN.

  * •

‘AP’ is absolute positional embedding using sinusoid”.

  * •

‘RP’ is the trainable relative positional embedding.

  * •

‘IP’ is image positional embedding.

###  III-E Result

How effective are pure language models in VRD?
We start with a vanilla BERT to show how a pure language model performs in
this visual-language task. When comparing the BERT model with the VRD-L model
[4], the transformer-based BERT model is better than the projection-based
model [4] for language modeling. As we mentioned in the dataset section, the
test set contains 1887 unseen relationships. Simply training on the VRD
training set may cause the model to struggle with unseen relationship
prediction. In experimental setting 2 (i.e., ES2 in Sec. II-E), we initialized
VReBERT with a pre-trained BERT-base model. The result shows pre-trained BERT
model achieved a better result. However, even with a large-scale pre-training,
the language model can only get $62\%$ in R@50. And during the experiment, we
notice that the BERT model always predicts a predicate with the higher
frequency, as shown in the first two rows of (Fig. 3), where the subject-
object pair is set to person-bench. The BERT prediction is basically the same
as the frequency of occurrence. It does make sense considering the statistical
property of the language models. This fact indicates language models have
difficulty in predicting the predicate without the visual information.

Figure 3: Predicate Frequency, BERT, and VReBERT Prediction Confidence. The
Language-only BERT model distributes its attention only based on the predicate
frequency, while the multi-modal VReBERT leverages the image feature to
propose visually reasonable predicates.

Can VReBERT leverage image features for relationship reasoning?
As we mentioned early, pure language does not perform well in visual
relationship detection. Our model with fine-tuned visual appearance (ES4)
achieved a much higher recall rate than the BERT model proving that the visual
appearance does help with the relationship reasoning. In ES5, we add the image
positional embedding to the image appearance. This embedding contains the
location and size of the object/subject. After the image positional embedding
is added, we observe a very consistent recall improvement over ES4. To analyze
the impact of the image features, we show a specific example of the
relationship reasoning with image features in the third row in (Fig. 3). We
notice that even though the predicate sleep on never appeared between person
and bench. VReBERT still has quite high confidence which prediction is
reasonable both in terms of spatial relation and appearance (e.g., a person
can sleep on a bench). With the support of the experiment results and the
mentioned case, we believe VReBERT can leverage image features for
relationship reasoning which complement each other.

Figure 4: Evaluation Losses of Different Training Settings. Initializing from
a large-scale pre-trained visual model can dramatically reduce the training
cost. With end-to-end fine-tuning, VReBERT achieved noticeable improvement
than using the pre-trained image feature.

Is end-to-end fine-tuning necessary for VRD?
ES3 uses a Fast R-CNN model pre-trained on the VRD dataset (after S2 training
as described in Sec. II-E). We pass a pre-trained image feature to the BERT
model just like [4]. We did not observe a significant improvement over the
pure language model. In ES4, we fine-tune VReBERT and Fast R-CNN end-to-end on
the ground truth ROIs and relationship labels (at S3 of multi-stage training)
as see that VReBERT with fine-tuned image features obtained a huge improvement
(R@50 from 61.40% to 83.24%, R@100 from 85.11% to 92.77%) over VReBERT with
the pre-trained image features. To intuitively show the impact of fine-tuning,
we analyzed the evaluation loss of different pre-training and fine-tuning
settings. We randomly choose 400 images as the evaluation set. We train our
model on the rest of the training set and evaluate it on the evaluation set.
We show in Fig. 4, that the setting with pre-trained image feature without
fine-tuning has much higher and unstable evaluation loss than the setting with
fine-tuned features. We argue that BERT-like models may have trouble utilizing
the pre-trained image feature extracted for multi-layer perceptron (MLP).
Thus, end-to-end fine-tuning is necessary for improving the recall rate.
Furthermore, the pre-training dramatically accelerates the training progress.
Are relative sequence positional embeddings useful?
We utilize relative sequence positional embedding for mainly two reasons.
First, the learnable positional embedding is necessary for our model to be an
approximator of universal sequence-to-sequence functions. Second, the
alignment of the image and language token could be improved by it. The second
reason is quite important for multi-modal problems like visual relationship
reasoning. With a good alignment, the self-attention mechanism can better
understand the dependency of the tokens. In our experiment, we see that ES6
has a visible but minor recall gain over ES5. It is hard to confirm that
relative sequence positional embedding will always lead to a performance gain
based only on the current experimental results. We try to visualize it in Fig.
5, where we plot a graph of the relative sequence positional embedding score.
This score will be added directly to the scaled attention score which can in-
turn impact VReBERT’s understanding of dependency. A higher score means higher
dependency on the token. From the figure, we can find out the embedding value
of corresponding text and image token is much higher than other tokens, which
indicates the relative sequence positional embedding properly learned the
alignment of the image and language. This further adds to the importance of
such learned positional embedding.

Overall, our full model outperforms the state-of-the-art result [9] in the
predicate prediction task by $2.93\%$ in R@50 and $0.61\%$ in R@100.

Figure 5: Relative Sequence Positional Embedding Score. This figure shows that
the trainable sequence positional embedding properly learned the alignment of
the image and text. For example. The embedding shows a significantly higher
value between the predicate mask and the predicate image token, which
indicates the predicate mask is highly related to the predicate image token in
sequence location.

###  III-F Zero-shot Predicate Prediction

Further to traditional Predicate Prediction, we evaluate VReBERT on
challenging zero-shot predicate prediction. The task is to predict the
predicate category based on the ground truth object position and label in a
zero-shot setting. For this task, we test our model only on the zero-shot test
set which contains $1877$ relationships that never appeared in the training
data. We design this test to evaluate our model’s generalization capability.

We mainly focus on the performance impact of pre-training on the large-scale
dataset. To compare with our VReBERT fully trained using the multi-stage
training strategy, we trained a VReBERT model from scratch without pre-
training on a large-scale language dataset (S1 in Sec. II-E). We also compare
VReBERT with other comparable methods (e.g., VRD [4], LKD [9]) in the same
zero-shot setting.

We show in Tab. II, that our full model achieves $65.30\%$ in R@50 and
$85.41\%$ in R@100. It is a significant improvement from the state-of-the-art
models. VTransE aimed to find a proper embedding for object and predicate.
When optimizing the embedding space, it is hard to make it smooth and
generalize well on the unseen dataset. Later works like [9] utilized both
internal and external knowledge for visual relation detection. We think the
performance boost is due to the large-scale pre-training that helps bridge the
semantic gap between the relationship tags and contextual visual relationship.
By doing so the model is able to predict may unseen relation (e.g., <person-
sleep on-bench) even though it has only seen examples of <person-on-bench>. We
hope our findings can help further research how to better utilize such pre-
training for other zero-shot multimodal tasks.

TABLE II: Zero-shot Predicate Prediction Performances Method |  R@50 (%) |  R@100 (%)
---|---|---
VRD-Full [4] |  29.77 |  50.04
LKD [9] |  56.81 |  76.42
VReBERT from scratch |  52.06 |  81.75
Pre-trained VReBERT |  $\mathbf{65.30}$ |  $\mathbf{85.41}$
Improvement over SOTA |  8.49 $\uparrow$ |  8.99 $\uparrow$

##  IV Conclusion

We proposed a simple and flexible framework for visual relationship detection.
Our model leverages multi-head self-attention to conquer two challenges faced
by traditional visual relationship detection methods, which are polysemy
understanding and context-aware prediction of multiple predicates in different
categories. By redesigning the embedding layer of the BERT model, we fully
take advantage of visual and linguistic features to predict the relationship.
The experimental results on Visual Relationship Dataset show that our
framework is in line with the best visual relationship detection methods. We
show that our proposed VReBERT model is able to achieve state-of-the-art
performance in the visual relationship detection tasks and significantly
improve zero-shot predicate prediction. The experiment on the zero-shot test
set proved that the multi-stage training strategy proposed in this paper gives
VReBERT good generalization ability.

## References

  * [1] S. Antol, A. Agrawal, J. Lu, M. Mitchell, D. Batra, C. L. Zitnick, and D. Parikh, “Vqa: Visual question answering,” in _Proceedings of the IEEE international conference on computer vision_ , 2015, pp. 2425–2433.
  * [2] P. Anderson, X. He, C. Buehler, D. Teney, M. Johnson, S. Gould, and L. Zhang, “Bottom-up and top-down attention for image captioning and visual question answering,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2018, pp. 6077–6086.
  * [3] M. Farazi, S. Khan, and N. Barnes, “Accuracy vs. complexity: A trade-off in visual question answering models,” _Pattern Recognition_ , p. 108106, 2021\.
  * [4] C. Lu, R. Krishna, M. Bernstein, and L. Fei-Fei, “Visual relationship detection with language priors,” in _European conference on computer vision_. Springer, 2016, pp. 852–869.
  * [5] H. Zhang, Z. Kyaw, S.-F. Chang, and T.-S. Chua, “Visual translation embedding network for visual relation detection,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2017, pp. 5532–5540.
  * [6] H. Ben-younes, R. Cadène, N. Thome, and M. Cord, “BLOCK: bilinear superdiagonal fusion for visual question answering and visual relationship detection,” in _The Thirty-Third AAAI Conference on Artificial Intelligence, AAAI 2019, The Thirty-First Innovative Applications of Artificial Intelligence Conference, IAAI 2019, The Ninth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2019, Honolulu, Hawaii, USA, January 27 - February 1, 2019_. AAAI Press, 2019, pp. 8102–8109.
  * [7] K. Liang, Y. Guo, H. Chang, and X. Chen, “Visual relationship detection with deep structural ranking,” in _Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018_ , S. A. McIlraith and K. Q. Weinberger, Eds. AAAI Press, 2018, pp. 7098–7105.
  * [8] W. Liao, B. Rosenhahn, L. Shuai, and M. Y. Yang, “Natural language guided visual relationship detection,” in _IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2019, Long Beach, CA, USA, June 16-20, 2019_. Computer Vision Foundation / IEEE, 2019, pp. 444–453.
  * [9] R. Yu, A. Li, V. I. Morariu, and L. S. Davis, “Visual relationship detection with internal and external linguistic knowledge distillation,” in _IEEE International Conference on Computer Vision, ICCV 2017, Venice, Italy, October 22-29, 2017_. IEEE Computer Society, 2017, pp. 1068–1076.
  * [10] H. Tan and M. Bansal, “Lxmert: Learning cross-modality encoder representations from transformers,” in _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing_ , 2019.
  * [11] W. Su, X. Zhu, Y. Cao, B. Li, L. Lu, F. Wei, and J. Dai, “Vl-bert: Pre-training of generic visual-linguistic representations,” in _International Conference on Learning Representations_ , 2020. [Online]. Available: https://openreview.net/forum?id=SygXPaEYvH
  * [12] Y.-C. Chen, L. Li, L. Yu, A. E. Kholy, F. Ahmed, Z. Gan, Y. Cheng, and J. Liu, “Uniter: Universal image-text representation learning,” in _ECCV_ , 2020\.
  * [13] S. Ren, K. He, R. B. Girshick, and J. Sun, “Faster R-CNN: towards real-time object detection with region proposal networks,” in _Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, December 7-12, 2015, Montreal, Quebec, Canada_ , C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett, Eds., 2015, pp. 91–99.
  * [14] T. Xiao, Y. Liu, B. Zhou, Y. Jiang, and J. Sun, “Unified perceptual parsing for scene understanding,” in _Computer Vision - ECCV 2018 - 15th European Conference, Munich, Germany, September 8-14, 2018, Proceedings, Part V_ , ser. Lecture Notes in Computer Science, V. Ferrari, M. Hebert, C. Sminchisescu, and Y. Weiss, Eds., vol. 11209. Springer, 2018, pp. 432–448.
  * [15] Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” _CoRR_ , vol. abs/2103.14030, 2021.
  * [16] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” in _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_. Minneapolis, Minnesota: Association for Computational Linguistics, Jun. 2019, pp. 4171–4186. [Online]. Available: https://aclanthology.org/N19-1423
  * [17] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in _2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016_. IEEE Computer Society, 2016, pp. 770–778.
  * [18] R. B. Girshick, “Fast R-CNN,” in _2015 IEEE International Conference on Computer Vision, ICCV 2015, Santiago, Chile, December 7-13, 2015_. IEEE Computer Society, 2015, pp. 1440–1448.
  * [19] M. Qi, W. Li, Z. Yang, Y. Wang, and J. Luo, “Attentive relational networks for mapping images to scene graphs,” in _IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019_. Computer Vision Foundation / IEEE, 2019, pp. 3957–3966.
  * [20] V. K. Nagaraja, V. I. Morariu, and L. S. Davis, “Modeling context between objects for referring expression understanding,” in _Computer Vision - ECCV 2016 - 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part IV_ , ser. Lecture Notes in Computer Science, B. Leibe, J. Matas, N. Sebe, and M. Welling, Eds., vol. 9908. Springer, 2016, pp. 792–807.
  * [21] C. Yun, S. Bhojanapalli, A. S. Rawat, S. J. Reddi, and S. Kumar, “Are transformers universal approximators of sequence-to-sequence functions?” in _8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020_. OpenReview.net, 2020.
  * [22] D. Qi, L. Su, J. Song, E. Cui, T. Bharti, and A. Sacheti, “Imagebert: Cross-modal pre-training with large-scale weak-supervised image-text data,” _CoRR_ , vol. abs/2001.07966, 2020.
  * [23] L. H. Li, M. Yatskar, D. Yin, C. Hsieh, and K. Chang, “Visualbert: A simple and performant baseline for vision and language,” _CoRR_ , vol. abs/1908.03557, 2019.
  * [24] Y. Wu, M. Schuster, Z. Chen, Q. V. Le, M. Norouzi, W. Macherey, M. Krikun, Y. Cao, Q. Gao, K. Macherey, J. Klingner, A. Shah, M. Johnson, X. Liu, L. Kaiser, S. Gouws, Y. Kato, T. Kudo, H. Kazawa, K. Stevens, G. Kurian, N. Patil, W. Wang, C. Young, J. Smith, J. Riesa, A. Rudnick, O. Vinyals, G. Corrado, M. Hughes, and J. Dean, “Google’s neural machine translation system: Bridging the gap between human and machine translation,” _CoRR_ , vol. abs/1609.08144, 2016.
  * [25] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” in _Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA_ , I. Guyon, U. von Luxburg, S. Bengio, H. M. Wallach, R. Fergus, S. V. N. Vishwanathan, and R. Garnett, Eds., 2017, pp. 5998–6008.
  * [26] Z. Huang, D. Liang, P. Xu, and B. Xiang, “Improve transformer models with better relative position embeddings,” in _Findings of the Association for Computational Linguistics: EMNLP 2020, Online Event, 16-20 November 2020_ , ser. Findings of ACL, T. Cohn, Y. He, and Y. Liu, Eds., vol. EMNLP 2020\. Association for Computational Linguistics, 2020, pp. 3327–3335.
  * [27] P. Shaw, J. Uszkoreit, and A. Vaswani, “Self-attention with relative position representations,” in _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT, New Orleans, Louisiana, USA, June 1-6, 2018, Volume 2 (Short Papers)_ , M. A. Walker, H. Ji, and A. Stent, Eds. Association for Computational Linguistics, 2018, pp. 464–468.
  * [28] Z. Yang, Z. Dai, Y. Yang, J. G. Carbonell, R. Salakhutdinov, and Q. V. Le, “Xlnet: Generalized autoregressive pretraining for language understanding,” in _Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada_ , H. M. Wallach, H. Larochelle, A. Beygelzimer, F. d’Alché-Buc, E. B. Fox, and R. Garnett, Eds., 2019, pp. 5754–5764.
  * [29] Z. Dai, Z. Yang, Y. Yang, J. G. Carbonell, Q. V. Le, and R. Salakhutdinov, “Transformer-xl: Attentive language models beyond a fixed-length context,” in _Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers_ , A. Korhonen, D. R. Traum, and L. Màrquez, Eds. Association for Computational Linguistics, 2019, pp. 2978–2988.
  * [30] G. E. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,” _CoRR_ , vol. abs/1503.02531, 2015.
  * [31] B. Dai, Y. Zhang, and D. Lin, “Detecting visual relationships with deep relational networks,” in _2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017_. IEEE Computer Society, 2017, pp. 3298–3308.


#  Study of Solutions of Certain kind of Non-Linear Differential Equations

Garima Pant and Manisha Saini  Garima Pant; department of mathematics,
university of delhi, delhi-110007, india<EMAIL_ADDRESS>department
of mathematics, university of delhi, delhi-110007, india.
<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract.

In this paper we study about the existence of solutions of certain kind of
non-linear differential and differential-difference equations. We give partial
answer to a problem which was asked by chen et al. in [13].

###### Key words and phrases:

Nevanlinna theory, entire function, difference equation, differential-
difference equation

###### 2020 Mathematics Subject Classification:

34M05, 30D35, 39B32

Research work of the first author is supported by research fellowship from
University Grants Commission (UGC), New Delhi, India.

The second author is Senior Research Fellow (UGC, New Delhi, India).

We assume that the readers are familiar with the standard notations of
Nevanlinna theory such as characteristic function $T(r,f)$, proximity function
$m(r,f)$, integrated counting function $N(r,f)$, unintegrated counting
function $n(r,f)$ and first main theorem for a meromorphic function $f$ which
is defined in the complex plane [2, 3, 18]. To make our paper self contained,
we present the elementary definitions of the order of growth $\rho(f)$ and
exponent of convergence of zeros $\lambda(f)$ of a meromorphic function $f$

$\rho(f)=\limsup_{r\to\infty}\frac{\log^{+}T(r,f)}{\log r}$

and

$\lambda(f)=\limsup_{r\to\infty}\frac{\log^{+}n(r,1/f)}{\log r},$

respectively. If $f$ is a meromorphic function, then $S(r,f)$ denotes such
quantities which are of growth $o(T(r,f))$ as $r\to\infty$, outside of a
possible exceptional set of finite linear measure. A meromorphic function
$h(z)$ is said to a small function of $f(z)$ if $T(r,h)=S(r,f)$ and vice
versa. Note that finite sum of the quantities which are of $S(r,f)$ kind is
again $S(r,f)$. A differential polynomial in $f$ is a polynomial in $f$ and
its derivatives with small function of $f$ as its coefficients. For a given
meromorphic function $f$ and a constant $c$, $f(z+c)$ is known as a shift of
$f$.
To study solubility and existence of solutions of non-linear differential
equations or differential-difference equations are interesting but also
difficult. Many researchers have been studied these equations, one may refer
to see [9, 10, 11, 15, 16].
In this paper, our aim is to present some studies on the solutions of certain
kind of non-linear differential equations and differential-difference
equations which have been studied even before. In 2021, Chen et al.[13]
studied the following non-linear differential-difference equation

$f^{n}(z)+wf^{n-1}(z)f^{\prime}(z)+q(z)e^{Q(z)}f(z+c)=u(z)e^{\nu(z)},$ (1)

where $n$ is a natural number, $w$ and $c\neq 0$ are constants. $q,Q,u$ and
$\nu$ are non constant polynomials.

In our first result, we study the same non-linear differential-difference
equation (1) and prove the following result:

######  Theorem A.

Suppose that $n\geq 3$ for $w\neq 0$ and $n\geq 2$ for $w=0$. Let $f$ be a
finite order transcendental entire solution of equation (1) satisfying
$\lambda(f)<\rho(f)$. Then $f$ satisfies one of the following:

  1. $(i)$

$\rho(f)<\deg\nu=\deg Q$ and $f=Ce^{-z/w}$, where $C$ is a non-zero constant.

  2. $(ii)$

$\rho(f)=\deg\nu=\deg Q$

Note that every non-vanishing transcendental entire function $f$ of non-zero
finite order satisfies $\lambda(f)<\rho(f)$. Therefore we get an immediate
corollary from the above theorem.

######  Corollary 1.

Suppose that $n\geq 3$ for $w\neq 0$ and $n\geq 2$ for $w=0$. If $f$ is a non-
vanishing transcendental solution of non-zero finite order of equation (1).
Then $f$ satisfies the conclusion of Theorem A.

Prior to Theorem A Theorem A, Chen et al.[13] proved the following result:

######  Theorem 1.

Assume $n\geq 3$ for $w\neq 0$ and $n\geq 2$ for $w=0$. Let $f$ be a non-
vanishing transcendental solution of finite order of equation (1). Then each
$f$ satisfies one of the following:

  1. $(i)$

$\rho(f)<\deg\nu=\deg Q$ and $f=Ce^{-z/w}$, where $C$ is a constant.

  2. $(ii)$

$\rho(f)=\deg Q\geq\deg\nu$

After stating the above theorem, Chen et al.[13] posed a problem namely
Problem $1.10$, in which they asked “Can the conclusion $(ii)$ in the above
theorem be further improved to $\rho(f)=\deg Q=\deg\nu$?”
In the Corollary 1, we have given partial answer to that problem.

Chen et al.[13] also studied a non-linear differential-difference equation in
which left hand side of equation (1) is unaltered and right hand side of the
same equation is replaced by $P_{1}e^{\lambda z}+P_{2}e^{-\lambda z}$. Then
they provided the following result:

######  Theorem 2.

Suppose $f$ is a transcendental entire solution with finite order of equation

$f^{n}(z)+wf^{n-1}(z)f^{{}^{\prime}}(z)+q(z)e^{Q(z)}f(z+c)=P_{1}e^{\lambda
z}+P_{2}e^{-\lambda z},$ (2)

where $n$ is a natural number, $w$ is a constant and $c,P_{1},P_{2},\lambda$
are non-zero constants, $q\not\equiv 0$ is a polynomial and $Q$ is a non-
constant polynomial. Then the following conclusions hold:

  1. (i)

If $n\geq 4$ for $w\neq 0$ and $n\geq 3$ for $w=0$, then each solution $f$
satisfies $\rho(f)=\deg Q=1$.

  2. (ii)

If $n\geq 1$ and $f$ is a solution which belongs to
$\Gamma_{0}=\\{e^{\alpha(z)}:\alpha(z)$ is a non-constant polynomial}, then

$f(z)=e^{(\lambda z/n)+a},\qquad Q(z)=-\frac{n+1}{n}\lambda z+b$

or

$f(z)=e^{(-\lambda z/n)+a},\qquad Q(z)=\frac{n+1}{n}\lambda z+b,$

where $a$ and $b$ are constants.

Now it is natural to ask what can we say about the solutions of equation (2)
when $n=3$ and $w\neq 0$? In this sequence, we prove the following result:

######  Theorem B.

Suppose that $w,c,P_{1},P_{2}$ and $\lambda$ are non-zero constants, $q(z)$ is
a non-zero polynomial, $Q(z)$ is a non constant polynomial. Then there does
not exist any finite order transcendental entire function satisfying equation

$f^{3}(z)+wf^{2}(z)f^{\prime}(z)+q(z)e^{Q(z)}f(z+c)=P_{1}e^{\lambda
z}+P_{2}e^{-\lambda z}.$ (3)

with a finite non-zero Borel exceptional value.

The following example shows that if $f$ is a transcendental entire function
satisfying equation (3), then $f$ may have zero as a Borel exceptional value.

######  Example 1.

The function $f(z)=2e^{3z}$ satisfies differential-difference equation

$f^{3}+f^{2}f^{{}^{\prime}}+e^{12z}f(z+\log 2)=32e^{9z}+16e^{-9z}.$

Here $f(z)$ has zero as a Borel exceptional value.

In 2020, Xue [7] and Chen et al. [1] studied a new kind of non-linear
differential equations and proved the following results:

######  Theorem 3.

[7] Suppose $n\geq 2$ and $k$ are integers, $P_{n-1}(z,f)$ is an algebraic
differential polynomial in $f(z)$ of degree at most $n-1$. Let $P_{i}$ and
$\alpha_{i}$ be non-zero constants for $i=1,2,3$, and
$|\alpha_{1}|>|\alpha_{2}|>|\alpha_{3}|$. If $f$ is a transcendental entire
function satisfying the following differential equation

$f^{n}(z)+P_{n-1}(z,f)=P_{1}e^{\alpha_{1}z}+P_{2}e^{\alpha_{2}z}+P_{3}e^{\alpha_{3}z},$
(4)

then $f(z)=Ce^{\alpha_{1}z/n}$, where $C$ is a non-zero constant such that
$C^{n}=P_{1}$, and $\alpha_{1},\alpha_{2},\alpha_{3}$ are in one line.

Note that an algebraic differential polynomial $P(z,f)$ is a polynomial in $f$
whenever $f$ is a polynomial in $f(z)$ and its derivatives with polynomials as
its coefficients.

######  Theorem 4.

[1] Suppose $n\geq 5$ is an integer and $Q_{d}(z,f)$ is a differential
polynomial in $f$ of degree $d\leq n-4$ with rational functions as its
coefficients. Also suppose $p_{1}(z),p_{2}(z),p_{3}(z)$ are non-zero rational
functions and $\alpha_{1},\alpha_{2},\alpha_{3}$ are non constant polynomials
such that
$\alpha_{1}^{{}^{\prime}},\alpha_{2}^{{}^{\prime}},\alpha_{3}^{{}^{\prime}}$
are distinct to each other. If $f$ is a meromorphic solution with finitely
many poles of the following differential equation

$f^{n}(z)+Q_{d}(z,f)=p_{1}(z)e^{\alpha_{1}(z)}+p_{2}(z)e^{\alpha_{2}(z)}+p_{3}(z)e^{\alpha_{3}(z)},$
(5)

then $\frac{\alpha_{1}^{{}^{\prime}}}{\alpha_{2}^{{}^{\prime}}}$,
$\frac{\alpha_{2}^{{}^{\prime}}}{\alpha_{3}^{{}^{\prime}}}$ are rational
numbers, and $f(z)$ must be of

$f(z)=r(z)e^{P(z)},$

where $r(z)$ is a non-zero rational function and $P(z)$ is a non constant
polynomial. Furthermore, there must exist positive integers
$m_{1},m_{2},m_{3}$ with $\\{m_{1},m_{2},m_{3}\\}=\\{1,2,3\\}$ and distinct
integers $k_{1},k_{2}$ with $1\leq k_{1},k_{2}\leq d$ such that
$\alpha_{m_{1}}^{{}^{\prime}}:\alpha_{m_{2}}^{{}^{\prime}}:\alpha_{m_{3}}^{{}^{\prime}}=n:k_{1}:k_{2}$,
$nP^{\prime}=\alpha_{m_{1}}^{{}^{\prime}}$, $Q_{d}(z,f)\equiv
p_{m_{2}}(z)e^{\alpha_{m_{2}}(z)}+p_{m_{3}}(z)e^{\alpha_{m_{3}}(z)}$.

After stating the above result, Chen et al. [1] mentioned that “in Theorem 4,
the condition $n\geq 5$ is necessary” in a remark namely ’Remark $1.2$’. By
some examples, they have also shown that if $n\leq 4,$ then Theorem 4 may not
hold. In this sequence we prove the following results:

######  Theorem C.

Let $n\geq 2$, and $P(z,f)$ be a linear differential polynomial in $f(z)$, its
derivatives with coefficients being rational functions. Suppose $P_{j}(z)$ and
$\alpha_{j}(z)$ $(j=1,2,3)$ are non-zero rational functions and non constant
polynomials, respectively. Also
$\alpha_{1}^{{}^{\prime}}(z),\alpha_{2}^{{}^{\prime}}(z)$and
$\alpha_{3}^{{}^{\prime}}(z)$ are distinct to each other. If $f$ is a
transcendental meromorphic solution of the following differential equation

$f^{n}(z)+P(z,f)=P_{1}(z)e^{\alpha_{1}(z)}+P_{2}(z)e^{\alpha_{2}(z)}+P_{3}(z)e^{\alpha_{3}(z)},$
(6)

such that $f$ has finitely many poles, then $f$ is of finite order but not in
the form of $f(z)=s(z)e^{q(z)}$, where $s(z)$ is a non-zero rational function
and $q(z)$ is a non constant polynomial.

Note that Theorem 4 is an improvement of Theorem 3 which can be seen by the
following examples.

######  Example 2.

The differential equation

$f^{2}+f^{{}^{\prime\prime}}-2f^{{}^{\prime}}-f=2e^{2z}+2e^{3z}+e^{4z}$

has a solution $f(z)=1+e^{z}+e^{2z}$ which is not in the required form as in
Theorem 3.

######  Example 3.

The differential equation

$f^{3}-2f^{{}^{\prime}}-4f=e^{9z}+2e^{3z}+6e^{z}$

has a solution $f(z)=e^{3z}+2$ which is not in the required form as in Theorem
3.

######  Example 4.

The differential equation

$f^{4}+\frac{3}{2}f^{{}^{\prime}}+f=16e^{8z}-32e^{6z}+23e^{4z}$

has a solution $f(z)=2e^{2z}-1$ which is not in the required form as in
Theorem 3.

We observe that the above examples also hold Theorem C. Next, we combine
Theorem 4, Theorem C and we have an immediate result:

######  Corollary 2.

Let $n\geq 5$, and $P(z,f)$ be a linear differential polynomial in $f(z)$, its
derivatives with rational functions as its coefficients. Suppose $P_{j}(z)$
and $\alpha_{j}(z)$ $(j=1,2,3)$ are non-zero rational functions and non
constant polynomials, respectively. Also
$\alpha_{1}^{{}^{\prime}}(z),\alpha_{2}^{{}^{\prime}}(z),\alpha_{3}^{{}^{\prime}}(z)$
are distinct to each other. Then there exist no meromorphic solution having
finitely many poles which satisfies equation (6).

Before stating next result we first define that $Q_{2}^{\star}(z,f)$ is a
differential polynomial of degree $2$ with rational functions $a_{i}(z)$ as
its coefficients and at least one coefficient of $f^{(l)}(l\geq 0)$ must be
non-zero.

######  Theorem D.

Suppose $P_{1},P_{2},P_{3}$ are non-zero constants, and
$\alpha_{1},\alpha_{2},\alpha_{3}$ are non-zero and distinct constants. If $f$
is an entire solution of the following differential equation

$f^{n}(z)+Q_{2}^{\star}(z,f)=P_{1}e^{\alpha_{1}z}+P_{2}e^{\alpha_{2}z}+P_{3}e^{\alpha_{3}z},$
(7)

where $n\geq 3$ and satisfying $\lambda(f)<\rho(f)$. Then $\rho(f)=1$ and
$f(z)$ must be in the form of $f(z)=Ce^{az}$, where $C$ and $a$ are non-zero
constant. Particularly, we can say that there exist positive integers
$k_{1},k_{2},k_{3}$ with $\\{k_{1},k_{2},k_{3}\\}=\\{1,2,3\\}$ such that
$\alpha_{k_{1}}:\alpha_{k_{2}}:\alpha_{k_{3}}=n:2:1$, $na=\alpha_{k_{1}}$ and
$C=P_{k_{1}}^{1/n}$.

By the following examples, we show that our hypothesis keeps necessary
conditions to conclude the result.

######  Example 5.

The differential equation

$f^{3}+3f^{2}=e^{3z}-6e^{2z}+9e^{z}$

has a solution $f(z)=e^{z}-3$, which is not in the required form, here
$3f^{2}$ is not in the form of $Q_{2}^{\star}(z,f)$.

######  Example 6.

The differential equation

$f^{3}+2f^{2}+f^{{}^{\prime 2}}=e^{3z}-3e^{2z}+4e^{z}$

has a solution $f(z)=e^{z}-2$, which is not in the required form, here
$2f^{2}+f^{{}^{\prime 2}}$ is not in the form of $Q_{2}^{\star}(z,f)$.

######  Example 7.

The differential equation

$f^{3}+ff^{{}^{\prime}}-f=e^{3z}-2e^{2z}+e^{z}$

has a solution $f(z)=e^{z}-1$, which is not in the required form and
$\lambda(f)=\rho(f)$.

######  Example 8.

The differential equation

$f^{4}-f^{2}-5f^{{}^{\prime 2}}=e^{4z}-4e^{3z}-2e^{z}$

has a solution $f(z)=e^{z}-1$, which is not in the required form and given
coefficients of exponent in the right hand side of the above equation don’t
preserve required ratio, here $-f^{2}-5f^{{}^{\prime 2}}$ is not in the form
of $Q_{2}^{\star}(z,f)$.

Note that Theorem D is an improvement of Theorem 3.

##  1\. Preliminary Results

In this section, we collect those lemmas which we use to prove our desired
results.

The following lemma is related to the proximity function of logarithmic
derivative of a meromorphic function $f$.

######  Lemma 1.

[2] Suppose $f$ is a transcendental meromorphic function and $k\geq 1$ is an
integer. Then

$m\left(r,\frac{f^{(k)}}{f}\right)=S(r,f).$

The following lemma estimates the characteristic function of a shift of a
meromorphic function $f$.

######  Lemma 2.

[14] Suppose $f$ is a meromorphic function of finite order $\rho$ and $c$ is a
non-zero complex constant. Then for every $\epsilon$,

$T(r,f(z+c))=T(r,f)+O(r^{\rho-1+\epsilon})+O(\log r).$

Next lemma is the difference analogue of the lemma on the logarithmic
derivative of a meromorphic function $f$ having finite order.

######  Lemma 3.

[14] Suppose $f$ is a meromorphic function with $\rho(f)<\infty$ and
$c_{1},c_{2}\in\mathbb{C}$ such that $c_{1}\neq c_{2}$, then for each
$\epsilon>0$, we have

$m\left(r,\frac{f(z+c_{1})}{f(z+c_{2})}\right)=O(r^{\rho-1+\epsilon}).$

The following lemma estimates the characteristic function of an exponential
polynomial $f$. This lemma can be seen in [17].

######  Lemma 4.

Suppose $f$ is an entire function given by

$f(z)=A_{0}(z)+A_{1}(z)e^{w_{1}z^{s}}+A_{2}(z)e^{w_{2}z^{s}}+...+A_{m}(z)e^{w_{m}z^{s}},$

where $A_{i}(z);0\leq i\leq m$ denote either exponential polynomial of degree
$<s$ or polynomial in $z$, $w_{i};1\leq i\leq m$ denote the constants and $s$
denotes a natural number. Then

$T(r,f)=C(Co(W_{0}))\frac{r^{s}}{2\pi}+o(r^{s}),$

Here $C(Co(W_{0}))$ is the perimeter of the convex hull of the set
$W_{0}=\\{0,\tilde{w_{1}},\tilde{w_{2}},...,\tilde{w_{m}}\\}$.

The following lemma plays a key role to prove all our results in this article.

######  Lemma 5.

[12] Suppose $f_{1},f_{2},...,f_{n}(n\geq 2)$ are meromorphic functions and
$h_{1},h_{2},...,h_{n}$ are entire functions satisfying

  1. (1)

$\sum_{i=1}^{n}f_{i}e^{h_{i}}\equiv 0$.

  2. (2)

For $1\leq j<k\leq n$, $h_{j}-h_{k}$ are not constants .

  3. (3)

For $1\leq i\leq n,1\leq m<k\leq n$,
$T(r,f_{i})=o(T(r,e^{(h_{m}-h_{k})}))$ as $r\to\infty$, outside a set of
finite logarithmic measure.

Then $f_{i}\equiv 0$ $(i=1,2,...,n).$

##  2\. Proof of Theorems

######  Proof of Theorem A.

Given that $f$ is a transcendental entire solution of finite order of equation
(1) satisfying $\lambda(f)<\rho(f)$.
Case 1: First, we consider $n\geq 3$ for $w\neq 0$, then using Weierstrass
factorisation theorem, $f$ can be written as

$f(z)=\alpha(z)e^{\beta(z)},$ (8)

where $\beta(z)$ is a non constant polynomial of degree $t$ and $\alpha(z)$ is
an entire function satisfies $\rho(\alpha)=\lambda(f)<\rho(f)=t$.
For simplicity, rewriting (1) as

$f^{n}+wf^{n-1}f^{\prime}+qe^{Q}f_{c}=ue^{\nu},$ (9)

where $f_{c}=f(z+c)$. Now, we study the following three sub cases.
Case 1.1: When $\rho(f)<\deg Q$, then we have $T(r,f)=S(r,e^{Q})$.
Also, it is well known that a meromorphic function and its derivative have
same order, thus we have $T(r,f^{\prime})=S(r,e^{Q})=T(r,f)$. Using Lemma 2,
we obtain

$T(r,f_{c})=S(r,e^{Q})$

.
From equation (9), we get

$\displaystyle T(r,ue^{\nu})$
$\displaystyle=T(r,f^{n}+wf^{n-1}f^{\prime}+qe^{Q}f_{c})$ $\displaystyle\leq
T(r,f^{n-1})+T(r,f+wf^{\prime})+T(r,q)+T(r,e^{Q})+T(r,f_{c})+\log 2$
$\displaystyle\leq nT(r,f)+T(r,f^{\prime})+T(r,e^{Q})+T(r,f_{c})+T(r,q)+O(1)$
$\displaystyle=T(r,e^{Q})+S(r,e^{Q})\qquad\mbox{(Since
$T(r,f)=T(r,f^{\prime})=T(r,f_{c})=S(r,e^{Q})$)}$

and

$\displaystyle T(r,e^{Q})$
$\displaystyle=T\left(r,\frac{qf_{c}e^{Q}}{qf_{c}}\right)$ $\displaystyle\leq
T(r,qf_{c}e^{Q}+wf^{n-1}f^{\prime}+f^{n})+T(r,wf^{n-1}f^{\prime}+f^{n})+T\left(r,\frac{1}{qf_{c}}\right)+O(1)$
$\displaystyle\leq
T(r,ue^{\nu})+nT(r,f)+T(r,f^{\prime})+T(r,f_{c})+T(r,q)+O(1)$
$\displaystyle=T(r,ue^{\nu})+S(r,e^{Q}).$

From the last two inequalities,

$T(r,e^{Q})=T(r,ue^{\nu})+S(r,e^{Q}).$

This implies $\deg Q=\deg\nu$. Now Differentiation of equation (9) gives

$nf^{n-1}f^{\prime}+w(n-1)f^{n-2}(f^{\prime})^{2}+wf^{n-1}f^{{}^{\prime\prime}}+Ae^{Q}=(u^{\prime}+u\nu^{\prime})e^{\nu},$
(10)

where, $A=q^{\prime}f_{c}+qQ^{\prime}f_{c}+qf^{\prime}_{c}$. Eliminating
$e^{\nu}$ from equation (9) and (10), then we obtain

$D_{1}e^{Q}+D_{2}=0,$

where

$D_{1}=(u^{\prime}+u\nu^{\prime})qf_{c}-uA,$
$D_{2}=(u^{\prime}+u\nu^{\prime})(f^{n}+wf^{n-1}f^{\prime})-u(nf^{n-1}f^{\prime}+w(n-1)f^{n-2}(f^{\prime})^{2}+wf^{n-1}f").$

Using Lemma 5, we have $D_{1}=D_{2}\equiv 0$.
From $D_{1}=0$, we get

$\frac{u^{\prime}}{u}+\nu^{\prime}=\frac{q^{\prime}}{q}+\frac{f^{\prime}_{c}}{f_{c}}+Q^{\prime}.$

On doing integration of the above equation, we have

$qf_{c}e^{Q}=C_{1}ue^{\nu}.$

where $C_{1}$ is a non-zero constant. If $C_{1}=1$, then
$qf_{c}e^{Q}=ue^{\nu}$ and by equation (9), we obtain

$f^{n}+wf^{n-1}f^{\prime}=0.$

This yields that $f=Ce^{-z/w}$, where $C$ is a non-zero constant. This is the
required conclusion $(i)$.
If $C_{1}\neq 1$, then $qf_{c}e^{Q}=C_{1}ue^{\nu}$. This implies that
$f=\frac{C_{1}u_{-c}e^{\nu_{-c}-Q_{-c}}}{q_{-c}},$ where $u_{-c}=u(z-c)$,
$\nu_{-c}=\nu(z-c)$ and $Q_{-c}=Q(z-c)$.
Substituting the above value of $f$ into the equation (9), we have

$\begin{split}\left(\frac{C_{1}u_{-c}}{q_{-c}}\right)^{n-1}\left[\frac{C_{1}u_{-c}}{q_{-c}}+C_{1}w\left(\left(\frac{u_{-c}}{q_{-c}}\right)^{{}^{\prime}}+\left(\frac{u_{-c}}{q_{-c}}(\nu_{-c}-Q_{-c})^{{}^{\prime}}\right)\right)\right]&e^{n(\nu_{-c}-Q_{-c})}=\\\
&(1-C_{1})ue^{\nu}.\end{split}$

Since $\deg Q=\deg\nu>\rho(f)=\deg\beta\geq 1$ and
$\rho(f)=\deg(\nu_{-c}-Q_{-c})$. Using Lemma 5, we have $(1-C_{1})u\equiv 0$.
This implies either $C_{1}=1$ or $u=0$, which is not possible.
Case 1.2: When $\rho(f)=\deg Q$, we substitute the value of $f$ from equation
(8) into equation (9)

$(\alpha^{n}+w\alpha^{n-1}(\alpha^{{}^{\prime}}+\beta^{{}^{\prime}}\alpha))e^{n\beta}+q\alpha_{c}e^{Q+\beta_{c}}=ue^{\nu},$
(11)

where $\beta_{c}=\beta(z+c)$. Now, we consider
$\beta(z)=a_{t}z^{t}+a_{t-1}z^{t-1}+...+a_{0},(a_{t}\neq 0)$ and
$Q(z)=b_{t}z^{t}+b_{t-1}z^{t-1}+...+b_{0},(b_{t}\neq 0)$ are polynomials of
degree $t$ provided $\rho(f)=\deg\beta=t$.
Let $\deg\nu<\deg Q=\rho(f)=\deg\beta=t$, then from equation (11), we have

$[\alpha^{n}+w\alpha^{n-1}(\alpha^{\prime}+\alpha\beta^{\prime})]e^{\alpha_{1}}e^{na_{t}z^{t}}+q\alpha_{c}e^{\alpha_{2}}e^{(b_{t}+a_{t})z^{t}}=ue^{\nu},$
(12)

where $\alpha_{1}=n(a_{t-1}z^{t-1}+...+a_{0})$ and
$\alpha_{2}=(b_{t-1}+cta_{t}+a_{t-1})z^{t-1}+...+(b_{0}+ca_{t}+a_{0})$ are
polynomials of degree at most $t-1$.
If $a_{t}\neq\pm b_{t}$ , then using Lemma 5, we get $u\equiv 0$, which is not
possible.
If $a_{t}=b_{t}$, then again using Lemma 5, we get the same contradiction.
If $a_{t}=-b_{t}$, then

$[\alpha^{n}+w\alpha^{n-1}(\alpha^{\prime}+\alpha\beta^{\prime})]e^{\alpha_{1}}e^{na_{t}z^{t}}=ue^{\nu}-q\alpha_{c}e^{\alpha_{2}}.$

Using Lemma 5, we get

$\alpha^{n-1}(\alpha+w(\alpha^{\prime}+\alpha\beta^{\prime}))\equiv 0.$

This gives $\alpha=C_{2}e^{-z/w-\beta}$, where $C_{2}$ is a non-zero constant,
which implies $\rho(\alpha)=\deg\beta=\rho(f)$. This is a contradiction to the
fact that $\rho(\alpha)<\rho(f)$. Thus $\deg\nu<\deg Q$ is not possible.
Let $\deg\nu>\deg Q=\rho(f)=t$ and we consider
$\nu(z)=\nu_{s}z^{s}+\nu_{s-1}z^{s-1}+...+\nu_{0}$ such that $s>t$, then from
equation (11), we have

$[\alpha^{n}+w\alpha^{n-1}(\alpha^{\prime}+\alpha\beta^{\prime})]e^{\alpha_{1}}e^{na_{t}z^{t}}+q\alpha_{c}e^{\alpha_{2}}e^{(b_{t}+a_{t})z^{t}}=ue^{\alpha_{3}}e^{\nu_{s}z^{s}},$

where $\alpha_{1}$, $\alpha_{2}$ are same polynomials as in equation (12) and
$\alpha_{3}=\nu_{s-1}z^{s-1}+...+\nu_{0}$ is a polynomial of degree at most
$s-1$.
If $a_{t}\neq\pm b_{t}$, then using Lemma 5, we get $u\equiv 0$, which is not
possible.
If $a_{t}=b_{t}$, then again using Lemma 5, we get $u\equiv 0$, which is not
possible.
If $a_{t}=-b_{t}$, then we also get same contradiction as for the above cases.
Therefore $\deg\nu>\deg Q$ is not possible. Hence $\deg\nu=\deg Q=\rho(f)$,
which is the required conclusion $(ii)$.
Case 1.3: If $\rho(f)>\deg Q$, then from equation (9)

$\displaystyle T(r,ue^{\nu})$
$\displaystyle=T(r,f^{n}+wf^{n-1}f^{\prime}+qe^{Q}f_{c})$ $\displaystyle\leq
T(r,f^{n-1})+T(r,f+wf^{\prime})+T(r,q)+T(r,e^{Q})+T(r,f_{c})+O(1)$
$\displaystyle\leq(n+1)T(r,f)+T(r,\frac{f+wf^{\prime}}{f})+S(r,f)$
$\displaystyle=(n+1)T(r,f)+S(r,f).$

This implies $\deg\nu\leq\rho(f)=\deg\beta$. Now, we discuss the following sub
cases:

  1. (i)

If $\deg\nu<\deg\beta$, we consider
$\beta(z)=a_{t}z^{t}+a_{t-1}z^{t-1}+...+a_{0},(a_{t}\neq 0)$ and
$Q(z)=b_{d}z^{d}+b_{d-1}z^{d-1}+...+b_{0},(b_{d}\neq 0)$ such that
$d<\rho(f)=t$, then from equation (11), we have

$[\alpha^{n}+w\alpha^{n-1}(\alpha^{\prime}+\alpha\beta^{\prime})]e^{\alpha_{1}}e^{na_{t}z^{t}}+q\alpha_{c}e^{\tilde{\alpha_{2}}}e^{a_{t}z^{t}}=ue^{\nu},$

where $\tilde{\alpha_{2}}=Q(z)+a_{t-1}z^{t-1}+...+a_{0}$ is a polynomial of
degree at most $t-1$. Then using Lemma 5 into the above equation, we get
$u\equiv 0$, which is not possible.

  2. (ii)

If $\deg\nu=\deg\beta$, then we consider
$\nu(z)=\nu_{t}z^{t}+\nu_{t-1}z^{t-1}+...+\nu_{0},\nu_{t}\neq 0$. From
equation (11)

$[\alpha^{n}+w\alpha^{n-1}(\alpha^{\prime}+\alpha\beta^{\prime})]e^{\alpha_{1}}e^{na_{t}z^{t}}+q\alpha_{c}e^{\tilde{\alpha_{2}}}e^{a_{t}z^{t}}=ue^{\tilde{\alpha_{3}}}e^{\nu_{t}z^{t}},$

where $\tilde{\alpha_{3}}=\nu_{t-1}z^{t-1}+...+\nu_{0}$ is a polynomial of
degree at most $t-1$.
If $\nu_{t}\neq na_{t}$ and $\nu_{t}\neq a_{t}$, then using Lemma 5,
$q\alpha_{c}\equiv 0$, which is not possible.
If $\nu_{t}=na_{t}$, then again using Lemma 5, we get the same contradiction
as above.
If $\nu_{t}=a_{t}$, then

$[\alpha^{n}+w\alpha^{n-1}(\alpha^{\prime}+\alpha\beta^{\prime})]e^{\alpha_{1}}e^{na_{t}z^{t}}+(q\alpha_{c}e^{\tilde{\alpha_{2}}}-ue^{\tilde{\alpha_{3}}})e^{a_{t}z^{t}}\equiv
0,$

Using Lemma 5,

$\alpha^{n-1}(\alpha+w(\alpha^{\prime}+\alpha\beta^{\prime}))\equiv 0.$

$\implies$ $\alpha+w(\alpha^{\prime}+\alpha\beta^{\prime})=0$.
This gives that $\alpha=C_{3}e^{-z/w-\beta}$. This implies
$\deg\alpha=\deg\beta=\rho(f)$, which is not possible.

Case 2: If we consider $n\geq 2$ with $w=0$, then similarly we study 3 sub
cases as for Case 1.
Case 2.1: When $\rho(f)<\deg Q$, then proceeding similar lines as in Case 1.1,
we get $\deg Q=\deg\nu$. On differentiating equation $\eqref{chweqsimple}$, we
have

$nf^{n-1}f^{\prime}+(q^{\prime}f_{c}+qQ^{\prime}f_{c}+qf^{\prime}_{c})e^{Q}=(u^{\prime}+u\nu^{\prime})e^{\nu},$
(13)

Eliminating $e^{\nu}$ from equation (9) and (13), then we obtain

$D_{1}e^{Q}+D_{0}=0,$

where

$D_{1}=(u^{\prime}+u\nu^{\prime})qf_{c}-u(q^{\prime}f_{c}+qQ^{\prime}f_{c}+qf^{\prime}_{c}),$
$D_{0}=(u^{\prime}+u\nu^{\prime})f^{n}-nuf^{n-1}f^{\prime}.$

Using Lemma 5, we have $D_{1}=D_{0}\equiv 0$.
From $D_{1}\equiv 0$, we get

$\frac{u^{\prime}}{u}+\nu^{\prime}=\frac{q^{\prime}}{q}+\frac{f^{\prime}_{c}}{f_{c}}+Q^{\prime}.$

On doing integration of the above equation, we have

$qf_{c}e^{Q}=C_{1}ue^{\nu}.$

where $C_{1}$ is a non-zero constant.
If $C_{1}=1$, then $qf_{c}e^{Q}=ue^{\nu}$ and by equation (9), we obtain

$f^{n}\equiv 0.$

Which is not possible. Hence conclution (i) does not hold.
If $C_{1}\neq 1$, then again proceeding same lines to Case 1.1, we get same
contradiction.
Case 2.2: When $\rho(f)=\deg Q$, Using same argument as for Case 1.2, we get
required conclusion (ii).
Case 2.3: When $\rho(f)>\deg Q$, Using same argument as for Case 1.3,we get
same conradiction.
Hence we complete the proof. ∎

######  Proof of Theorem B.

We prove our result by contradiction. Suppose that $f$ is a finite order
transcendental entire function satisfying equation (3) with a non-zero finite
Borel exceptional value. Then we study following three cases:
Case 1: If $\rho(f)<1$, then using similar technique as in [[13],Theorem
1.11], we get contradiction.
Case 2: If $\rho(f)>1$ and let $a(\neq 0)$ be a finite Borel exceptional value
of $f$. Then using Weierstrass factorisation theorem, $f(z)$ can be
represented in the form

$f(z)=\alpha(z)e^{\beta(z)}+a,$ (14)

where $\beta(z)=a_{t}z^{t}+a_{r}z^{t}+...+a_{0};a_{t}\neq 0$ is a polynomial
of degree $t>1$ and $\alpha(z)$ is an entire function such that
$\rho(\alpha)<\deg\beta.$
From equation (3), we have

$\displaystyle\qquad\qquad m(r,e^{Q(z)})$
$\displaystyle=m\left(r,\frac{q(z)e^{Q(z)}f(z+c)}{q(z)f(z+c)}\right)$
$\displaystyle=m\left(r,\frac{P_{1}e^{\lambda z}+P_{2}e^{-\lambda
z}-f^{2}(z)(f(z)+wf^{{}^{\prime}}(z))}{q(z)f(z+c)}\right)$ $\displaystyle\leq
m\left(r,\frac{P_{1}e^{\lambda z}+P_{2}e^{-\lambda
z}}{q(z)f(z+c)}\right)+m\left(r,\frac{f^{2}(z)(f(z)+wf^{{}^{\prime}}(z))}{q(z)f(z+c)}\right)+O(1)$
$\displaystyle\leq m\left(r,\frac{1}{q(z)f(z+c)}\right)+m(r,P_{1}e^{\lambda
z}+P_{2}e^{-\lambda z})+m(r,f^{2})+$ $\displaystyle
m\left(r,\frac{1}{q(z)}\right)+m\left(r,\frac{f(z)}{q(z)f(z+c)}\right)+m\left(r,\frac{f^{\prime}(z)}{q(z)f(z+c)}\right)+O(1)$

Applying Lemma 1 and 3 into the above inequality, we have

$\displaystyle T(r,e^{Q(z)})$ $\displaystyle\leq
m\left(r,\frac{1}{q(z)f(z+c)}\right)+2T(r,f)+m\left(r,\frac{f^{\prime}(z)}{q(z)f(z+c)}\right)+S(r,f)$
$\displaystyle\leq
m\left(r,\frac{f(z)}{q(z)f(z+c)}\right)+m\left(r,\frac{1}{f(z)}\right)+2T(r,f)+m\left(r,\frac{f^{\prime}(z)f(z)}{q(z)f(z)f(z+c)}\right)$
$\displaystyle\qquad\qquad\qquad\qquad\qquad+S(r,f)$ $\displaystyle\leq
3T(r,f)+S(r,f).$

This implies that $\deg Q\leq\rho(f)=t$ and we also know that $\deg Q\geq 1$.
First suppose $1\leq\deg Q<t$, then substituting the value of $f(z)$ from
equation (14) into equation (3), we have

$\begin{split}A_{1}(z)e^{3\beta(z)}+A_{2}(z)e^{2\beta(z)}&+A_{3}(z)e^{\beta(z)}+aq(z)e^{Q(z)}+\\\
&q(z)\alpha(z+c)e^{Q(z)+\beta(z+c)}-P_{1}e^{\lambda z}-P_{2}e^{-\lambda
z}+a^{3}=0,\end{split}$ (15)

where

$\displaystyle A_{1}(z)$
$\displaystyle=\alpha^{3}(z)+w\alpha^{3}(z)\beta^{{}^{\prime}}(z)+w\alpha^{2}(z)\alpha^{{}^{\prime}}(z)$
$\displaystyle A_{2}(z)$
$\displaystyle=3a\alpha^{2}(z)+2wa\beta^{{}^{\prime}}(z)\alpha^{2}(z)+2wa\alpha(z)\alpha^{{}^{\prime}}(z)$
$\displaystyle A_{3}(z)$
$\displaystyle=3a^{2}\alpha(z)+wa^{2}\alpha(z)\beta^{{}^{\prime}}(z)+wa^{2}\alpha^{{}^{\prime}}(z).$

On simplifying equation (15), we get

$\begin{split}A_{1}(z)e^{3\beta(z)}+A_{2}(z)e^{2\beta(z)}&+(A_{3}(z)e^{\beta_{1}(z)}+q(z)\alpha(z+c)e^{\delta(z)})e^{a_{t}z^{t}}+\\\
&(aq(z)e^{Q(z)}-P_{1}e^{\lambda z}-P_{2}e^{-\lambda z}+a^{3})=0,\end{split}$
(16)

where $\beta_{1}(z)=a_{t-1}z^{t-1}+a_{t-2}z^{t-2}+...+a_{0}$ and
$\delta(z)=Q(z)+a_{t}[\binom{t}{1}cz^{t-1}+\binom{t}{2}c^{2}z^{t-2}+...+c^{t}]+a_{t-1}(z+c)^{t-1}+...+a_{0}$
are polynomials of degree at most $t-1$. Applying Lemma 5 into equation (16),
we have

$aq(z)e^{Q(z)}-P_{1}e^{\lambda z}-P_{2}e^{-\lambda z}+a^{3}\equiv 0$ (17)

Again applying Lemma 5 into equation (17), we have the following sub cases:

  1. (i)

If $Q(z)\neq\pm\lambda z$, then we have $P_{1}\equiv 0\equiv P_{2}=a^{3}$,
which is not possible.

  2. (ii)

If $Q(z)=\lambda z$, then we have $P_{2}\equiv 0\equiv a^{3}$, which is not
possible

  3. (iii)

If $Q(z)=-\lambda z$, then we have $P_{1}\equiv 0\equiv a^{3}$, which is not
possible.

Now suppose $\deg Q=t$, and let
$Q(z)=b_{t}z^{t}+b_{t-1}z^{t-1}+...+b_{0};b_{t}\neq 0$ is a polynomial. Then
from equation (15), we have

$\begin{split}A_{1}(z)e^{3\beta_{1}(z)}e^{3a_{t}z^{t}}&+A_{2}(z)e^{2\beta_{1}(z)}e^{2a_{t}z^{t}}+A_{3}(z)e^{\beta_{1}(z)}e^{a_{t}z^{t}}+aq(z)e^{Q_{1}(z)}e^{b_{t}z^{t}}+\\\
&q(z)\alpha(z+c)e^{Q_{1}(z)+\beta_{1}(z+c)}e^{b_{t}z^{t}+a_{t}(z+c)^{t}}-(P_{1}e^{\lambda
z}+P_{2}e^{-\lambda z}-a^{3})=0,\end{split}$ (18)

where $\beta_{1}(z)=a_{t-1}z^{t-1}+a_{t-2}z^{t-2}+...+a_{0}$ and
$Q_{1}(z)=b_{t-1}z^{t-1}+b_{t-2}z^{t-2}+...+b_{0}$ are polynomials of degree
at most $t-1$. Next we discuss the following sub cases:

  1. (i)

If $b_{t}\neq ka_{t}$ for any $k=1,2,3$, and $b_{t}\neq-a_{t}$ then applying
Lemma 5, we have $aq(z)e^{Q_{1}(z)}\equiv 0$, this implies either $a\equiv 0$
or $q(z)\equiv 0$, which is not possible.

  2. (ii)

If $b_{t}=ka_{t}$ for some $k=1,2,3$, say $b_{t}=a_{t}$, then equation (18)
becomes

$\begin{split}A_{1}(z)e^{3\beta_{1}(z)}e^{3a_{t}z^{t}}&+(A_{2}(z)e^{2\beta_{1}(z)}+q(z)\alpha(z+c)e^{\tilde{\delta}(z)})e^{2a_{t}z^{t}}+\\\
&(A_{3}(z)e^{\beta_{1}(z)}+aq(z)e^{Q_{1}(z)})e^{a_{t}z^{t}}-(P_{1}e^{\lambda
z}+P_{2}e^{-\lambda z}-a^{3})=0,\end{split}$

where
$\tilde{\delta}(z)=Q_{1}(z)+\beta_{1}(z+c)+a_{t}[\binom{t}{1}cz^{t-1}+\binom{t}{2}c^{2}z^{t-2}+...+c^{t}]$
is a polynomial of degree at most $t-1$. Applying Lemma 5 into the above
equation, we have $P_{1}e^{\lambda z}+P_{2}e^{-\lambda z}-a^{3}\equiv 0$,
which is not possible.

  3. (iii)

If $b_{t}=-a_{t}$, then equation (18) becomes

$\begin{split}A_{1}(z)e^{3\beta_{1}(z)}e^{3a_{t}z^{t}}&+A_{2}(z)e^{2\beta_{1}(z)}e^{2a_{t}z^{t}}+A_{3}(z)e^{\beta_{1}(z)}e^{a_{t}z^{t}}+aq(z)e^{Q_{1}(z)}e^{b_{t}z^{t}}+\\\
&(q(z)\alpha(z+c)e^{\tilde{\delta}(z)}-P_{1}e^{\lambda z}+P_{2}e^{-\lambda
z}-a^{3})=0,\end{split}$

Applying Lemma 5 into the above equation, we have $aq(z)e^{Q_{1}(z)}\equiv 0$,
this implies either $a\equiv 0$ or $q(z)\equiv 0$, which is not possible.

Case 3: If $\rho(f)=1$, then we can represent $f(z)$ as

$f(z)=\gamma(z)e^{bz+d}+a,$ (19)

where $b\neq 0,d\in\mathbb{C}$ and $\gamma(z)$ is an entire such that
$\rho(\gamma)<1$.
Now proceeding to similar lines as in case of $\rho(f)>1$, we have $\deg Q=1$.
Thus we can put $Q(z)=pz+s$, $(0\neq p,s\in\mathbb{C})$ in the equation (3),
we have

$B_{1}(z)e^{3bz}+B_{2}(z)e^{2bz}+B_{3}(z)e^{bz}+B_{4}(z)e^{(p+b)z}+aq(z)e^{s}e^{pz}+a^{3}-P_{1}e^{\lambda
z}-P_{2}e^{-\lambda z}=0,$ (20)

where

$\displaystyle B_{1}(z)$
$\displaystyle=(\gamma^{3}(z)+w\gamma^{2}(z)\gamma^{{}^{\prime}}(z)+wb\gamma^{3}(z))e^{3d}$
$\displaystyle B_{2}(z)$
$\displaystyle=(3a\gamma^{2}(z)+2aw\gamma(z)\gamma^{{}^{\prime}}(z)+2abw\gamma^{2}(z))e^{2d}$
$\displaystyle B_{3}(z)$
$\displaystyle=(3a^{2}\gamma(z)+a^{2}w\gamma^{{}^{\prime}}(z)+a^{2}bw\gamma(z))e^{d}$
$\displaystyle B_{4}(z)$ $\displaystyle=q(z)\gamma(z+c)e^{bc+s+d}.$

Here we discuss the following sub cases:

  1. (i)

If $kb\neq p$ and $kb\neq\pm\lambda$ for any $k=1,2,3$, then applying Lemma 5
to equation (20), we have $P_{1}\equiv 0\equiv P_{2}$, which is not possible.

  2. (ii)

If $kb=p$ for some $k=1,2,3$ and $kb\neq\pm\lambda$ for any $k=1,2,3$, then
applying Lemma 5 to equation (20), we obtain same contradiction as for
previous case.

  3. (iii)

If $kb\neq p$ for any $k=1,2,3$ and $kb=\lambda$ for some $k=1,2,3$, then
applying Lemma 5 to equation (20), we have $P_{2}\equiv 0$, which is not
possible.

  4. (iv)

If $kb\neq p$ for any $k=1,2,3$ and $kb=-\lambda$ for some $k=1,2,3$, then
applying Lemma 5 to equation (20), we have $P_{1}\equiv 0$, which is not
possible.

  5. (v)

If $kb=p$ and $kb=\lambda$ for some $k=1,2,3$, then applying Lemma 5 to
equation (20), we have $P_{2}\equiv 0\equiv a^{3}$, which is not possible.

  6. (vi)

If $kb=p$ and $kb=-\lambda$ for some $k=1,2,3$, then again applying Lemma 5 to
equation (20), we have $P_{1}\equiv 0\equiv a^{3}$, which is not possible.

Thus we complete the proof.
∎

######  Proof of Theorem C.

Suppose that $f$ is a transcendental meromorphic function satisfying equation
(6) and given that $f$ has finitely many poles, then from equation (6), and
Lemma 1 and 4,

$\displaystyle nT(r,f)=T(r,f^{n})$
$\displaystyle=T(r,P_{1}e^{\alpha_{1}(z)}+P_{2}e^{\alpha_{2}(z)}+P_{3}e^{\alpha_{3}(z)}-P(z,f))$
$\displaystyle\leq
T(r,P_{1}e^{\alpha_{1}(z)}+P_{2}e^{\alpha_{2}(z)}+P_{3}e^{\alpha_{3}(z)})+T(r,P(z,f))+\log
2$ $\displaystyle\leq
T(r,P_{1}e^{\alpha_{1}(z)}+P_{2}e^{\alpha_{2}(z)}+P_{3}e^{\alpha_{3}(z)})+m\left(r,\frac{P(z,f)}{f}f\right)+$
$\displaystyle\qquad\qquad\qquad\qquad N(r,P(z,f))+\log 2$ $\displaystyle\leq
Ar^{m}+T(r,f)+S(r,f),$

where

$A=\frac{\text{sum of the leading coefficients
of}\alpha_{1}(z),\alpha_{2}(z)\text{and}\alpha_{3}(z)}{\pi}$

and

$m=\max\\{\deg\alpha_{1}(z),\deg\alpha_{2}(z),\deg\alpha_{3}(z)\\}.$

Thus we have $(n-1)T(r,f)\leq Ar^{m}+S(r,f)$ which implies that $f$ has finite
order.
Now we prove rest part of the theorem by contradiction. Suppose
$f(z)=s(z)e^{q(z)}$ is a solution of equation (6), where $s(z)$ is a non-zero
rational function and $q(z)$ is a non constant polynomial, then from equation
(6), we get

$\displaystyle
P_{1}(z)e^{\alpha_{1}(z)}+P_{2}(z)e^{\alpha_{2}(z)}+P_{3}(z)e^{\alpha_{3}(z)}$
$\displaystyle=f^{n}(z)+P(z,f)$
$\displaystyle=f^{n}(z)+\sum_{i=0}^{i=l}a_{i}(z)f^{(i)}(z)+\tilde{a_{0}}(z)$
$\displaystyle=(s(z)e^{q(z)})^{n}+\sum_{i=0}^{i=l}a_{i}(z)(s(z)e^{q(z)})^{(i)}+\tilde{a_{0}}(z)$

This implies

$P_{1}(z)e^{\alpha_{1}(z)}+P_{2}(z)e^{\alpha_{2}(z)}+P_{3}(z)e^{\alpha_{3}(z)}-s^{n}(z)e^{nq(z)}-Q(z)e^{q(z)}-\tilde{a_{0}}(z)=0,$
(21)

where
$Q(z)=a_{0}(z)s(z)+a_{1}(z)[s^{{}^{\prime}}(z)+s(z)q^{{}^{\prime}}(z)]+...+a_{l}(z)[s^{l}(z)+...+s(z)(q^{{}^{\prime}}(z))^{l}]$
is a rational function. Now we discuss the following cases:

  1. (i)

If $q(z)-\alpha_{j}(z)\neq$ constant, and $nq(z)-\alpha_{j}(z)\neq$ constant,
for any $j=1,2,3.$ Then applying Lemma 5 into equation (21), we have
$P_{1}(z)\equiv 0\equiv P_{2}(z)=P_{3}(z)$, which is not possible.

  2. (ii)

If $q(z)-\alpha_{j}(z)=$ constant, for some $j=1,2,3$, say
$q(z)-\alpha_{1}(z)=$ constant, and $nq(z)-\alpha_{j}(z)\neq$ constant, for
any $j=1,2,3$. Let $q(z)=b_{r}z^{r}+b_{r-1}z^{r-1}+...+b_{0}$ and
$\alpha_{1}(z)=b_{r}z^{r}+b_{r-1}z^{r-1}+...+c_{0}$. Then
$q(z)-\alpha_{1}(z)=b_{0}-c_{0}$ and equation (21) becomes

$(P_{1}(z)e^{c_{0}}-Q(z)e^{b_{0}})e^{\tilde{\alpha_{1}}(z)}+P_{2}(z)e^{\alpha_{2}(z)}+P_{3}(z)e^{\alpha_{3}(z)}-s^{n}(z)e^{nq(z)}-\tilde{a_{0}}(z)=0,$

where $\tilde{\alpha_{1}}(z)=b_{r}z^{r}+b_{r-1}z^{r-1}+...+b_{1}z.$ Applying
Lemma 5 to the above equation, we have $P_{2}(z)\equiv 0\equiv P_{3}(z)$,
which is not possible.

  3. (iii)

If $q(z)-\alpha_{j}(z)\neq$ constant, for any $j=1,2,3$ and
$nq(z)-\alpha_{j}(z)=$ constant, for some $j=1,2,3$, say
$nq(z)-\alpha_{1}(z)=$ constant. Then applying Lemma 5 into equation (21), we
get the same contradiction as for just previous case.

  4. (iv)

If $q(z)-\alpha_{j}(z)=$ constant, for some $j=1,2,3$, say
$q(z)-\alpha_{1}(z)=$ constant and $nq(z)-\alpha_{k}(z)=$ constant, for some
$k$, say $nq(z)-\alpha_{2}(z)=$ constant. Then applying Lemma 5 into equation
(21), we have $P_{3}(z)\equiv 0$, which is not possible.

Hence we complete the proof. ∎

######  Proof of Theorem D.

Since $f$ is an entire function, then from equation (7) and Lemma 1, we have

$\displaystyle
T(r,P_{1}e^{\alpha_{1}z}+P_{2}e^{\alpha_{2}z}+P_{3}e^{\alpha_{3}z})$
$\displaystyle=T(r,f^{n}+Q_{2}^{*}(z,f))$ $\displaystyle\leq
nT(r,f)+T\left(r,\frac{Q_{2}^{*}(z,f)}{f^{2}}f^{2}\right)$ $\displaystyle\leq
nT(r,f)+T\left(r,\frac{Q_{2}^{*}(z,f)}{f^{2}}\right)+2T(r,f)$
$\displaystyle\leq(n+3)T(r,f)+S(r,f)$

As we know that
$\rho(P_{1}e^{\alpha_{1}z}+P_{2}e^{\alpha_{2}z}+P_{3}e^{\alpha_{3}z})=1$ by
using Lemma 4, thus we get $\rho(f)\geq 1$.
First suppose $\rho(f)>1$ and given that $f$ is an entire satisfying equation
(7) with $\lambda(f)<\rho(f)$. Then using Weierstrass factorization theorem,
$f$ must be a transcendental entire function in the form of

$f(z)=H(z)e^{\beta(z)},$

where $\beta(z)$ is a polynomial of degree $r>1$ and $H(z)$ is an entire
function such that $\rho(H)<r$. Now from equation (7), we have

$\displaystyle
H^{n}(z)e^{n\beta(z)}+\sum_{i=1}^{i=2}b_{i}(z)e^{i\beta(z)}+a_{0}(z)=P_{1}e^{\alpha_{1}z}+P_{2}e^{\alpha_{2}z}+P_{3}e^{\alpha_{3}z},$

where $b_{1}(z),b_{2}(z)$ are meromorphic functions of order $<r$ and
$a_{0}(z)$ is a rational function. This implies

$H^{n}(z)e^{n\beta(z)}+b_{2}(z)e^{2\beta(z)}+b_{1}(z)e^{\beta(z)}+(a_{0}(z)-P_{1}e^{\alpha_{1}z}-P_{2}e^{\alpha_{2}z}-P_{3}e^{\alpha_{3}z})=0$

Applying Lemma 5 into the above equation, we have

$a_{0}(z)-P_{1}e^{\alpha_{1}z}-P_{2}e^{\alpha_{2}z}-P_{3}e^{\alpha_{3}z}\equiv
0,$ (22)

which is not possible. Otherwise, again applying Lemma 5 into equation (22),
we get $P_{1}\equiv 0\equiv P_{2}=P_{3}$, which is contradiction. Thus
$\rho(f)=1$.
Now considering

$f(z)=H(z)e^{az+b},$ (23)

where $0\neq a,b\in\mathbb{C}$ and $H(z)$ is an entire function of order $<1$.
From equation (7), we have

$H^{n}(z)e^{n(az+b)}+b_{2}(z)e^{2(az+b)}+b_{1}(z)e^{az+b}+a_{0}(z)-P_{1}e^{\alpha_{1}z}-P_{2}e^{\alpha_{2}z}-P_{3}e^{\alpha_{3}z}=0$

Applying Lemma 5 into the above equation, there must exist positive integers
$k_{1},k_{2}$ and $k_{3}$ with $\\{k_{1},k_{2},k_{3}\\}=\\{1,2,3\\}$ such that
$na=\alpha_{k_{1}},2a=\alpha_{k_{2}},a=\alpha_{k_{3}}$ which gives
$\alpha_{k_{1}}:\alpha_{k_{2}}:\alpha_{k_{3}}=n:2:1$. We also have
$H^{n}(z)e^{nb}=P_{k_{1}}$, this gives $H(z)\equiv P_{k_{1}}^{1/n}e^{-b}$ is a
constant, $b_{2}(z)e^{2b}\equiv P_{k_{2}}$, $b_{1}(z)e^{b}\equiv P_{k_{3}}$,
and $a_{0}(z)\equiv 0$. Substituting the value of $H(z)$ into equation (23),
we have $f(z)=P_{k_{1}}^{1/n}e^{az}=Ce^{az}$, where $C$ is a non-zero
constant. ∎

## References

  * [1] Chen, J. F., Lian, G., Expressions of meromorphic solutions of a certain type of nonlinear complex differential equations, Bulletin of the Korean Mathematical Society, 57(4) (2020), 1061-1073.
  * [2] Laine, I., Nevanlinna Theory and Complex Differential Equations, W. de Gruyter, Berlin (1993).
  * [3] Hayman, W.K.,Meromorphic Functions, Clarendon Press, Oxford (1964)
  * [4] Halburd, R.G. and Korhonen, R.J., Difference analogue of the lemma on the logarithmic derivative with applications to difference equations, J. Math. Anal. Appl., 314 pp.477–487 (2006).
  * [5] Halburd, R.G. and Korhonen, R.J., Nevanlinna theory for the difference operator, Ann. Acad. Sci. Fenn. Math., 31 (2006), 463–478.
  * [6] Halburd, R., Korhonen, R. and Tohge, K., Holomorphic curves with shift-invariant hyperplane preimages, Transactions of the American Mathematical Society, 366, no. 8 (2014), 4267-4298.
  * [7] Xue, B., Entire solutions of certain type of non-linear differential equations. Mathematica Slovaca, 70(1) (2020), 87-94.
  * [8] L. Mirsky, An Introduction to Linear Algebra, Oxford, at the Clarendon Press, 1955.
  * [9] Li, P., Entire solutions of certain type of differential equations, Journal of mathematical analysis and applications, 344(1), pp.253-259 (2008).
  * [10] Li P., Entire solutions of certain type of differential equations II, Journal of mathematical analysis and applications, 375(1), pp.310-319 (2011).
  * [11] Li, P. and Yang, C.C., On the nonexistence of entire solutions of a certain type of nonliear differential equations, J. Math. Anal. Appl., 320, 827-835 (2006).
  * [12] C.C.Yang, H.X.Yi: Uniqueness Theory of Meromorphic Functions, Science Press, Kluwer Academic, Dordrecht, Beijing, (2003).
  * [13] Chen, W., Hu, P. and Wang, Q.: Entire Solutions of Two Certain Types of Non-linear Differential-Difference Equations. Computational Methods and Function Theory, 21(2), pp.199-218 (2021).
  * [14] Chiang, Y.M. and Feng, S.J.: On the Nevanlinna characteristic of f(z+$\eta$) and difference equations in the complex plane, The Ramanujan Journal, 16(1), pp.105-129 (2008).
  * [15] Rong J, Xu J., Three results on the nonlinear differential equations and differential-difference equations, Mathematics, 539 (2019).
  * [16] Wang Q, Wang Q.: Study on the existence of solutions to two specific types of differential-difference equations, Turkish Journal of Mathematics, 43(2), pp.941-54 (2019).
  * [17] Wen, Z.T., Heittokangas, J, Lain, I, Exponential polynomials as solutions of certain nonlinear difference equations. Acta Mathematica Sinica, English Series, 28(7),pp.1295-306 (2012).
  * [18] Yang Lo, Value Distribution Theory, Translated and revised from the 1982 Chinese Original, Springer-Verlag, Berlin (1993).