Freshness or Accuracy, Why Not Both? Addressing Delayed Feedback via
  Dynamic Graph Neural Networks
The delayed feedback problem is one of the most pressing challenges in
predicting the conversion rate since users' conversions are always delayed in
online commercial systems. Although new data are beneficial for continuous
training, without complete feedback information, i.e., conversion labels,
training algorithms may suffer from overwhelming fake negatives. Existing
methods tend to use multitask learning or design data pipelines to solve the
delayed feedback problem. However, these methods have a trade-off between data
freshness and label accuracy. In this paper, we propose Delayed Feedback
Modeling by Dynamic Graph Neural Network (DGDFEM). It includes three stages,
i.e., preparing a data pipeline, building a dynamic graph, and training a CVR
prediction model. In the model training, we propose a novel graph convolutional
method named HLGCN, which leverages both high-pass and low-pass filters to deal
with conversion and non-conversion relationships. The proposed method achieves
both data freshness and label accuracy. We conduct extensive experiments on
three industry datasets, which validate the consistent superiority of our
method.
Xiaolin Zheng, Zhongyu Wang, Chaochao Chen, Feng Zhu, Jiashu Qian
# Freshness or Accuracy, Why Not Both? Addressing Delayed Feedback via Dynamic Graph Neural Networks

###### Abstract

The _delayed feedback_ problem is one of the most pressing challenges in predicting the conversion rate since users' conversions are always delayed in online commercial systems. Although new data are beneficial for continuous training, without complete feedback information, i.e., conversion labels, training algorithms may suffer from overwhelming _fake negatives_. Existing methods tend to use multitask learning or design data pipelines to solve the delayed feedback problem. However, these methods have a trade-off between data freshness and label accuracy. In this paper, we propose Delayed Feedback Modeling by Dynamic Graph Neural Network (DGDFEM). It includes three stages, i.e., preparing a data pipeline, building a dynamic graph, and training a CVR prediction model. In the model training, we propose a novel graph convolutional method named HLGCN, which leverages both high-pass and low-pass filters to deal with conversion and non-conversion relationships. The proposed method achieves both data freshness and label accuracy. We conduct extensive experiments on three industry datasets, which validate the consistent superiority of our method.

 Recommender System, Conversion Rate Prediction, Delayed Feedback, Dynamic Graph, Importance Sampling

## I Introduction

Conversion rate (CVR)-related problems are fundamental in the setting of E-commerce. Most CVR-related methods focus on the CVR prediction [1, 2, 3], which aims to estimate the probabilities of specific user behaviors, e.g., buying recommended items. However, if conversions are not collected in time, i.e., the user feedback is delayed, it is difficult to predict CVR accurately.

Here, we take two examples to introduce the delayed feedback problem. Fig. 1(a) illustrates why this problem exists. Without loss of generality, suppose these clicks occur at time \(t_{1}\). For most clicks, there are no conversions, e.g., sample \(s_{1}\), \(s_{3}\), and \(s_{4}\). For the rest, their corresponding conversions are delayed for different times, e.g., sample \(s_{2}\) and \(s_{5}\). When setting an observation window at time \(t_{3}\), we can only observe the conversion of sample \(s_{5}\) at time \(t_{2}\). To distinguish whether these samples will convert, we should postpone the observation window until the _attribution window_ ends at time \(t_{5}\). An attribution window is a period of time in which we can claim that a conversion is led by a click. As a result, the delivery of training samples is significantly delayed. Although the delay feedback problem exists, most online commercial systems train their CVR prediction models in real-time data streams to capture data distribution changes. Fig. 1(b) illustrates the importance of delayed feedback modeling with a real-world dataset, i.e., Criteo Sponsored Search Traffic [4]. From it, we can observe half of the conversions in several hours but need to wait a couple of days or even longer to observe the rest. The label accuracy of training samples increases with more observed conversions while data freshness decreases. However, both data freshness and label accuracy are necessary for online commercial systems. Training with new data enhances real-time response capability for online systems. Training with accurate sample labels enhances model performance.

Towards addressing the delayed feedback problem, we can divide existing methods into two types, i.e., _mutitask learning_ and _designing data pipelines_. The former [5, 6, 7, 8, 9] uses additional models to assist in learning the CVR estimation. But it waits a long time to collect all conversions, leading to recommendation performance degradation. The latter [10, 11, 12, 13, 14, 15] designs new data pipelines for CVR prediction models. But it faces a trade-off between data freshness and label accuracy simultaneously. In summary, the above models cannot solve the delayed feedback problem.

To address the delayed feedback problem successfully, we propose **D**elayed **F**eedback **M**odeling by **D**ynamic **G**raph Neural Network (DGDFEM). The framework of DGDFEM includes three stages, i.e., preparing a data pipeline (Stage 1), building a dynamic graph (Stage 2), and training a CVR prediction model (Stage 3). **In Stage 1**, we design a novel data pipeline

Fig. 1: Introduction of the delayed feedback problem.
that achieves the best data freshness. To capture data freshness, we deliver each sample, i.e., a user-item click, as soon as it appears, even when it is _unlabeled_. These delivered unlabeled samples are used to build the dynamic graph. To achieve high label accuracy, we set a time window to wait for each sample's conversion. When the time window ends, we mark the sample as either positive or negative and deliver it again for the following two stages. Finally, after fake negatives are converted, we calibrate them as positive and deliver them one more time. **In Stage 2**, we build a dynamic graph to facilitate CVR prediction by capturing frequent changes in data distribution. We build the dynamic graph upon a sequence of chronologically delivered samples. In the graph, nodes are the users and items from samples, and edge attributes come from sample labels. **In Stage 3**, we take users and items in the delivered samples as seeds to sample multi-hop neighbors from the dynamic graph. To deal with conversion and non-conversion relationships, we propose a novel graph convolutional method, namely HLGCN, which aggregates features through _high-pass_ and _low-pass_ filters. High-pass filters capture node differences among non-conversion relationships. Low-pass filters retain node commonalities among conversion relationships. To alleviate the noises introduced by fake negatives, we further propose _distribution debias_ and prove its effectiveness theoretically. Our method achieves both data freshness and label accuracy.

We summarize the main contributions of this paper as follows: **(1)** We propose a novel delayed feedback modeling framework that highlights label accuracy with the best data freshness. **(2)** We propose the first dynamic graph neural network to solve the delayed feedback problem, which can capture data distribution changes and hence facilitates CVR predictions. **(3)** We conduct extensive experiments on three large-scale industry datasets. Consistent superiority validates the success of our proposed framework.

## II Related Work

### _Delayed Feedback Modelling_

In solving the delayed feedback problem, we can divide existing methods into two types, i.e., multitask learning and designing data pipelines.

**Multitask Learning.** This method uses additional models to assist in learning the CVR estimation. DFM [5] designs two models for estimating CVR and delay time. One successor of DFM, i.e., NoDeF [6], proposes fitting the delay time distribution by neural networks. Another successor [7] extracts pre-trained embeddings from impressions and clicks to predict CVR and leverages post-click information to estimate delay time. DLA-DF [8] trains a CVR predictor and propensity score estimator based on a dual-learning algorithm. MTDFM [9] trains the actual CVR network by simultaneously optimizing the observed conversion rate and non-delayed positive rate. These methods wait a long time to collect all conversions, leading to recommendation performance degradation. In contrast, our method delivers samples as soon as they appear, achieving the best data freshness.

**Designing Data Pipelines.** This method designs new data pipelines to change the way data is delivered. FNW and FNC [10] deliver all samples instantly as negative and duplicate them as positive when their conversions occur. But these two methods introduce numerous fake negatives into model training. FSIW [11] waits a long time for conversions. But it uses stale training data and does not allow the correction, even when the conversion occurs afterward. ES-DFM [12] sets a short time window to wait for conversions and rectifies these fake negatives. Based on ES-DFM, DEFER [13] and DEFUSE [14] further duplicate and ingest real negatives for training. But ES-DFM, DEFER, and DEFUSE cannot simultaneously achieve label accuracy and data freshness. As the time window extends, these three methods observe more conversion feedback, i.e., label accuracy becomes higher, but data freshness gets lower. FTP [15] constructs an aggregation policy on top of multi-task predictions, where each task captures the feedback pattern during different periods. But some tasks in FTP are trained with accurate but stale data, while others are trained with new but inaccurate data. These methods face a trade-off between data freshness and label accuracy. In contrast, our method achieves both data freshness and label accuracy.

### _Dynamic Graph Neural Networks_

In dynamic graph neural networks, we can divide existing methods into two types, i.e., discrete-time dynamic graphs and continuous-time dynamic graphs. _Discrete-time dynamic graph_ represents a dynamic graph as a series of static graph snapshots at different time intervals. DySAT [16] computes node representations through joint self-attention within and across the snapshots. DGNF [17] captures the missing dynamic propagation information in graph snapshots by dynamic propagation. TCDGE [18] co-trains a linear regressor to embed edge timespans and infers a common latent space for all snapshot graphs. _Continuous-time dynamic graph_ represents a dynamic graph as event sequences, e.g., creating nodes and adding edges. DyGNN [19] updates node states involved in an interaction and propagates the updates to neighbors. GNN-DSR [20] models the short-term dynamic and long-term static representations of users and items and aggregate neighbor influences. HDGCN [21] explores dynamic features based on a content-propagation heterogeneous graph. TGN [22] combines different graph-based operators with memory modules to aggregate temporal-topological neighborhood features and learns time-featured interactions. All these methods leverage stale or even wrong interactions in delayed feedback scenarios since the samples used to build the graph are delayed and mislabeled. In contrast, our method provides the newest data with accurate labels for the dynamic graph and proposes HLGCN to deal with conversion and non-conversion relationships in the graph.

## III Method

In this section, we present the framework of DGDFEM followed by its main stages in detail. DGDFEM has three stages,
i.e., preparing a data pipeline, building a dynamic graph, and training a CVR prediction model. In the first stage, we prepare a data pipeline that achieves high label accuracy with the best data freshness. In the second stage, we build a dynamic graph, which captures data distribution changes. In the third stage, we propose a graph convolutional method named HLGCN to deal with conversion and non-conversion relationships. Besides, we propose _distribution debias_ to alleviate the noises from fake negatives. DGDFEM achieves both data freshness and label accuracy, which solves the delayed feedback problem.

### _Preparing Data Pipeline_

**Preliminary.** There are three possible types of samples in the data pipeline of DGDFEM, namely _real negative_, _fake negative_, and _positive_. For ease of description, we let \(t_{d}\) denote the delay time of conversion feedback, \(l_{w}\) denote the time window, and \(l_{a}\) denote the attribution window length. We define three possible sample types as follows. **(1)**_Fake negative_ (\(l_{w}\leq t_{d}\leq l_{a}\)) is the sample whose conversion occurs beyond the time window. It is incorrectly delivered as negative at first. **(2)**_Positive_ (\(t_{d}\) < \(l_{w}\)) is the sample whose conversion occurs within the time window. **(3)**_Real negative_ (\(t_{d}\) > \(l_{a}\)) is the sample that does not convert eventually. We set \(t_{d}=\infty\) for these samples.

From the above definition, we observe that time window length \(l_{w}\) determines the proportions of fake negatives and positives. When we set \(l_{w}\to 0\), we transform all positives into fake negatives. When we set \(l_{w}\to l_{a}\), we eliminate all fake negatives.

**Our Proposed Data Pipeline.** We describe our proposed data pipeline through an example illustrated in Fig. 3. Suppose there are three samples, denoted as \(s_{1}\), \(s_{2}\), and \(s_{3}\), representing 'fake negative,' 'positive,' and'real negative,' respectively. We first deliver each sample, i.e., a user-item click, instantly into the data pipeline as soon as it appears in the commercial system, even when it is unlabeled. We set a time window to wait for each sample's conversion and deal with different sample types in various ways. **(1)** For fake negative \(s_{1}\), which arrives at time \(t_{1}\), we set a time window between time \(t_{1}\) and \(t_{1}+l_{w}\). When the window ends at time \(t_{1}+l_{w}\), we mark it as negative and deliver it for a second time as we do not observe the conversion feedback within the window. Finally, we calibrate it as positive and deliver it one more time when its conversion occurs at time \(t_{5}\). **(2)** For positive \(s_{2}\), which arrives at time \(t_{2}\), we set a time window between time \(t_{2}\) and \(t_{2}+l_{w}\). When the window ends at time \(t_{2}+l_{w}\), we mark it as positive and deliver it again as its conversion occurs at time \(t_{3}\), i.e., within the time window. **(3)** For real negative \(s_{3}\), which arrives at time \(t_{4}\), when the time window ends at time \(t_{4}+l_{w}\), we mark it as negative and deliver it.

We utilize these three sample types differently. We deliver unlabeled samples to build the dynamic graph and negative samples. to train the model. We deliver positive samples to build the dynamic graph and train the model.

**Comparison with Existing Data Pipelines.** We further analyze the differences in the data pipelines between our method and the state-of-the-art methods, i.e., FNW [10] and ES-DFM [12] to demonstrate our motivations. **(1)** FNW marks each sample as negative when it appears and delivers it instantly for training the model, e.g., sample \(s_{1}\) at time \(t_{1}\). This introduces numerous fake negatives into model training, which lowers label accuracy. In contrast, DGDFEM delivers each sample for model training after the time window ends. There is sufficient time for most samples to convert, i.e., DGDFEM achieves high label accuracy. **(2)** ES-DFM marks all samples as either positive or negative and delivers each sample after its time window, e.g., sample \(s_{1}\) at time \(t_{1}+l_{w}\). ES-DFM postpones the delivery for each sample until the end of the time window, which lowers data freshness. In contrast, DGDFEM first delivers samples instantly to build a dynamic graph, achieving the best data freshness. To this end, DGDFEM enjoys high label accuracy with the best data freshness.

### _Building the Dynamic Graph_

**Motivation of the Dynamic Graph.** Conversions correlate with data distribution changes. For example, when an item

Fig. 3: Data pipelines of FNW, ES-DFM, and DGDFEM.

Fig. 2: The proposed DGDFEM framework. The solid arrowed lines present the framework workflow. The dashed arrowed lines separate the changes in the graph at different times.
is on sale, users are more inclined to buy it. As a result, conversions associated with this item are more likely to occur.

Leveraging a dynamic graph can better capture the data distribution changes, facilitating CVR prediction. We can represent users and items in delivered samples as a graph. In the graph, nodes are users and items, and edge attributes come from sample labels. Our dynamic graph explicitly models data distribution changes as variations of node attributes and edges.

**Dynamic Graph Definition.** We build the dynamic graph upon a sequence of chronologically delivered samples, i.e., \(\mathcal{G}=\{s(t_{1}),s(t_{2}),...\}\) at times \(0\leq t_{1}\leq t_{2}\leq\ldots\). Besides, we set \(s=(u,i,y)\) where \(u\) and \(i\) are the user and item in sample \(s\), and \(y\in\{-1,0,1\}\) denotes the sample label, with \(-1\) denoting the unlabeled one, \(0\) denoting the negative one, and \(1\) denoting the positive one.

**Building the Dynamic Graph.** Delivered samples are involved in _node-wise events_ and _edge-wise events_ to build the dynamic graph chronologically [22]. The former creates nodes or updates node features, and the latter adds or deletes edges.

We represent a node-wise event by \(v_{j}(t)\), where \(j\) is the node index and \(v\) is the vector attribute associated with the sample. This event _only_ occurs for unlabeled samples as they carry the latest data information. In contrast, positive and negative samples are duplicates of previous unlabeled samples. If node \(j\) is not in \(\mathcal{G}\), the event creates it along with its attributes. Otherwise, the event updates its feature. Take Fig. 2 as an example. unlabeled sample \(s_{2}\) is involved in two node-wise events at time \(t_{2}\), i.e., adding a new node \(u_{2}\) to the graph and updating the feature of node \(i_{2}\).

We represent an edge-wise event as \(e_{uij}(t)\), where \(y\) is the sample label. It occurs for unlabeled and positive samples, i.e., \(s\) with \(y\in\{-1,1\}\) as negative samples (\(s\) with \(y=0\)) may be fake negative and represent wrong interactions. We keep at most \(m\)_recent_ edges for each node and delete the rest to improve efficiency. In Fig. 2, unlabeled sample \(s_{1}\) is first involved in an edge-wise event to add an unlabeled edge between user \(u_{1}\) and item \(i_{1}\) at time \(t_{1}\). It is then involved in another edge-wise event to connect user \(u_{1}\) and item \(i_{1}\) with a positive edge when it becomes positive at time \(t_{6}\).

After describing node-wise and edge-wise events, we define the dynamic graph at time \(t\) as follows. In time interval \(T\), we denote temporal sets of nodes and edges as \(\mathcal{V}_{T}=\{j:\exists v_{j}(t)\in\mathcal{G},t\in T\}\) and \(\mathcal{E}_{T}=\{(u,i,y):\exists e_{uij}(t)\in\mathcal{G},t\in T\}\). We denote a snapshot of dynamic graph \(\mathcal{G}\) at time \(t\) as \(\mathcal{G}_{t}=(\mathcal{V}_{[0,t]},\mathcal{E}_{[0,t]})\). We denote the neighbors of node \(j\) at time \(t\) as \(\mathcal{N}_{j}(t)=\{m:(j,m)\in\mathcal{E}_{[0,t]}\}\) and the \(k\)-hop neighborhood as \(\mathcal{N}_{j}^{k}(t)\).

### _Training CVR Prediction Model_

This section introduces training a CVR prediction model in DGDFEM. We use positive and negative samples for model training. For each positive sample, the model training process occurs before updating the dynamic graph to prevent the model from leveraging future information. We can further divide the whole training process into four steps.

**Neighbor Sampling.** Online commercial systems require real-time services for recommendation and advertising [10, 23]. To balance model performance and response time, we take the user and item as seeds to sample \(k\)-hop neighbors from dynamic graph \(\mathcal{G}_{t}\). We use Fig. 2 to exemplify neighbor sampling with \(k=2\). For sample \(s_{3}\) delivered at time \(t_{7}\), its corresponding user \(u_{3}\) and item \(i_{3}\) are used to sample \(2\)-hop neighbors from the dynamic graph. To simplify the description, we continue to use \(\mathcal{G}_{t}\) to represent the sampled graph.

**Node Embedding.** To describe nodes in the sampled graph, we embed them based on their attributes. Node features consist of sparse and dense features. The former is embedded through embedding matrices, whereas the latter is scaled into \([0,1]\) through min-max normalization. We use a multilayer perceptron (MLP) model to map them into the same dimension, as user and item features do not necessarily share the same dimension size. We define the embedding process as,

\[e_{j}^{(0)}=\mathrm{MLP}([x_{j,1},x_{j,2},...,x_{j,m},...]),\]

where \(x_{j,m}\) denotes the \(m\)-th processed feature of node \(j\in\mathcal{N}_{i}^{K}(t)\bigcup\mathcal{N}_{i}^{K}(t)\), \([\cdot]\) denotes the concatenation function, and \(e_{j}^{(0)}\) denotes the initial embedding of node \(j\).

**HLGCN.** After processing node embeddings, we propose a novel graph convolutional method named HLGCN to gather features from neighbors. In our setting, a successful graph convolutional method should deal with non-conversion and conversion relationships in the dynamic graph. Traditional graph convolutional methods [24, 25] can only handle conversion relationships as they use low-pass filters to retain node commonalities.

Our proposed HLGCN uses both _high-pass_ and _low-pass_ filters. The former captures node differences among non-conversion relationships, whereas the latter retains node commonalities among conversion relationships. It aggregates features from neighbors, as shown in (1). We introduce its implementation details later in Section III-D.

\[e_{u}^{(l)}=g(e_{u}^{(0)},e_{u}^{(l-1)},\{e_{i}^{(l-1)}\mid i\in\mathcal{N}_{ u}(t)\}), \tag{1}\]

where \(e_{u}^{(l)}\) is the aggregated embedding of node \(u\) from layer \(l\in\{1\ldots k\}\), and \(g(\cdot)\) denotes the graph convolution.

**Distribution Debias.** Although we calibrate fake negatives in our data pipeline, these samples shift actual data distribution as fake negatives are duplicated. The gap between actual and biased data distribution results in training noises. Here, we leverage the importance sampling [26] to alleviate the training noises. Compared with ESDFM [12], we predict sample probabilities of being positive and fake negative to assist in learning CVR. We define the prediction process as,

\[\hat{y}_{p} =\mathrm{Sigmoid}(\mathrm{MLP}_{p}([e_{u}^{(k)},e_{i}^{(k)}])),\] \[\hat{y}_{fn} =\mathrm{Sigmoid}(\mathrm{MLP}_{fn}([e_{u}^{(k)},e_{i}^{(k)}])),\] \[\hat{y}_{cvr} =\mathrm{Sigmoid}(\mathrm{MLP}_{cvr}([e_{u}^{(k)},e_{i}^{(k)}])).\]

When updating the CVR prediction model, we take \(\hat{y}_{fn}\) and \(\hat{y}_{p}\) as values, cut down their backpropagation, and update
these two prediction models separately. We optimize the CVR prediction model by minimizing the loss as,

\[\mathcal{L}=-\sum_{(u,i,y_{ui})\in\mathcal{S}}(y_{ui}(1+\hat{y}_{fn}) log(\hat{y}_{cvr}) \tag{2}\] \[+(1-y_{ui})(1+\hat{y}_{fn})(1-\hat{y}_{p}-\hat{y}_{fn})log(1-\hat{y }_{cvr})),\]

where \(\mathcal{S}\) is a set of positive and negative samples drawn from the data pipeline, and \(y_{ui}\) is the sample label, with \(y_{ui}=1\) denoting a positive sample and \(y_{ui}=0\) denoting a negative one.

**Proposition 1**.: _The distribution debias can correct distribution shifts._

Proposition 1 argues that our distribution debias alleviates training noises resulting from the fake negatives, which boosts CVR prediction accuracy. We give its detailed proof in Appendix -A.

### Hlgcn

We have provided our motivation for HLGCN in Section III-C that it should be able to deal with non-conversion and conversion relationships in the dynamic graph. Here, we introduce its technical details. HLGCN calculates user preferences to determine each edge's filter type and weight as conversions only depend on user preferences [7]. It then uses appropriate filters to exploit both types of relationships and aggregates neighbor features. Fig. 4 illustrates the aggregation process of HLGCN. It first uses a high-pass filter for the edge between user \(u_{1}\) and item \(i_{1}\) at time \(t_{1}\) and then replaces it with a low-pass filter at time \(t_{6}\). This is because our data pipeline delivers sample \(s_{1}\) as positive at time \(t_{6}\). The edge between user \(u_{1}\) and item \(i_{1}\) changes from unlabeled to positive. In other words, the preference of user \(u_{1}\) on item \(i_{1}\) is maximized at time \(t_{6}\).

**Filter Definition.** A high-pass filter \(F_{H}\) amplifies high-frequency signals, while a low-pass filter \(F_{L}\) magnifies low-frequency signals [27]. We state the definitions filter \(F_{H}\) and \(F_{L}\) as,

\[F_{H} =\epsilon I-D_{t}^{-1/2}A_{t}D_{t}^{-1/2},\] \[F_{L} =\epsilon I+D_{t}^{-1/2}A_{t}D_{t}^{-1/2},\]

where \(A_{t}\) denotes adjacency matrix of \(\mathcal{G}_{t}\), \(D_{t}\) denotes diagonal degree matrix, \(I\) denotes the identity matrix, and \(\epsilon\) is a hyper-parameter determining the retention proportion of a node's initial embedding.

We use \(F_{H}\) to capture node difference and \(F_{L}\) to retain node commonality. We define their aggregation process as,

\[\hat{e}_{u}^{(l)} =F_{H}\cdot H_{t}=\epsilon\cdot e_{u}^{(0)}-\sum_{i\in\mathcal{N} _{u}(t)}e_{i}^{(l-1)}/\sqrt{d_{u}(t)d_{i}(t)}, \tag{3}\] \[\hat{e}_{u}^{(l)} =F_{L}\cdot H_{t}=\epsilon\cdot e_{u}^{(0)}+\sum_{i\in\mathcal{N} _{u}(t)}e_{i}^{(l-1)}/\sqrt{d_{u}(t)d_{i}(t)},\]

where \(H_{t}\) denotes node embeddings and \(d_{u}(t)\) denotes the degree of node \(u\). Besides, \(\hat{e}_{u}^{(l)}\) and \(\hat{e}_{u}^{(l)}\) are aggregated embeddings of node \(u\) at the \(l\)-th layer through \(F_{H}\) and \(F_{L}\), respectively.

**Aggregating Features from Neighbors.** Since we sample \(k\)-hop neighbors from \(\mathcal{G}_{t}\), we use \(k\) layers of HLGCN to aggregate features from neighbors. By combining (1) and (3), we define its graph convolution \(g(\cdot)\) as,

\[e_{u}^{(l)} =g(e_{u}^{(0)},e_{u}^{(l-1)},\{e_{i}^{(l-1)}\mid i\in\mathcal{N} _{u}(t)\}) \tag{4}\] \[=\hat{w}\cdot\hat{e}_{u}^{(l)}+\hat{w}\cdot\hat{e}_{u}^{(l)}\] \[=\epsilon(\hat{w}_{ui}+\hat{w}_{ui})\cdot e_{i}^{(0)}\] \[+\sum_{i\in\mathcal{N}_{u}(t)}(\hat{w}_{ui}-\hat{w}_{ui})\cdot e _{i}^{(l-1)}/\sqrt{d_{u}(t)d_{i}(t)}\] \[=\epsilon\cdot e_{u}^{(0)}+\sum_{i\in\mathcal{N}_{u}(t)}p_{ui} \cdot e_{i}^{(l-1)}/\sqrt{d_{u}(t)d_{i}(t)},\]

where \(\hat{w}_{ui}\) and \(\hat{w}_{ui}\) denote the weight of the high-pass and low-pass filter for edge \(e_{uiy}(t)\). We set \(\hat{w}_{ui}+\hat{w}_{ui}=1\) as these two types of filters share the same processing on initial node embeddings. We let user preference for the item \(p_{ui}\) determine the types and weight of the used filters, i.e., \(\hat{w}_{ui}-\hat{w}_{ui}=p_{ui}\).

The above definition enables HLGCN to use appropriate filters to exploit conversion and non-conversion relationships.

**(1)** When \(p_{ui}<0\), it uses \(F_{H}\) to capture node difference. The weight of \(F_{H}\) increases as \(p_{ui}\) becomes lower. **(2)** When \(p_{ui}>0\), it uses \(F_{L}\) to retain node commonality. The weight of \(F_{L}\) increases as \(p_{ui}\) increases. **(3)** When \(p_{ui}\approx 0\), it only keeps initial node embeddings.

**Calculating User Preference.** Selecting a proper filter for each edge requires a priori knowledge of user preference. However, user preference varies over time, e.g., a user may gradually lose interest in an item, decreasing conversion probability. Therefore, HLGCN calculates user preferences each time to support filter selection.

Fig. 4: An illustration of the used filter for each edge in HLGCN. The shades of node colors in the dynamic graph reflect feature changes over time. The filter color reflects the change in filter type, and the filter thickness demonstrates the weight over time.
_For unlabeled edges_, ConvE [28], an effective knowledge representation learning model, fits our scenario. Since aggregated user and item embeddings in graph neural networks represent their general preferences and properties [29, 30], it can transform user embedding \(e_{u}^{(l)}\) into preference \(p_{ui}\) through item embedding \(e_{i}^{(l)}\). We define the preference calculation process as,

\[p_{ui} =\tanh(c_{ui}),\] \[c_{ui} =\mathrm{ConvE}(e_{u}^{(l)},e_{i}^{(l)})=\mathrm{Flatten}( \mathrm{CNN}(f(e_{u}^{(l)},e_{i}^{(l)})))W_{p},\]

where \(f(\cdot)\) is a 2D reshaping and concatenation function, which converts \(e_{u}^{(l)}\) and \(e_{i}^{(l)}\) into two matrices and concatenates them row by row. \(\mathrm{CNN}\) denotes a 2D convolutional layer, \(\mathrm{Flatten}(\cdot)\) reshapes its output into a vector, \(W_{p}\) denotes a projection matrix, and \(\tanh\) is the hyperbolic tangent function, which maps the result into \([-1,1]\).

_For positive edges_, we set \(p_{ui}=1\) as \(e_{uiy}(t)\) is formed by the conversion between user \(u\) and item \(i\), i.e., the preference is maximized.

## IV Experiments

In this section, we conduct empirical studies on three large-scale datasets to answer the following four research questions. **RQ1**: How does DGDFEM perform, compared with state-of-the-art methods for delayed feedback modeling (Section IV-B)? **RQ2**: Are the data pipeline and HLGCN beneficial in DGDFEM (Section IV-C)? **RQ3**: What are the impacts of different parameters on DGDFEM (Section IV-D)? **RQ4**: How does HLGCN deal with conversion and non-conversion relationships in our dynamic graph (Section IV-E)?

### _Experimental settings_

**Datasets.** We conduct experiments on three industrial datasets, i.e., _TENCENT_, _CRITEO2_, and _CIKM_. The statistics of these datasets are shown in Table I. TENCENT is an industry dataset for social advertising competition1. It includes 14 days of click and conversion logs. We exclude data from the last two days. CRITEO2 is a _newly-released_ industry dataset for the sponsored search of Criteo [4]. It represents the Criteo live traffic data in 90 days and publishes feature names additionally, _which enables graph construction_. CIKM is a competition dataset used in CIKM 2019 Ecomm AI2. It provides a sample of user behavior data in Taobao within 14 days. We keep the first click and conversion for each pair of the user and item to adapt to the delayed feedback problem.

Footnote 1: [https://algo.qq.com/?lang-en](https://algo.qq.com/?lang-en)

Footnote 2: [https://tianchi.aliyun.com/competition/entrance/231721/information?lang-en](https://tianchi.aliyun.com/competition/entrance/231721/information?lang-en)

We divide all datasets into two parts evenly according to the click time of each sample. We use the first part to pretrain CVR prediction models offline and divide the second part into multiple pieces by hours. We train models on the \(t\)-th hour data and test them on the \((t+1)\)-th data. We reconstruct training data in the second part according to different data pipelines and remain evaluation data unchanged. We report the _average_ performance across hours on the evaluation data. Note that, among all datasets, _CRITEO2_ has the highest proportion of delayed conversions.

**Comparison Methods.** We implement the following competitive methods as competitors for our proposed approach and categorize these methods into three classes.

**(1) Baselines**. We implement two baseline methods, namely **PRETRAIN** and **NODELAY**. PRETRAIN and NODELAY share the same model structure with DFM methods. We train PRETRAIN offline and use it for online evaluation directly. We train NODELAY with ground-truth labels and take its performance as the upper bound of all DFM methods as there is no delayed conversion.

**(2) Delayed Feedback Modeling (DFM)**. We implement four delayed feedback modeling methods, including **FNW**[10], **FSIW**[11], **ES-DFM**[12], **DEFER**[13], and **DEFUSE**[14]. These methods are widely deployed in online commercial systems. We use precisely the same data pipelines and loss functions as their original papers.

**(3) Graph Neural Network (GNN)**. We implement three graph neural networks, including **NGCF**[29], **FAGCN**[27], and **TGN-attn**[22]. NGCF refines user and item representations via information from multi-hop neighbors. FAGCN uses high-pass and low-pass filters to model assortative and disassortative networks. Since NGCF and FAGCN are _static_ graph neural networks, we build their graphs with offline training data and train their models with the online data stream. TGN-attn, a _dynamic_ graph neural network, has the best performance among all variants of TGN. We use positive samples to build the dynamic graph for TGN-attn, avoiding the damage of fake negatives.

**Evaluation Metrics.** We adopt two widely-used metrics for CVR evaluation, which describe model recommendation performance from different perspectives.

**(1) Area Under ROC Curve (AUC)**. AUC [31] is widely used in CVR predictions. It measures the ranking performance between conversions and non-conversions; the higher, the better.

**(2) Negative Log Likelihood (NLL)**. NLL [32] is sensitive to the absolute value of CVR prediction. It emphasizes the quality of the probabilistic model, and the lower, the better.

Following ESDFM [12] and DEFER [13], we compute relative improvement of our model over PRETRAIN as follows. We take these results to demonstrate the performance of DGDFEM straightforwardly; the higher, the better.

\[\%\mathrm{Improv.}=\frac{\mathrm{Metric}_{\texttt{DGDFEM}}-\mathrm{Metric }_{\texttt{PRETRAIN}}}{\mathrm{Metric}_{\texttt{NODELAY}}-\mathrm{Metric}_{ \texttt{preTRAIN}}}.\]

**Experimental Settings.** We introduce experimental settings as follows. We use Adam [33] as an optimizer with a learning rate of \(0.0001\). We set the batch size to \(1,024\) for all competitors. We repeat experiments ten times with different random seeds and report their average results. Following ES-DFM [12], we search time window length \(l_{w}\) from \(0.25\) hour. We set the maximum time window length for searching to \(24\) as it is the common attribution window length in many business scenarios [13]. We optimize all hyperparameters carefully through
grid search for all methods to give a fair comparison. The consumed resources vary with methods and hyperparameters.

### _Overall Performance Comparisons (RQ1)_

Table II shows the comparison results on all datasets. From it, we observe that: **(1)** In DFM methods, FNW and FSIW are inferior to ES-DFM as they either focus on data freshness or label accuracy. ES-DFM, DEFER, and DEFUSE achieve superior performance in DFM methods on all datasets as they balance accuracy and freshness. These results demonstrate that data freshness and label accuracy are essential for delayed feedback modeling. **(2)** GNN methods generally outperform DFM methods as GNN can better represent user preferences and item properties. Among GNN methods, NGCF surprisingly outperforms TGN-attn on _CRITEO2_. This result demonstrates that leveraging a dynamic graph does not always benefit delayed feedback modeling, as the samples used to build the dynamic graph are delayed and mislabeled. **(3)** DGDFEM is superior to both DFM and GNN methods. It even outperforms NODLAY on _CRITEO2_ and _CIKM_. These results demonstrate that our framework contributes to delayed feedback modeling as it obtains accurate conversion feedback and keeps the CVR prediction model fresh.

### _Ablation Study (RQ2)_

We verify the effectiveness of different components in DGDFEM via ablation study. We implement variants of our proposed method. **(1)**_Pretrain_ and _Oracle_ are the lower and upper bounds of DGDFEM. _Pretrain_ replaces the dynamic graph with a static graph built with offline training data. _Oracle_ builds the dynamic graph and trains models with ground-truth labels. **(2)** When building the dynamic graph, _w/o Unlabel_ removes unlabeled samples, while _w/o POS_ removes positive samples. **(3)**_GCN_[24] and _GAT_[25] are two substitutes for HLGCN. _w/o ConvE_ replaces ConvE by two linear layers with Leaky ReLu [34] in HLGCN. **(4)**_w/o DD_ replaces distribution debias with binary cross entropy loss.

From Table III, we conclude: **(1)** DGDFEM largely outperforms _Pretrain_ and performs closer to _Oracle_. _Pretrain_ cannot capture frequent data distribution changes. _Oracle_ degrades our data pipeline, HLGCN, and distribution debias, as they are specially designed for delayed feedback modeling, limiting the upper bound of our performance. **(2)** For data pipelines, depending on positive or unlabeled samples alone cannot achieve the best recommendation performance. The delivered positive samples are critical in the E-commerce scenario, i.e., _CIKM_, as positive behaviors strongly reflect user preference. In contrast, the delivered unlabeled samples are necessary for the scenario with highly delayed conversions, i.e., _CRITEO2_. **(3)** For HLGCN, high-pass and low-pass filters are indispensable to achieve ideal performance. The underperformance of _GAT_ and _GCN_ demonstrates that conventional graph neural networks are unsuitable for our dynamic graph, as they only use low-pass filters. The difference between _w/o ConvE_ and DGDFEM demonstrates that \(\mathrm{ConvE}\) can better calculate the user preference for the item. **(4)** The result of _w/o DD_ shows that our distribution debias benefits model performance, as it corrects the distribution shifts. **(5)** All components in DGDFEM are essential and our proposed data pipeline is the most crucial part for delayed feedback modeling.

### _Parameter Analyses (RQ3)_

We evaluate the impacts of different model parameters on DGDFEM and state-of-the-art methods with _CRITEO2_. _CRITEO2_ best reflects the delayed feedback problem as it has the highest proportion of delayed conversions. From Fig. 5, we conclude: **(1)** For time window length \(l_{w}\), all methods except DGDFEM has performance degradation as \(l_{w}\) increases. This
result demonstrates that DGDFEM successfully solves the trade-off between data freshness and label accuracy. In DGDFEM, we can either set high \(l_{w}\) to increase model performance or use low \(l_{w}\) to save resources. **(2)** For hop number \(k\), the performance of all methods except FAGCN slightly improves as \(k\) increases. This indicates that signals from deeper neighbors are less beneficial for delayed feedback modeling than other scenarios. **(3)** For neighbor number \(m\), the performance of DGDFEM, TGN-attn, and FAGCN has no significant relationship with \(m\), whereas the performance of NGCF slightly improves as \(m\) increases. This indicates that it is unnecessary to set large \(m\) for delayed feedback scenarios. **(4)** For retention proportion \(\epsilon\), DGDFEM and FAGCN perform much worse with \(\epsilon=0\), indicating that only depending on neighbor features is insufficient. DGDFEM prefers a higher retention proportion of initial node embeddings, whereas FAGCN prefers a lower one. Our proposed HLGCN is intrinsically different from FAGCN, although they both use high-pass and low-pass filters. FAGCN aims to capture node differences or retain node commonalities among conversion relationships. In contrast, HLGCN aims to deal with conversion and non-conversion relationships in the dynamic graph.

### _Case Study (RQ4)_

As motivated in Section III-D, HLGCN should use a low-pass filter for an edge with high conversion possibility, and vice versa. It calculates user preferences to _directly_ determine the filter weight and type (see (4)). Thus, the CVR of the system should correlate with the average user preferences when time elapses. To illustrate it clearly, we use a sliding window with different lengths to average CVR and filter weight in the window. We set the window length as 8 hours for both _TENCENT_ and _CIKM_, and 72 hours for _CRITEO2_. From Fig. 6, we can observe that the average user preferences correlate with the CVR. This phenomenon is particularly significant on _TENCENT_ and _CIKM_. These results demonstrate that HLGCN can use correct filters to deal with non-conversion and conversion relationships.

## V Conclusion

As the first work to exploit graph neural network in delayed feedback modeling, DGDFEM successfully solves the trade-off between label accuracy and data freshness. The framework of DGDFEM includes three stages, i.e., preparing the data pipeline, building the dynamic graph, and training the CVR prediction model. We further propose a novel graph convolutional method named HLGCN. The dynamic graph captures frequent data distribution changes and HLGCN leverages appropriate filters for the edges in the graph. We conduct extensive experiments on three large-scale industry datasets and the results validate the success of our framework.

## Acknowledgment

This work was supported in part by the National Key R&D Program of China (No. 2022YFF0902704) and the "Ten Thousand Talents Program" of Zhejiang Province for Leading Experts (No. 2021R52001).

Fig. 5: Performance with different Parameters.

Fig. 6: Filter and CVR variations with time elapse