# Entire Chain Uplift Modeling with Context-Enhanced Learning for Intelligent
Marketing

Yinqiu Huang  Chongqing UniversityChongqingChina<EMAIL_ADDRESS>,  Shuli
Wang  MeituanChengduChina<EMAIL_ADDRESS>,  Min Gao  Chongqing
UniversityChongqingChina<EMAIL_ADDRESS>,  Xue Wei  MeituanChengduChina
<EMAIL_ADDRESS>,  Changhao Li  MeituanChengduChina
<EMAIL_ADDRESS>,  Chuan Luo  MeituanChengduChina
<EMAIL_ADDRESS>,  Yinhua Zhu  MeituanChengduChina
<EMAIL_ADDRESS>,  Xiong Xiao  MeituanChengduChina
<EMAIL_ADDRESS>and  Yi Luo  MeituanChengduChina<EMAIL_ADDRESS>
(2018)

###### Abstract.

Uplift modeling, vital in online marketing, seeks to accurately measure the
impact of various strategies, such as coupons or discounts, on different users
by predicting the Individual Treatment Effect (ITE). In an e-commerce setting,
user behavior follows a defined sequential chain, including impression, click,
and conversion. Marketing strategies exert varied uplift effects at each stage
within this chain, impacting metrics like click-through and conversion rate.
Despite its utility, existing research has neglected to consider the inter-
task across all stages impacts within a specific treatment and has
insufficiently utilized the treatment information, potentially introducing
substantial bias into subsequent marketing decisions. We identify these two
issues as the chain-bias problem and the treatment-unadaptive problem. This
paper introduces the Entire Chain UPlift method with context-enhanced learning
(ECUP), devised to tackle these issues. ECUP consists of two primary
components: 1) the Entire Chain-Enhanced Network, which utilizes user behavior
patterns to estimate ITE throughout the entire chain space, models the various
impacts of treatments on each task, and integrates task prior information to
enhance context awareness across all stages, capturing the impact of treatment
on different tasks, and 2) the Treatment-Enhanced Network, which facilitates
fine-grained treatment modeling through bit-level feature interactions,
thereby enabling adaptive feature adjustment. Extensive experiments on public
and industrial datasets validate ECUP’s effectiveness. Moreover, ECUP has been
deployed on the Meituan food delivery platform, serving millions of daily
active users, with the related dataset released for future research.

Uplift modeling, Context-enhanced learning, Entire chain modeling

††copyright: acmcopyright††journalyear: 2018††doi:
XXXXXXX.XXXXXXX††conference: Make sure to enter the correct conference title
from your rights confirmation emai; June 03–05, 2018; Woodstock, NY††price:
15.00††isbn: 978-1-4503-XXXX-X/18/06††copyright: none††ccs: Applied computing
Economics††ccs: Information systems Personalization

##  1\. Introduction

Major e-commerce platforms routinely devise marketing strategies to enhance
user engagement and amplify Gross Merchandise Volume, commonly utilizing
incentives such as advertising, discounts, and coupons (Lin et al., 2017; Zhao
et al., 2019). The implementation of such strategies incurs not only
substantial costs but also demands numerous iterations and adjustments,
necessitating an experienced operations team and incurring considerable trial
and error expenses. Uplift modeling, an innovative predictive analysis
technology rooted in causal inference (Yao et al., 2021), has garnered notable
results in the realm of intelligent marketing (Liu et al., 2023). The uplift
model adeptly forecasts the Individual Treatment Effect (ITE) under various
treatments, such as issuing a specific coupon, thereby assisting e-commerce
platforms in crafting precise, effective marketing strategies. The process is
shown in Fig. 1, and most platforms adopt a two-stage mode. After obtaining
the uplift score, using the operations research optimization technology for
treatment allocation under cost constraints. The uplift model is the
cornerstone of the entire process.

Figure 1. Overview of major platform marketing process.

Uplift modeling aims to precisely discern the differential in a user’s
response with and without treatment, subsequently outputting an uplift score,
such as the increase in conversion rate resulting from a specific coupon
allocation. Unlike traditional supervised learning, uplift modeling presents a
counterfactual problem: Observations are limited to the outcomes when an
individual is either treated (treatment group) or not treated (control group)
and actual ground-truth labels (uplift score) are absent.

Existing uplift methodologies mainly establish response models on multiple
treatment functions concurrently to deduce counterfactual outcomes. The
S-learner (Künzel et al., 2019) estimates the conditional average outcome
under various treatments, calculating the ITE (i.e., uplift score) through
subtraction. Several variants have been developed on this foundation, such as
the T-learner and X-learner (Künzel et al., 2019), among others. Within this
domain, various deep learning methods, commended for their exceptional
feature-extraction capabilities, serve as fundamental models of uplift
frameworks.

Nonetheless, prevailing uplift methods manifest two salient limitations.
Firstly, existing methods display insufficient modeling of the complete
process chain. In e-commerce scenarios, users typically follow a sequential
pattern, designated as ${impression}$ $\rightarrow$ ${click}$ $\rightarrow$
${conversion}$, and crafting marketing strategies usually requires integrating
the uplift of multiple user behaviors in the chain. However, current research
predominantly predicts the uplift score for a single task within the chain,
neglecting the inter-task impact despite the model making inferences
throughout the entire chain space. This will result in biased results because
treatment has varying degrees of impact on each task. For instance, a user
group may exhibit a positive click uplift score and a negative conversion
uplift score under a specific treatment, yet the culminating click-and-
conversion uplift score may still be positive. Relying solely on the
prediction of the negative uplift score of the terminal task from a model
trained on the click set can yield entirely opposing results, potentially
wreaking havoc on subsequent marketing decisions. This phenomenon, referred to
as ”chain-bias” problem, is illustrated in Fig. 2. Concurrently, data sparsity
in subsequent tasks compromises the model’s generalization capabilities.

Secondly, they demonstrate insufficient utilization of the treatment.
Treatment notably influences user responses, with individuals displaying
varied behaviors under different treatments. For instance, we observed that
students are more inclined to place orders when offered high-value coupons on
the Meituan food delivery platform, a trend not as prevalent among white-
collar workers. However, prior methods typically learn a static representation
for each feature, neglecting the adaptation of the same feature under
disparate treatments. This lack of adaptability makes capturing individual
behavioral distinctions across various treatments challenging. While some
researchers leverage the attention mechanism to model the interplay between
treatment and non-treatment features (Xu et al., 2022; Liu et al., 2023), they
predominantly utilize normalized attention weights to adjust both feature
types. Unfortunately, this vector-level interaction approach lacks the
flexibility to seize each feature’s fluctuating significance under various
treatments. This phenomenon is referred to as a ”treatment-unadaptive”
problem.

Figure 2. Chain-bias: the discrepancy between the CVR uplift score and the
CTCVR uplift score, only considering CVR results will bring bias to subsequent
decisions.

In this paper, we introduce the Entire Chain UPlift method with context-
enhanced learning (ECUP), addressing the aforementioned challenges through a
thorough consideration of both chain and treatment information. ECUP is
primarily composed of two parts. The first component, the Entire Chain-
Enhanced Network, applies the user’s sequential pattern to proficiently learn
the response functions of each task across the entirety of the chain space. In
tandem, a Task-Enhanced Network merges treatment-enhanced features with task
information, thereby enhancing context-aware uplift modeling to solve the
chain-bias. This component not only comprehensively considers the conversion
funnel in uplift modeling, reducing the adverse effects of different uplifts
in previous and subsequent tasks, but also solves the data sparsity problem in
subsequent tasks. The second component, the Treatment-Enhanced Network,
discerns correlation data between the treatment and personalized features by a
Treatment Awareness Unit, subsequently directing the refinement of treatment-
aware features. It then strikes a balance between the initial and treatment-
aware embedding by a Treatment-Enhanced Gate through bit-level weights that
facilitate a flexibly adaptive adjustment of features under various
treatments.

In summary, our contributions are as follows:

  * •

Problem. We identify and propose a solution to the chain-bias problem in
uplift modeling. To the best of our knowledge, it is the first endeavor in
large-scale online systems. Furthermore, our method exhibits versatility and
promptly applies to various chain uplift modeling tasks.

  * •

Model. We introduce a novel entire chain context-enhanced uplift modeling
method, ECUP, adept at comprehensively considering the conversion funnel in
uplift modeling and adapting features under varying treatments, thereby
addressing the chain-bias and treatment-unadaptive problem.

  * •

Data. We release a large-scale, comprehensive, unbiased coupon usage dataset
from the Meituan food delivery platform111https://github.com/MTDJDSP/MT-LIFT.
To our knowledge, this represents the first public, unbiased industrial
dataset featuring multiple treatments and comprehensive chain label
information intended to facilitate research in causal inference.

  * •

Evaluation. Through extensive experiments on both offline and online A/B
tests, we show that our ECUP significantly outperforms the state-of-the-art
methods. ECUP has been deployed in Meituan and serves millions of daily active
users.

##  2\. PRELIMINARIES

###  2.1. Problem Definition

We define users’ sequential behavior patterns in marketing scenarios as
${impression\rightarrow click\rightarrow conversion}$. We assume the observed
dataset to be $\mathcal{S}=\left.\left\\{\left(x_{i},t_{i},y_{i}\rightarrow
z_{i}\right)\right\\}\right|_{i=1}^{N}$, with a sample drawn from a
distribution with domain $X\times T\times Y\times Z$, where $X$ is feature
space, $T$ is treatment space, $Y$ and $Z$ are label spaces, and $N$ is the
total number of impressions; $x_{i}$ is the feature vector of the i-th
instance, which includes user features, item features, and contextual
features; $t_{i}$ is the treatment ID received by the i-th instance, and the
total number of treatments is K, i.e., $t_{i}\in\\{1,2,\ldots,K\\}$; $y_{i}$
and $z_{i}$ are binary labels, with $y_{i}=1$ or $z_{i}=1$ representing click
or conversion, respectively; $y_{i}\rightarrow z_{i}$ represents the order
dependency in the chain, and when a subsequent task in the chain occurs, the
label of the previous task is 1. There are three associated probabilities:
post-view click-through rate (CTR), post-click conversion rate (CVR), and
post-view click&conversion rate (CTCVR):

(1) $\begin{gathered}pCTR=p(y=1\mid x),\\\ pCVR=p(z=1\mid y=1,x),\\\
pCTCVR=p(y=1,z=1\mid x)=pCTR\times pCVR.\end{gathered}$

We follow the Neyman-Rubin potential outcome framework (Rubin, 2005) to define
the ITE estimation problem. We set $y_{i}(0)$ and $y_{i}(k)$ to represent the
potential outcome of $pCTR$ for the i-th instance in the control and k-th
treatment group, respectively. Similarly, $z_{i}(0)$ and $z_{i}(k)$ represent
the potential outcome of $pCTCVR$ for the i-th instance in the control and
k-th treatment group, respectively. The challenge of estimating ITE is that we
can only observe one potential outcome, either $y_{i}(0)$ or $y_{i}(k)$,
$z_{i}(0)$ or $z_{i}(k)$, so there is no ground-truth label used for
supervised training. We estimate the ITE under the k-th treatment for each
instance, denoted as

(2) $\displaystyle\tau^{Y}_{k}(x)=\mathbb{E}_{\mathbb{P}}(Y(k)-Y(0)\mid
x)=\mathbb{E}_{\mathbb{P}}(Y\mid t=k,x)-\mathbb{E}_{\mathbb{P}}(Y\mid t=0,x),$
$\displaystyle\tau^{Z}_{k}(x)=\mathbb{E}_{\mathbb{P}}(Z(k)-Z(0)\mid
x)=\mathbb{E}_{\mathbb{P}}(Z\mid t=k,x)-\mathbb{E}_{\mathbb{P}}(Z\mid t=0,x),$

where $\tau^{Y}_{k}(x)$ and $\tau^{Z}_{k}(x)$ are the uplift scores of the CTR
task and CTCVR task respectively. In the inference stage, we estimate the
individual treatment effects of all samples for treatment assignment. For the
terminal task (conversion) of the current chain, we consider the impact of the
previous tasks by modeling the potential outcome of $pCTCVR$ instead of $pCVR$
to obtain unbiased ITE estimation $\tau^{Z}_{k}(x)$.

###  2.2. Initial Embedding Representation

Given an instance $(x_{i},t_{i},y_{i},z_{i})$, we encode not only its non-
treatment features but also its treatment feature. For the sparse non-
treatment features of sample $x_{i}$, we map the features through a parameter
matrix $E$, and for dense features, we transform them to the same dimension
through an affine transformation:

(3) ${e}_{ij}^{x}\in\mathbb{R}^{1\times
d}=\begin{cases}{W}_{j}*x_{ij}+{b}_{j},&x_{ij}\text{ is a continuous
feature,}\\\ \operatorname{lookup}\left({E}_{j},e_{ij}\right),&x_{ij}\text{ is
a sparse feature,}\end{cases}$

where W and b are learnable parameters. Then, we concatenate them to get the
initial embedding representation of $x_{i}$:

(4) ${E}_{i}^{x}\in\mathbb{R}^{f\times
d}=\left[{e}^{x}_{i1};{e}^{x}_{i2};\ldots;{e}^{x}_{if}\right],$

where $d$ is the dimension size of each field, and $f$ is the number of
feature fields. For the treatment feature, we treat it as a sparse feature and
obtain the treatment embedding ${E}_{i}^{tr}\in\mathbb{R}^{1\times d}$ by
feature mapping through the parameter matrix $E$. In addition, to achieve
task-enhanced learning, we obtain each task’s features
${E}_{i}^{ta}\in\mathbb{R}^{1\times d}$ in the chain through parameter matrix
mapping, where $ta\in\\{0,1\\}$ represents the click and conversion task.

##  3\. How is chain-bias manifested

To study the uplift relationship at each stage in the chain, we analyze the
public dataset CRITEO-UPLIFT and the dataset MT-LIFT collected from Meituan,
detailed in Section 5.1.1. We randomly segment the data and draw the uplift
graph with several selected segments, as shown in Fig. 3. We define
$r_{k}^{t}$ and $r_{k}^{c}$ as the number of samples with $Z=1$ (conversion)
in the k segment for the treatment and control groups, and $n_{k}^{t}$
represents the number of samples for the treatment group, while $n_{k}^{c}$
represents the number of samples for the control group. Next, we calculate the
actual uplift $\frac{r_{k}^{t}}{n_{k}^{t}}-\frac{r_{k}^{c}}{n_{k}^{c}}$ of
each segment for drawing, where we calculate the actual uplift score of CTCVR
(impression set) and CVR (click set).

From the result, the changing trends of CTCVR and CVR between different
segments are not entirely consistent, and we cannot get the final actual
uplift for the chain just by CVR uplift on the click set. The main reason for
this phenomenon is that users have different attention points at each stage of
sequential behavior, resulting in varying degrees of impact of treatment on
behavior at different stages of the chain. If we only use the uplift model
trained on the click set, it will have a devastating bias on later marketing
decisions. Therefore, we propose the ECUP to model the uplift score in the
entire chain space and inject task prior information to capture the impact of
treatment on different tasks for solving the chain-bias.

Figure 3. Actual uplift of CVR and CTCVR for randomly segmented segments. The
trend of changes in the uplift score of CVR and CTCVR is not entirely
consistent.

##  4\. Method

In this section, we will elaborate on our proposed ECUP. As shown in Fig. 4,
the overall architecture of ECUP consists of two main components:

  * •

Entire Chain-Enhanced Network (ECENet), which uses the user’s sequential
pattern to estimate the results of each task in the entire chain space and
uses a Task-Enhanced Network (TAENet) to inject task prior information to
achieve context-enhanced representation, capturing the different uplifts on
each task to avoid the negative impact of chain-bias.

  * •

Treatment-Enhanced Network (TENet), which aims to guide the treatment-aware
refinement for initial features and implements bit-level adaptive adjustment
of embedding representations under different treatments to solve the
treatment-unadaptive problem.

Figure 4. The architecture of ECUP. It models the outcome in the entire chain
space, implements treatment-enhanced embedding representation through the
TENet, and then injects task prior information using the TAEGate to achieve
contextual awareness of treatment and task information, solving the chain-bias
and treatment-unadaptive problems.

###  4.1. Entire Chain-Enhanced Network (ECENet)

ECENet models the uplift score of each task in the entire chain space, which
injects task information based on treatment-enhanced embedding (detailed in
Section 4.2) to enhance both the treatment and task information, solve the
chain-bias and alleviate the data sparsity of the subsequent task, whose
structure is shown in Fig. 4.

Individuals respond differently to different tasks in a specific treatment. We
aim to capture task-aware feature representation, while traditional methods
learn multi-task targets through multiple towers. Each tower is a stacked deep
neural network (DNN), and its expressive ability is so limited that it cannot
capture the different effects of treatment on different tasks, and giving
specific prior knowledge to each layer of DNN can effectively inject specific
information (Chang et al., 2023). Therefore, we propose the Task-Enhanced
network (TAENet) to deeply fuse task prior with treatment-enhanced
representations (the output of TENET) to achieve context-enhanced parameter
learning. It performs personalized scaling of DNN’s parameters to balance the
sparsity of features in different contexts.

####  4.1.1. Task-Enhanced Network (TAENet)

To capture the responses of each instance in different contexts, we adaptively
scale DNN’s parameters with priori information via a gating mechanism. We use
task information as priori input and then model them with treatment-enhanced
embedding through a multi-head attention network. To ensure the stability of
the initial and treatment embedding, we only update the task embedding
$E^{ta}$. Specifically, we treat task information $E^{ta}$ as Query and
treatment-enhanced embedding $E^{tr}$ as Key and Value to capture context-
enhanced prior information:

(5) $\begin{gathered}head_{i}=Attention(E^{{ta}}W^{Q}_{i},\\{\oslash
E^{r}\\}W^{K}_{i},\\{\oslash E^{r}\\}W^{V}_{i}),\\\ \begin{aligned}
E^{pri}\in\mathbb{R}^{1\times d}&=MultiHeadAttention(Q,K,V)\\\
&=Concat(head_{1},...,head_{h})W^{O},\end{aligned}\end{gathered}$

where $\oslash$ represents no gradient propagation, $h$ is the number of
attention heads. Next, we inject prior information into the model through a
Task-Enhanced Gate (TAEGate), which is a multi-layer perception (MLP):

(6) $\delta_{ta}\in[0,\gamma]=\gamma*MLP(concat(E^{pri})),$

the middle layer of MLP uses the Relu activation function, and the last layer
uses the Sigmoid activation function. $\gamma$ is a regulatory factor that
regulates the scaling size. Next, we use the output of TAEGate to scale with
each layer of each DNN tower except the last layer. For one layer in DNN, we
perform:

(7) ${E}^{te}=\delta_{ta}\otimes{H},$

where $H\in\mathbb{R}^{h\times T}$ means that the layer has $h$ hidden units,
and there are $T$ tasks in the entire chain, and $T=2$ in this paper. Note
that $\delta_{ta}\in\mathbb{R}^{h\times T}$ will be divided into $T$ vectors
of dimension $h$ and execute the Equation 7 with the layer of each task, and
then use the scaled one as the input of the next layer of DNN to achieve task-
enhanced representation learning. The output of the last layer of each task
uses the sigmoid activation function and is not affected by TAEGate. TAEGate
captures the impact of treatment on different tasks by injecting task prior
information, combined with treatment-enhanced embedding to achieve context-
enhanced learning for addressing chain-bias.

####  4.1.2. Training & Inference

Unlike ESMM (Ma et al., 2018a), which only shares the underlying parameters,
we share all parameters except the last layer of each task. We use the $pCTR$
and $pCTCVR$ in model training to calculate the loss across the entire sample
space:

(8) $\displaystyle
L=\sum_{i=1}^{N}l(y_{i},pCTR_{i})+\sum_{i=1}^{N}l(y_{i}\&z_{i},pCTR_{i}\times
pCVR_{i})+\lambda||\theta||,$

where $l$ is the cross-entropy loss function, $\lambda$ and $||\theta||$ are
the tradeoff parameter and the regularization terms, and $y_{i}\&z_{i}$
indicates the sequential pattern from click to conversion, using the
sequential dependencies to capture relationships between tasks in the chain.
The parameter-sharing mechanism dramatically alleviates data sparsity in
subsequent tasks, and TAEGate ensures that the model can still capture the
differential impact of treatment on each task during parameter sharing.

At the end of model training, ECUP is regarded as an S-learner, and we obtain
ITE through Equation LABEL:eq1.

###  4.2. Treatment-Enhanced Network (TENet)

The structure of TENet is shown in Fig. 5, which consists of two parts: the
Treatment-Aware Unit (TAU) and the Treatment Enhancement Gate (TEGate).
Inspired by (Wang et al., 2022), we propose bit-level weights to achieve fine-
grained treatment-enhanced feature refinement. We first generate treatment-
aware features and bit-level weights through two independent TAUs. Next, we
use the obtained weights to balance the initial and treatment-aware features
through the TEGate adaptively.

####  4.2.1. Treatment-Aware Unit (TAU)

TAU consists of two parts: 1) Self-attention network, which is used to capture
the cross relationship between user, item, and contextual features; 2)
treatment information extractor, which is used to encode the treatment. TAU
interacts with the cross-features extracted from self-attention with treatment
information to obtain treatment-aware features and bit-level weight.

Figure 5. The architecture of Treatment-Enhanced Network. It obtains
treatment-awareness embedding and bit-level feature weights through two
independent Treatment-Aware Units (TAUs) and then uses the TEGate to balance
treatment-awareness embedding and initial embedding for achieving treatment-
enhanced representation.

Self-attention network. We capture the cross-feature relationships between all
feature pairs to obtain a refined representation of individuals in their
natural state. Specifically, we map the initial features
$E^{x}\in\mathbb{R}^{f\times d}$ into three different matrices:

(9) ${Q},{K},{V}={E^{x}W}^{{Q}},{E^{x}W}^{{K}},{E^{x}W}^{{V}},$

where ${W}^{{Q}},{W}^{{K}},{W}^{{V}}\in\mathbb{R}^{d\times d_{k}}$ is the
learnable parameter matrix, and $d_{k}$ is the attention size. Next, we get
the cross-feature through the attention matrices:

(10) $E^{V}\in\mathbb{R}^{f\times
d_{k}}=Attention(Q,K,V)=Softmax(\frac{QK^{T}}{\sqrt{d_{k}}})V.$

Finally, we restore its dimensions through a linear transformation layer:

(11) ${E}^{{att}}\in\mathbb{R}^{f\times d}=E^{V}{W}^{{P}}.$

Treatment information extractor (TIE). Individuals’ behavior often changes
significantly in different treatments. To realize the adaptation of each
feature to different treatments, we use a TIE to model treatment information.
The input of the TIE is the treatment embedding $E^{tr}$, and we perform
feature extraction through an MLP:

(12) ${E}^{{bit}}\in\mathbb{R}^{1\times d}=MLP(E^{tr}).$

Each layer of MLP uses the Relu activation function, and the output dimension
of the last layer of MLP remains the same as the initial dimension.

After obtaining the treatment information, we use $E^{bit}$ to measure the
cross-representation $E^{att}$ to get the treatment-aware representation:

(13) ${E^{TAU}}=E^{att}\otimes E^{bit},$

where $\otimes$ is the element-wise product. We achieve treatment awareness by
fusing cross-features and treatment information, i.e., features have different
representations in different treatments.

We use two independent TAUs to get treatment-aware representation and bit-
level weights:

(14) ${E^{TAU}=TAU_{G}(E^{x},E^{tr})\in\mathbb{R}^{f\times
d},~{}W_{b}=TAU_{W}(E^{x},E^{tr})\in\mathbb{R}^{f\times d}.}$

Next, we will introduce how to obtain treatment-enhanced representations
through $E^{TAU}$ and $W_{b}$.

####  4.2.2. Treatment-Enhanced Gate (TEGate)

The initial features indicate the natural state of the instance, and some
valuable information may be lost if the initial features are not entirely
considered (Wang et al., 2022). Therefore, after getting the treatment-aware
embedding and bit-level weights, we use a gating mechanism to fuse the two
parts at the bit-level to get the final treatment-enhanced embedding:

(15) ${E^{r}}\in\mathbb{R}^{f\times
d}={E^{x}}\otimes\sigma({W_{b}})+{E^{TAU}}\otimes(1-\sigma\left({W_{b}}\right)),$

where $\sigma(\cdot)$ is the sigmoid function. To explicit the expression of
treatment, we concatenate treatment features on this basis:

(16) ${E^{r}}\in\mathbb{R}^{(f+1)\times d}=Concat(E^{r},E^{tr}).$

Compared with the traditional feature interaction by attention weight, our
proposed TENet mainly has the following two advantages: 1) it balances the
initial and treatment-aware embedding to adapt features in different
treatments effectively; 2) it uses bit-level feature interaction to achieve
treatment-enhanced in a more fine-grained way.

##  5\. Experiments

We conducted extensive experiments to verify and analyze the performance of
ECUP in uplift modeling. In this section, first, we introduce the content of
the datasets, including the public dataset and the dataset collected from
Meituan. We analyze it to elaborate on the chain-bias problem faced by the
real industrial scene and further introduce the implementation details,
baselines, and experimental settings. Then, we compare the performance of ECUP
with baselines and analyze the role of each component of ECUP. Finally, we
verify the performance of ECUP on a real large-scale online system.

Table 1. The statistics of datasets. Dataset | CRITEO-UPLIFT | MT-LIFT
---|---|---
Size | 13,979,592 | 5,541,842
Features | 12 | 99
Average Visit/Click Ratio | 4.70% | 33.49%
Average Conversion Ratio | 0.29% | 6.82%
Relative Average Visit/Click Uplift | 27.07% | 56.81%
Average Visit/Click Uplift | 1.03% | 13.49%
Relative Average Conversion Uplift | 59.45% | 169.23%
Average Conversion Uplift | 0.12% | 5.19%
|  |

###  5.1. Experimental Setup

####  5.1.1. Datasets

We evaluate our method on both public and real-world industrial datasets, and
the detailed statistics are shown in Table 1.

CRITEO-UPLIFT (Diemert et al., 2021): Criteo AI Labs constructed it through a
collection of online controlled experiments on the network for large-scale
advertising uplift modeling. It has nearly 14 million instances, 12 features,
binary treatments, and two binary labels: visit and conversion. We estimate
the visit rate and conversion rate as two tasks in the chain and randomly
split the dataset for training and testing with a ratio of 7/3.

Our collected dataset MT-LIFT: To comprehensively evaluate the proposed
method, we collected a dataset from two months of intelligent coupon marketing
scenarios for food delivery in the Meituan app, which is China’s leading local
living platform. It has multiple treatments and label information for the
entire chain. To eliminate the impact of confounding factors on uplift
modeling, we collect it from randomized controlled trials, where treatments
were randomly assigned to ensure consistent potential distribution between the
treatment and control groups, and the causal graph of the impression space is
shown in Fig. 6. The dataset contains nearly 5.5 million instances, 99
features, treatment information, and two labels: click and conversion. Among
them, treatment is not binary and has five options. We estimate CTR and CVR as
two tasks in the chain, using last week’s data for testing and the remaining
data for training.

Figure 6. Causal graph of the intelligent coupon marketing scenario during the
collection period of our dataset MT-LIFT. The treatment (T) is independent of
the individual features (X), and the two jointly affect subsequent click (Y)
and conversion (Z) behaviors.

####  5.1.2. Implementation Details

We implement all the deep learning baselines and ELEU with PyTorch 1.9.0 using
NVIDIA A100-SXM4-80GB. We use the area under the uplift curve (AUUC) and the
area under the qini curve (QINI) as evaluation metrics, which are standard
metrics to evaluate uplift models and computed using CausalML packages (Chen
et al., 2020). We use Adam as the optimizer, and the batch_size is $2^{11}$,
the total number of layers of each tower is $L$ is 3, the number of layers of
treatment information extractor is 1, the number of layers of Task-Enhanced
Gate is 2, and the number of heads of the multi-head attention mechanism is 2.
For the remaining hyper-parameters, we search them using AUUC as a primary
evaluation metric, and the range of the values of the hyper-parameters is
shown in Table 2.

Table 2. Hyper-parameters and their value range. Name | Range | Functionality
---|---|---
$d$ |  {$2^{3}$, $2^{4}$, $2^{5}$} | Embedding dimension
$lr$ |  {$1e^{-4}$, $1e^{-3}$, $1e^{-2}$} | Learning rate
$h$ |  {$2^{7}$, $2^{8}$, $2^{9}$} |  | Task’s hidden units of first layer
---
$h_{gate}$ |  {$2^{6}$, $2^{7}$, $2^{8}$, $2^{9}$} |  | TAEGate’s hidden units
---
|  |
Table 3. Performance comparison with other models.

| CTCVR-Task | CVR-Task
---|---|---
Dataset | CRITEO-UPLIFT | MT-LIFT | CRITEO-UPLIFT | MT-LIFT
Metrics | AUUC | QINI | AUUC | QINI | AUUC | QINI | AUUC | QINI
Causal Forest | 0.826 | 0.332 | 0.694 | 0.231 | 0.636 | 0.107 | 0.670 | 0.172
CEVAE | 0.635 | 0.131 | 0.577 | 0.079 | 0.643 | 0.133 | 0.525 | 0.049
CFRNet | 0.895 | 0.399 | 0.649 | 0.116 | 0.547 | 0.046 | 0.592 | 0.101
TarNet | 0.634 | 0.138 | 0.711 | 0.068 | 0.551 | 0.052 | 0.610 | 0.066
DragonNet | 0.889 | 0.395 | 0.533 | 0.005 | 0.545 | 0.016 | 0.521 | 0.003
DESCN | 0.902 | 0.406 | 0.706 | 0.219 | 0.502 | 0.011 | 0.633 | 0.138
ANU | 0.886 | 0.397 | 0.773 | 0.233 | 0.607 | 0.097 | 0.634 | 0.142
EFIN | 0.905 | 0.410 | 0.781 | 0.251 | 0.634 | 0.148 | 0.631 | 0.267
ECUP | 0.911 | 0.417 | 0.813 | 0.291 | 0.691 | 0.162 | 0.711 | 0.187

####  5.1.3. Baselines

To demonstrate the effectiveness of the ECUP, we selected the following
competitive baselines:

  * •

Causal Forest (Athey et al., 2019): A non-parametric ITE estimation model
based on generalized random forest.

  * •

CEVAE (Louizos et al., 2017): A neural network latent variable model based on
variational autoencoders for estimating individual and population causal
effects.

  * •

CFRNet (Shalit et al., 2017): A deep learning architecture using integral
probability metrics to adjust treatment and control group distance for ITE
estimation.

  * •

TarNet (Shalit et al., 2017): A variant of CFRNet without balancing
regularization, which learns the common information representation of
treatment and control groups and then estimates ITE through their respective
towers.

  * •

DragonNet (Shi et al., 2019): A neural network architecture for causal
estimation based on the sufficiency of the propensity score with targeted
regularization.

  * •

DESCN (Zhong et al., 2022): A method for jointly learning the distribution of
treatment and response functions in the entire sample space, which captures
the integrated information of the treatment propensity, the response, and the
hidden treatment effect through a cross network in a multi-task learning
manner.

  * •

ANU (Xu et al., 2022): An uplift model with an attention mechanism, which
generates after-treatment representation with relevant features of the given
treatment, utilizes attention mechanisms to map the original covariate space
into a latent space, sharing information about treatment and control groups.

  * •

EFIN (Liu et al., 2023): An explicit feature interaction aware uplift network,
which models treatment features, uses attention weights to describe the
sensitivity of non-treatment features to a particular treatment and uses an
intervention constraint module to increase the model’s robustness.

####  5.1.4. Experimental Settings

We evaluate the performance at two tasks: 1) Uplift estimation of the CTCVR
task, which we evaluate on the dataset of all impressions. The existing uplift
baselines are all single-task estimation methods. For a fair comparison, we
construct two models for them, the CTR and the CVR models, and obtain outputs
by calculating $pCTCVR=pCTR\times pCVR$. The CTR and CVR models are trained on
the impression set and click set, respectively, and the model architecture and
hyperparameters are consistent, which eliminates the negative impact of chain-
bias to a certain extent. 2) Uplift estimation of CVR tasks on the click
dataset.

###  5.2. Overall Performance

In the experiment, since most baselines can only handle binary treatment
scenarios, we extended it reasonably to adapt to industrial data with multiple
treatments, such as changing the network structure from 2-head to multi-head
or constructing K independent models to model different treatments separately.
For industrial dataset, we consider them as multiple binary treatment tasks
and report their average performance. We report the performance of each model
on two datasets in Table 3. Among the baselines, DESCN proposes to model the
uplift score in the entire space, but it refers to the entire treatment and
control group data space, which is difficult to expand to multiple treatments
and cannot solve the chain-bias.

For the results of the CTCVR task, we can draw the following conclusions: 1)
Some methods show significant performance degradation on MT-LIFT data, which
indicates that capturing user behavior changes under various treatments is
more difficult in feature-rich marketing scenarios. 2) The methods EFIN and
ANU, which consider the interaction between treatment information and other
features, bring relatively significant improvements. For MT-LIFT data,
separate modeling of treatment information brings more significant
improvements. 3) Although we separately model CTR and CVR to alleviate the
chain-bias in baselines, ECUP achieves excellent performance on the vast
majority of metrics, thanks to fine-grained treatment interactions and entire
chain space modeling, effectively solving the chain-bias and treatment-
unadaptive problems.

We can observe similar results on the CVR task, and ECUP achieved optimal
performance on most metrics. ECUP models uplift in the entire chain space, and
the rich data of previous tasks of the chain can effectively alleviate the
data sparsity problem of subsequent tasks, achieving better uplift estimation.

###  5.3. Ablation Study

To assess the effectiveness of each component in ECUP, we conducted a series
of ablation studies on the CTCVR task using the MT-LIFT dataset. Specifically,
we build several variants of the ECUP:

  * •

w/o TENet: A variant of ECUP without the TENet, which concatenates the
treatment feature with other features as input to the model.

  * •

w/ Attention: A variant of ECUP with the attention module, which uses a multi-
head attention mechanism to replace TENet, models the interaction between
treatment and other features.

  * •

w/o TAEGate: A variant of ECUP without the TAEGate, which does not inject the
task’s prior information and only captures the impact of treatment on
different tasks through the DNN layer.

  * •

w/o ECENet: A variant of ECUP without the ECENet, which does not model ITE in
the entire chain space, does not inject task prior information and trains
directly on the click set. This variant evaluates the performance of CVR
uplift in real application scenarios (impression space).

Table 4 shows the results of the ablation study, and we can draw the following
conclusions. Without the TENet, the model’s performance decreases; after using
the attention mechanism to interact the treatment with other features, the
performance has improved but is still not as good as TENet, which illustrates
the importance of fine-grained treatment-enhanced feature adaptation. The same
results can be observed without the TAEGate; injecting task prior information
can help the model capture individual response changes for different tasks.
Without the ECENet, the model performance degrades most severely, indicating
that the model trained on the click set makes achieving excellent performance
in real application scenarios difficult.

Through the component analysis of ECUP, we conclude that 1) all components in
ECUP contribute to the ITE estimation performance improvement; 2) the ECENet
structure is crucial as it accurately captures the degree of individual
response changes to different tasks under a specific treatment, eliminating
the negative impact of chain-bias on subsequent decisions.

Table 4. The contributions of different components of ECUP. | CTCVR-Task | CVR-Task
---|---|---
| AUUC | QINI | AUUC | QINI
w/o TENet | 0.786 | 0.289 | 0.672 | 0.178
w/ Attention | 0.793 | 0.289 | 0.686 | 0.185
w/o TAEGate | 0.785 | 0.283 | 0.678 | 0.169
w/o ECENet | 0.743 | 0.269 | 0.630 | 0.147
ECUP | 0.813 | 0.291 | 0.711 | 0.187
|  |  |  |

###  5.4. Performance on Online System (Meituan)

To evaluate the online performance of ECUP, we deployed it on the Meituan and
conducted a rigorous A/B test for two weeks. Our online scenario adopts a two-
stage structure, first obtaining each user’s rating on each coupon through the
uplift model and then assigning treatment. The baseline model on the online
platform is a T-learner, and its basic is a neural network (NN) model, a CVR
uplift model trained on the click set. We use three important online metrics
for evaluation: the marketing Return On Investment (ROI), the Gross
Merchandise Volume (GMV), and the Order Quantity (OQ).

Table 5 shows the improvement of ECUP relative to the baseline. The results
indicate that ECUP has significantly improved in all online metrics compared
to the baseline, achieving significant business benefits. This means that our
ECUP can more effectively capture user behavior changes under different
treatments and obtain more accurate uplift scores.

Table 5. Online A/B test results. | ROI | GMV | OQ
---|---|---|---
Baseline (T-learner + NN) | 0.0% | 0.0% | 0.0%
ECUP |  +7.4% |  +5.3% |  +5.1%
|  |  |

##  6\. Related Work

###  6.1. Uplift Modeling

Existing uplift research mainly has two directions: 1) Accurate uplift
estimation. The meta-learning method has become a popular framework in uplift
estimation by easily integrating existing models (Yao et al., 2021). The
T-learner (Künzel et al., 2019) applies two models to learn conditional
treated/control outcomes and estimates ITE through the subtraction between the
two models in the inference stage. In contrast, the S-learner (Künzel et al.,
2019) treats treatment as a feature and estimates the combined outcome by a
single model, estimating ITE through the subtraction between outcomes from
different treatments. The performance of the basic model has a cascading
effect on the meta-learner, and the deep learning model has become the basic
model of each framework due to its excellent feature extraction capabilities
(Xu et al., 2022; Yoon et al., 2018; Bica et al., 2020; Yao et al., 2021). 2)
Bias calibration. Since the actual treatment is allocated by particular
treatment strategies (not by random), there are differences in the potential
distribution and data volume between the treatment and control groups (called
treatment bias), and existing research performs bias calibration at the data
and model levels (Imbens and Rubin, 2015; Kuang et al., 2017; Rosenbaum, 1987;
Stuart, 2010; Abadie et al., 2004). Tree-based models (Chipman et al., 2010;
Wager and Athey, 2018) and some meta-learning (Künzel et al., 2019) can
alleviate this problem, and various deep learning methods have achieved better
performance in distribution debiasing (Zhong et al., 2022; Ke et al., 2021).

However, existing methods have yet to study chain-bias, which will have a
severe negative impact on later marketing decisions. In this paper, we achieve
accurate and unbiased uplift estimation by context-enhanced representation and
modeling the uplift influence relationship between entire chain tasks, which
can quickly generalize to multi-treatment and other chain uplift estimation
problems.

###  6.2. Multi-Task Learning

This paper uses the multi-task method to model the uplift of the entire chain.
Multi-task learning aims to model multiple related tasks simultaneously and
learn shared representations from multi-task supervision signals. Compared
with single-task learning, the memory footprint is reduced due to their
inherent layer sharing, and the effective use of related tasks to share
complementary information or act as a regularizer for each other can
effectively improve performance (Vandenhende et al., 2021). Before the era of
deep learning, researchers tried to model common information between tasks to
get better generalization performance (Daumé III and Kumar, 2013; Zhou et al.,
2011). As deep learning gradually matures, researchers use multi-task
supervision signals to learn shared representations. In the hard parameter
sharing days, the parameter set was divided into shared and task-specific
parameters, usually branched to the task-specific head by a shared encoder
(Brüggemann et al., 2020; Guo et al., 2020). To avoid negative transfer and
achieve better performance, researchers propose soft parameter sharing, where
each task is assigned its own parameters, and using a feature sharing
mechanism to handle cross-task dialogue (Gao et al., 2019). Besides, other
methods use gating mechanisms and expert networks to improve fine-grained
learning (Ma et al., 2018b; Chang et al., 2023).

Although multi-task learning has been successful on many domains, research in
uplift modeling still needs to be done. We aim to bridge the gap in this
research direction by designing a multi-task learning method to solve the
chain-bias and alleviate the data sparsity problem in subsequent tasks.

##  7\. CONCLUSIONS AND FUTURE WORK

In this paper, we studied the uplift modeling in a chain. We first elaborated
on the chain-bias and treatment-unadaptive problems in uplift modeling and
proposed the Entire Chain UPlift method with context-enhanced learning (ECUP)
to address these challenges. ECUP models the uplift score in the entire chain
space through the Entire Chain Enhanced Network to solve the chain-bias and
data sparsity in subsequent tasks, capturing the impact of treatment on
different tasks by injecting prior task information. ECUP further achieves
fine-grained and treatment-aware feature adaptation through the Treatment
Enhanced Network to accurately grasp the impact of features on results under
different treatments. We demonstrated that ECUP outperforms state-of-the-art
models through extensive offline and online experiments and released a large-
scale unbiased dataset collected from Meituan for future research.

Unbiased data construction requires a particular cost and a period of online
performance loss, which cannot be easily constructed in many scenarios. In the
future, we will explore multiple chain uplift modeling methods to reduce the
model’s requirements for data quality.

## References

  * (1)
  * Abadie et al. (2004) Alberto Abadie, David Drukker, Jane Leber Herr, and Guido W Imbens. 2004.  Implementing matching estimators for average treatment effects in Stata.  _The stata journal_ 4, 3 (2004), 290–311.
  * Athey et al. (2019) Susan Athey, Julie Tibshirani, and Stefan Wager. 2019\.  GENERALIZED RANDOM FORESTS.  _The Annals of Statistics_ 47, 2 (2019), 1148–1178.
  * Bica et al. (2020) Ioana Bica, James Jordon, and Mihaela van der Schaar. 2020\.  Estimating the effects of continuous-valued interventions using generative adversarial networks.  _Advances in Neural Information Processing Systems_ 33 (2020), 16434–16445.
  * Brüggemann et al. (2020) David Brüggemann, Menelaos Kanakis, Stamatios Georgoulis, and Luc Van Gool. 2020.  Automated Search for Resource-Efficient Branched Multi-Task Networks. In _31st British Machine Vision Conference 2020, BMVC 2020_. BMVA Press, 359.
  * Chang et al. (2023) Jianxin Chang, Chenbin Zhang, Yiqun Hui, Dewei Leng, Yanan Niu, Yang Song, and Kun Gai. 2023.  Pepnet: Parameter and embedding personalized network for infusing with personalized prior information. In _Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining_. 3795–3804.
  * Chen et al. (2020) Huigang Chen, Totte Harinen, Jeong-Yoon Lee, Mike Yung, and Zhenyu Zhao. 2020.  Causalml: Python package for causal machine learning.  _arXiv preprint arXiv:2002.11631_ (2020).
  * Chipman et al. (2010) Hugh A Chipman, Edward I George, and Robert E McCulloch. 2010\.  BART: Bayesian additive regression trees.  (2010).
  * Daumé III and Kumar (2013) Abhishek Kumar Hal Daumé III and A Kumar. 2013.  Learning task grouping and overlap in multi-task learning. In _International Conference on Machine Learning_. 1723–1730.
  * Diemert et al. (2021) Eustache Diemert, Artem Betlei, Christophe Renaudin, Massih-Reza Amini, Théophane Gregoir, and Thibaud Rahier. 2021.  A large scale benchmark for individual treatment effect prediction and uplift modeling.  _arXiv preprint arXiv:2111.10106_ (2021).
  * Gao et al. (2019) Yuan Gao, Jiayi Ma, Mingbo Zhao, Wei Liu, and Alan L Yuille. 2019.  Nddr-cnn: Layerwise feature fusing in multi-task cnns by neural discriminative dimensionality reduction. In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_. 3205–3214.
  * Guo et al. (2020) Pengsheng Guo, Chen-Yu Lee, and Daniel Ulbricht. 2020\.  Learning to branch for multi-task learning. In _International conference on machine learning_. PMLR, 3854–3863.
  * Imbens and Rubin (2015) Guido W Imbens and Donald B Rubin. 2015.  _Causal inference in statistics, social, and biomedical sciences_.  Cambridge University Press.
  * Ke et al. (2021) Wenwei Ke, Chuanren Liu, Xiangfu Shi, Yiqiao Dai, S Yu Philip, and Xiaoqiang Zhu. 2021\.  Addressing exposure bias in uplift modeling for large-scale online advertising. In _2021 IEEE International Conference on Data Mining (ICDM)_. IEEE, 1156–1161.
  * Kuang et al. (2017) Kun Kuang, Peng Cui, Bo Li, Meng Jiang, and Shiqiang Yang. 2017.  Estimating treatment effect in the wild via differentiated confounder balancing. In _Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining_. 265–274.
  * Künzel et al. (2019) Sören R Künzel, Jasjeet S Sekhon, Peter J Bickel, and Bin Yu. 2019.  Metalearners for estimating heterogeneous treatment effects using machine learning.  _Proceedings of the national academy of sciences_ 116, 10 (2019), 4156–4165.
  * Lin et al. (2017) Ying-Chun Lin, Chi-Hsuan Huang, Chu-Cheng Hsieh, Yu-Chen Shu, and Kun-Ta Chuang. 2017.  Monetary discount strategies for real-time promotion campaign. In _Proceedings of the 26th International Conference on World Wide Web_. 1123–1132.
  * Liu et al. (2023) Dugang Liu, Xing Tang, Han Gao, Fuyuan Lyu, and Xiuqiang He. 2023.  Explicit Feature Interaction-aware Uplift Network for Online Marketing.  _arXiv preprint arXiv:2306.00315_ (2023).
  * Louizos et al. (2017) Christos Louizos, Uri Shalit, Joris M Mooij, David Sontag, Richard Zemel, and Max Welling. 2017\.  Causal effect inference with deep latent-variable models.  _Advances in neural information processing systems_ 30 (2017).
  * Ma et al. (2018b) Jiaqi Ma, Zhe Zhao, Xinyang Yi, Jilin Chen, Lichan Hong, and Ed H Chi. 2018b.  Modeling task relationships in multi-task learning with multi-gate mixture-of-experts. In _Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining_. 1930–1939.
  * Ma et al. (2018a) Xiao Ma, Liqin Zhao, Guan Huang, Zhi Wang, Zelin Hu, Xiaoqiang Zhu, and Kun Gai. 2018a.  Entire space multi-task model: An effective approach for estimating post-click conversion rate. In _The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval_. 1137–1140.
  * Rosenbaum (1987) Paul R Rosenbaum. 1987\.  Model-based direct adjustment.  _Journal of the American statistical Association_ 82, 398 (1987), 387–394.
  * Rubin (2005) Donald B Rubin. 2005\.  Causal inference using potential outcomes: Design, modeling, decisions.  _J. Amer. Statist. Assoc._ 100, 469 (2005), 322–331.
  * Shalit et al. (2017) Uri Shalit, Fredrik D Johansson, and David Sontag. 2017\.  Estimating individual treatment effect: generalization bounds and algorithms. In _International conference on machine learning_. PMLR, 3076–3085.
  * Shi et al. (2019) Claudia Shi, David Blei, and Victor Veitch. 2019.  Adapting neural networks for the estimation of treatment effects.  _Advances in neural information processing systems_ 32 (2019).
  * Stuart (2010) Elizabeth A Stuart. 2010\.  Matching methods for causal inference: A review and a look forward.  _Statistical science: a review journal of the Institute of Mathematical Statistics_ 25, 1 (2010), 1.
  * Vandenhende et al. (2021) Simon Vandenhende, Stamatios Georgoulis, Wouter Van Gansbeke, Marc Proesmans, Dengxin Dai, and Luc Van Gool. 2021.  Multi-task learning for dense prediction tasks: A survey.  _IEEE transactions on pattern analysis and machine intelligence_ 44, 7 (2021), 3614–3633.
  * Wager and Athey (2018) Stefan Wager and Susan Athey. 2018.  Estimation and inference of heterogeneous treatment effects using random forests.  _J. Amer. Statist. Assoc._ 113, 523 (2018), 1228–1242.
  * Wang et al. (2022) Fangye Wang, Yingxu Wang, Dongsheng Li, Hansu Gu, Tun Lu, Peng Zhang, and Ning Gu. 2022.  Enhancing CTR prediction with context-aware feature representation learning. In _Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval_. 343–352.
  * Xu et al. (2022) Guoqiang Xu, Cunxiang Yin, Yuchen Zhang, Yuncong Li, Yancheng He, Jing Cai, and Zhongyu Wei. 2022.  Learning discriminative representation base on attention for uplift. In _Pacific-Asia Conference on Knowledge Discovery and Data Mining_. Springer, 200–211.
  * Yao et al. (2021) Liuyi Yao, Zhixuan Chu, Sheng Li, Yaliang Li, Jing Gao, and Aidong Zhang. 2021\.  A survey on causal inference.  _ACM Transactions on Knowledge Discovery from Data (TKDD)_ 15, 5 (2021), 1–46.
  * Yoon et al. (2018) Jinsung Yoon, James Jordon, and Mihaela Van Der Schaar. 2018\.  GANITE: Estimation of individualized treatment effects using generative adversarial nets. In _International conference on learning representations_.
  * Zhao et al. (2019) Kui Zhao, Junhao Hua, Ling Yan, Qi Zhang, Huan Xu, and Cheng Yang. 2019.  A unified framework for marketing budget allocation. In _Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_. 1820–1830.
  * Zhong et al. (2022) Kailiang Zhong, Fengtong Xiao, Yan Ren, Yaorong Liang, Wenqing Yao, Xiaofeng Yang, and Ling Cen. 2022.  DESCN: Deep Entire Space Cross Networks for Individual Treatment Effect Estimation. In _Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining_. 4612–4620.
  * Zhou et al. (2011) Jiayu Zhou, Jianhui Chen, and Jieping Ye. 2011.  Clustered multi-task learning via alternating structure optimization.  _Advances in neural information processing systems_ 24 (2011).