# Comparison Of Deep Object Detectors On A New Vulnerable Pedestrian Dataset

Devansh Sharma  Department of Computer Science, Bowling Green State University
Tihitina Hade  Department of Computer Science, Bowling Green State University
Qing Tian  Department of Computer Science, Bowling Green State University

###### Abstract

Pedestrian safety is one primary concern in autonomous driving. The under-
representation of vulnerable groups in today’s pedestrian datasets points to
an urgent need for a dataset of vulnerable road users. In this paper, we first
introduce a new vulnerable pedestrian detection dataset, BG Vulnerable
Pedestrian (BGVP) dataset to help train well-rounded models and thus induce
research to increase the efficacy of vulnerable pedestrian detection. The
dataset includes four classes, i.e., Children Without Disability, Elderly
without Disability, With Disability, and Non-Vulnerable. This dataset consists
of images collected from the public domain and manually-annotated bounding
boxes. In addition, on the proposed dataset, we have trained and tested five
state-of-the-art object detection models, i.e., YOLOv4, YOLOv5, YOLOX, Faster
R-CNN, and EfficientDet. Our results indicate that YOLOX and YOLOv4 perform
the best on our dataset, YOLOv4 scoring 0.7999 and YOLOX scoring 0.7779 on the
mAP 0.5 metric, while YOLOX outperforms YOLOv4 by 3.8 percent on the mAP
0.5:0.95 metric. Generally speaking, all five detectors do well predicting the
With Disability class and perform poorly in the Elderly Without Disability
class. YOLOX consistently outperforms all other detectors on the mAP
(0.5:0.95) per class metric, obtaining 0.5644, 0.5242, 0.4781, and 0.6796 for
Children Without Disability, Elderly Without Disability, Non-vulnerable, and
With Disability, respectively. Our dataset and codes are available at
https://github.com/devvansh1997/BGVP.

##  I INTRODUCTION

The need for new standards for pedestrian safety has grown in the era of
autonomous vehicles and advanced driver assistance systems (ADAS). Since 2018,
there have been 11 documented instances of Tesla Auto-pilot causing
collisions, and there have been about 37 test car crashes involving Uber,
which has resulted in four fatalities [1]. In the US, 104K instances of
pedestrian-related non-fatal accidents were documented in 2020 [2]. People
over the age of 65 made up 20 percent of all pedestrian fatalities in 2020,
while 1 in 5 children under the age of 15 who died in collisions were
pedestrians [2]. 61 percent of adult users, according to a Brookings
Institution survey, say they feel “uneasy” in autonomous vehicles [3]. Even
with new standards of safety in autonomous vehicles, there is some dissonance
in how the public perceives the dependability of these self-driving
mechanisms. When a self-driving Uber car hit a pedestrian crossing the street
in 2018, the incident resulted in the death of the victim. The vehicle failed
to react swiftly enough to avoid the crash because the software had
incorrectly labeled the pedestrian as a “False-Positive” [4]. All these
reports imply the need of well trained object detectors that don’t just cater
to the average pedestrian, but also to the more vulnerable groups.

Visual detection has recently become crucial to numerous industries, including
autonomous driving. However, deep visual detectors face the risk of being
biased and performing imprecisely on susceptible groups of people since
vulnerable road users are underrepresented in most, if not all, popular
pedestrian datasets. Such bias against at-risk pedestrians has less to do with
the model architecture than the data used for training.

“Pedestrian” usually refers to a person walking along a road or in a nearby
area. Most datasets ignore the vulnerability of pedestrians. Given the
scarcity and variety of vulnerable pedestrians on the road, most of today’s
datasets fail to distinguish them and label them differently. However, the
object detectors must be able to assess pedestrians who typically are more
vulnerable in order to appropriately handle a traffic situation involving such
groups. In light of this, we make two contributions in this work. A vulnerable
pedestrian dataset will be introduced as the initial contribution, giving the
community the resources they need to work on and enhance the effectiveness of
object detection algorithms for vulnerable pedestrians. Our dataset, which
includes underrepresented vulnerable road users, can also be used as a
supplement to current general pedestrian datasets. The second contribution is
that we examine and benchmark the performance of numerous cutting-edge
detectors on our new dataset, including YOLOv4[5], YOLOv5[6], YOLOX[7], Faster
R-CNN[8], and EfficientDet[9].

The remainder of this paper is structured as follows: Section II introduces
the related work, Section III presents the dataset, Section IV describes the
models used for benchmarking our dataset, Section V presents experiments and
discussion, and lastly, Section VI concludes the paper.

##  II RELATED WORK

We have witnessed an increase in datasets focusing on particular categories of
objects as a result of the increased use of object detection for automation
and security. With the use of these datasets, researchers try to customize the
model’s performance so that it performs well on both, a wide range of object
classes and on specific subset of objects within these classes. Pedestrians
are one such object that pertains to the autonomous driving industry, where
the ability to detect various pedestrians is one key factor of robust self-
driving software.

In [10], Kumar et al.present the P-DESTRE dataset that uses UAV video
surveillance data for pedestrian detection and tracking. Richly Annotated
Pedestrian (RAP)[11] is an extensive dataset that provides pedestrian data
from an uncontrolled scene with various viewpoints of the same area. Aimed at
object tracking, [12] provides pedestrian data comprised of pedestrians’ and
cyclists’ trajectories. OpenPTDS[13] has a similar approach where they
introduced a dataset created from real-life experiments. They target the
problem of pedestrian detection from the viewpoint of safety and security
around metropolitan buildings. The Caltech Pedestrian Dataset, which was
released in 2009, is regarded as one of the top pedestrian datasets to date.
It included an annotation tool and a substantial amount of video footage with
annotated frames. Along with comparing models (at the time) against their
dataset, it also supplied a fresh standard for testing object detection
algorithms on pedestrian data [14]. In [15], along with a new pedestrian
dataset, a new per-frame methodology was introduced to analyze scale and
occlusion. Using the thermal channel provided a different perspective for
pedestrian detection tasks to identify in dark environments and provided a
dataset of color-thermal pair images [16].

Existing works on vulnerable pedestrians focus on offering a system,
algorithm, or model. Song et al. explore safety advancements and offer an
action character deduction and analysis module for video streams focusing on
vulnerable pedestrians in [17]. Pedestrian-Oriented Forewarning System
(POFS)[18] is another example mechanism implemented using smartphone
communication to detect a possible accident. Their paper identifies
pedestrians who are distracted by their mobile phones as vulnerable
pedestrians and presents the aforementioned system to alert them in an
adaptive manner.

Most of the work in this field of pedestrian detection either does not focus
on vulnerable pedestrians or ignores the potential impact of the data. The
absence of a dataset focusing on vulnerable road users translates to biases
against vulnerable people such as children, the elderly, and people with
disability. In this paper, we try to tackle this issue by taking a data-
centric solution. We introduce a new dataset focusing on vulnerable
pedestrians or road users. The vulnerability that we are interested in is
related to a road user’s age and physical disability. We hope that this new
benchmark dataset will stem further research in this particular field.

---
Figure 1: Simple traffic scenes in our BGVP dataset

##  III BG Vulnerable Pedestrian (BGVP) dataset

Vulnerable pedestrians deserve more attention from self-driving vehicles. It
is currently not the case because of their under-representation in most
training data, which results in unfair and discriminatory treatment of them.
Such a bias motivates us to create a dataset focusing on vulnerable
pedestrians. There is a stark difference between an elderly disabled
pedestrian and a young adult in their early 20s, crossing the road. Most
accidents occur due to human errors, which may be avoided if autonomous
driving becomes a standard. However, this requires exceptional functionality
from the machines or software operating the vehicles. The object detectors
should be able to quickly and precisely detect pedestrians and categorize them
based on their vulnerability type. The vulnerability type is crucial
information as it can help estimate the pedestrian’s actions in a traffic
scenario or emergency. The actions of a child aged 5-6, even when accompanied
by an adult, can be considered unpredictable and risky. The detection of the
child must be quick to control speed and predict the child’s future positions
with respect to the vehicle. This desirable behavior necessitates autonomous
driving models to train/learn on a dataset where such vulnerable groups have a
significant presence.

Over the years, many pedestrian datasets have been published, but none of them
focus on or mention the vulnerability of pedestrians [15][10][19][11][20]. We
provide a new dataset that meets this demand and will enable testing of future
object detection models against it to determine how well a particular model
can learn pedestrian vulnerability.

We need to comprehend the many classes and how each pedestrian is qualified to
fall into a certain class before delving into the dataset statistics. The
bounding box instances in our dataset are divided into four categories:
“Children Without Disability”, “Elderly Without Disability”, “With
Disability”, and “Non-vulnerable”.

  * •

Children without Disability: This vulnerable pedestrian class encompasses any
pedestrian between 1 to 16 years of age. Most children are considered
vulnerable because their behavior is potentially erratic, and they may not
understand traffic laws. Some early teens can also be considered a risk to a
self-driving car, which is why we include them in the same category. Children
without a physical disability are considered to fall into this category.

  * •

Elderly without Disability: Elderly pedestrians are particularly at risk. Even
though the Elderly may be well versed with traffic laws, their age may
restrict many physical motions, and on average, they tend to require more time
to complete the same action compared to a non-vulnerable pedestrian. The
minimum age of the senior age demographic is subjective and differs across
cultures. In this paper, this group is determined by the first author to be
people who appear in the age group of 50 or older. People from this age group
without any physical disability are deemed eligible for this class.

  * •

With Disability: The most vulnerable group of all consists of pedestrians who
have a physical impairment and may require some form of assistance. The
pedestrian could be of any age group, but if they have a physical disability,
they are categorized into this class because their age-related constraints are
overshadowed by the disability. In our dataset, physical disability aids that
we consider are wheelchairs, mobility walkers, scooters, crutches, and walking
canes.

  * •

Non-Vulnerable: This group of pedestrians comprises of people that do not fall
into any of the classes mentioned above, and it is safe to say that they are
less vulnerable in traffic situations than other categories. These pedestrians
do not have any physical (or visible) disability. They are supposed to
act/respond quickly, and understand the situation better than vulnerable
pedestrians. This is not to say that these pedestrians are not at all
vulnerable; rather, their level of vulnerability in a traffic situation is
lower than that of the categories mentioned above.

---
Figure 2: Complicated traffic scenes in our BGVP dataset

Figure 1 displays rows with a small sample of images from each class. Figure 2
displays a random selection of photos from our database that feature complex
traffic scenes along with how our annotations appear on top of them
111category labels are not shown for clarity. Figure 2 images have various
class combinations, and some cover all classes. Our dataset comprises 2,000
images with a total of 5,932 bounding box annotations. On average, there are
three annotations in each image and the median image size is 600 $\times$ 408
pixels. The dataset contains an assortment of images with different
dimensions. The dimension of the largest image is 6000 $\times$ 4000 and the
smallest is 99 $\times$ 159\. All images that we have collected are from the
public domain and they are available on the internet for free. “Children
Without Disability” is the most prominent vulnerable pedestrian class, with
1,646 bounding box annotations, “Elderly Without Disability” has 815 bounding
box instances, “With Disability” has 942 instances, and “Non-vulnerable” has
2,529 instances. Fig. 3 visualizes the distribution of bounding box categories
in the dataset. Our dataset is divided into 1,405 images as the training set,
399 images as the validation set, and 196 images as the testing set.

---
Figure 3: Annotations distribution among the four categories

The majority of the photos we have gathered show these vulnerable people in
traffic situations, which should aid in providing the detection models with
pertinent contextual information which can benefit training. The contextual
information typically includes traffic lights, zebra crossings, vehicles, and
pedestrians (vulnerable or non-vulnerable).

It is also essential to find the right balance in the annotations that would
help the model learn instead of confusing it even more, and our annotations
would help in this regard. We annotate all pedestrians whose categories human
eyes can easily recognize. If more annotations are created than necessary,
that may lead to incorrect results. In our case, we have images that contain
many pedestrians, but many in the background are too small in scale to be
recognizable (we are unsure about what categories those pedestrians fall
into). In such cases, we choose to not annotate them rather than label them
incorrectly and mislead the detectors.

Model | Backbone | Input Size | Params | GFLOPs | mAP 0.5 | mAP 0.5:0.95
---|---|---|---|---|---|---
EfficientDet-D0 | Efficient-B0 | 512 | 3.9 M | 2.5 | 0.7048 | 0.4512
Faster R-CNN | Resnet-50 | 640 | 42 M | 180 | 0.7330 | 0.4860
YOLOv4 | CSPDarknet-53 | 640 | 27.6 M | 90.2 | 0.7999 | 0.5231
YOLOv5-s | Modified CSP v5 | 640 | 7.2 M | 16.9 | 0.7000 | 0.4800
YOLOX-s | Modified CSP v5 | 640 | 8.94 M | 26.8 | 0.7779 | 0.5616
TABLE I: Model Description and mAP Scores

##  IV Models for Vulnerable Pedestrian Detection

In this paper, we experiment with a variety of state-of-the-art object
detection models on our introduced benchmark. We cover both single-stage and
two-stage visual detectors, i.e., YOLOv4[5], YOLOv5[6], YOLOX[7], Faster
R-CNN[8], and EfficientDet[9].

###  IV-A Single-stage detectors

Due to their high speed and efficiency, single-stage detectors are usually
preferable in time-sensitive scenarios like autonomous driving. This is also
the reason why most of the detectors tested in this paper belong to this
category.

YOLOs are a most famous family of one-stage detectors that have revolutionized
the object detection community. Redmon and his colleagues presented the first
three versions of YOLO [21, 22, 23], making important advancements along the
way. YOLOv4 [5] uses CSPDarknet53 backbone, SPP additional module and PANet
path-aggregation neck. The detector head is similar to that of YOLOv3. YOLOv4
also introduces new data augmentation methods such as Mosaic and Self-
Adversarial Training(SAT). YOLOv5 was released shortly after YOLOv4 by
Ultralytics via GitHub[6]. YOLOv5 is similar to YOLOv4, but one difference is
that YOLOv5 leverages auto-learning bounding box anchors. YOLOv5 also has
different sizes, ranging from YOLOv5-S to YOLOv5-X, with S indicating the
smallest width and depth while X denoting the largest. YOLOX [7] starts with
YOLOv3 as its base and makes major changes, including (1) a decoupled head for
classification, regression and localization, (2) dropping the use of anchors,
(3) SimOTA advanced label assignment strategy, and (4) more advanced data
augmentations techniques.

EfficientDet [9], developed by Google AI, is another single-stage detector,
which is highly scalable and fast. EfficientDet uses EfficientNet[24] as the
backbone and a newly introduced BiFPN feature network. BiFPN enables easy and
fast feature fusion and allows bi-directional information flow.

###  IV-B Two-stage Detector

Compared to single-stage detectors, two-stage detectors are usually slow due
to the extra stage of regions of interest (RoI) proposal. Faster R-CNN [8] is
the most (at least one of the most) well-known two-stage detectors from
Facebook AI Research (FAIR). It is faster than its predecessor Fast R-CNN [25]
because instead of passing all region proposals to the feature-extracting
convnet, the convolution operation in Faster R-CNN is done only once per image
to generate a whole feature map. The region proposal task is done by the
proposed Region Proposal Network (RPN) in the latent feature space.

Model |  Children Without Disability |  Elderly Without Disability |  Non-vulnerable |  With Disability
---|---|---|---|---
EfficientDet-D0 |  0.4996 |  0.2932 |  0.3650 |  0.6469
Faster R-CNN |  0.5194 |  0.3889 |  0.3970 |  0.6371
YOLOv4 |  0.5360 |  0.4764 |  0.4494 |  0.6304
YOLOv5-s |  0.5120 |  0.3880 |  0.4040 |  0.6140
YOLOX-s |  0.5644 |  0.5242 |  0.4781 |  0.6796
TABLE II: Per-class mAP 0.5:0.95 of the models

##  V Experiments and Discussion

In this section, we benchmark the models previously mentioned on our
introduced BGVP dataset and report their results.

###  V-A Experimental Setup

We follow each model’s authors’ recommendations to set the hyper-parameters.
We train the five state-of-the-art models on the proposed BGVP dataset. Our
training is performed on google colab and the Ohio Supercomputer Center (OSC)
clusters. We report our results in terms of the Mean Average Precision (mAP).
We use mAP 0.5 and mAP 0.5:0.95 as our primary metrics to analyze model
performance.

(a) YOLOX

(b) YOLOv4

(c) EfficientDet

(d) Faster R-CNN

(e) YOLOv5

Figure 4: Confusion Matrix

###  V-B Experimental Results

Table I shows the different models and their respective mAP scores. YOLOX and
YOLOv4 perform the best among all. YOLOv4 has the highest score on mAP 0.5,
scoring 0.7999 and YOLOX is closely behind scoring 0.7779. YOLOX achieved the
best mAP 0.5:0.95 score (0.5616), outperforming YOLOv4 by 3.8 percent. Faster
R-CNN, a two-stage detector, also performed reasonably well and scored 0.7330
for mAP 0.5 and 0.4860 for the mAP 0.5:0.95 metric.

In Table II, we report per-class scores for the mAP 0.5:0.95 metric. We can
see that most, if not all, models have better mAP scores for the “With
Disability” class. This could be due to the presence of regular forms that
make up the physical walking aid, such as wheelchairs and walking canes.
Relatively speaking, the models do not perform well in “Elderly without
Disability” and “Non-vulnerable” classes. This is perhaps due to the
unavoidable ambiguity and subjectivity when determining whether a subject
meets the criteria for belonging to the senior age demographic. This
uncertainty can also be seen in Figure 4, which shows the confusion matrix
results of all models. EfficientDet and YOLOv5 have a fair share of incorrect
predictions of an elderly pedestrian as non-vulnerable. The YOLOX performs
much better than other detectors in this regard. Performance on the class
“Children without Disability” seems more consistent across all models and
averages to 0.5262 (mAP 0.5:0.95) while for the “Elderly without Disability”
class, the average is 0.4141. We also notice that all detectors perform poorly
on images with a darker background. Many times they do not detect a pedestrian
at all, irrespective of the category.

##  VI Conclusion

In this paper, we have presented a new object detection data set, namely BGVP,
for vulnerable pedestrians. We have collected 2,000 images and have annotated
5,932 bounding box instances from four categories, i.e., “Children Without
Disability”, “Elderly without Disability”, “With Disability”, and “Non-
Vulnerable”. After collection and annotation, we trained and tested five
state-of-the-art visual detectors and compared their results. We hope that
this dataset can serve the community and motivate/facilitate more future
research in this field. The dataset can also be utilized to fine-tune existing
object detectors for more precise and less biased detection of vulnerable
pedestrians, helping pave the way for safer autonomous driving.

## Acknowledgment

This work would not have been possible without the computing resources
provided by the Ohio Supercomputer Center. Also, the authors would like to
acknowledge Akinkunle A. Akinola for his help in this project.

## References

  * [1] M. Tuwiner, “49 Self-Driving Car Accident Rate Statistics and Facts.” https://annapolisclassiccars.com/self-driving-cars-statistics-facts/, 2022\.
  * [2] “Pedestrian Safety.” https://www.cdc.gov/transportationsafety/pedestrian_safety/index.html, 2022\.
  * [3] A. Kopestinsky, “25 Astonishing Self-Driving Car Statistics For 2022.” https://policyadvice.net/insurance/insights/self-driving-car-statistics/, 2022\.
  * [4] “Fatalities Caused by Autonomous Vehicles.” https://qaconsultants.com/blog/fatalities-caused-by-autonomous-vehicles/, 2022\.
  * [5] A. Bochkovskiy, C.-Y. Wang, and H.-Y. M. Liao, “Yolov4: Optimal speed and accuracy of object detection,” arXiv preprint arXiv:2004.10934, 2020.
  * [6] “Ultrlytics - YOLOv5.” https://github.com/ultralytics/yolov5, 2022.
  * [7] Z. Ge, S. Liu, F. Wang, Z. Li, and J. Sun, “Yolox: Exceeding yolo series in 2021,” arXiv preprint arXiv:2107.08430, 2021.
  * [8] S. Ren, K. He, R. Girshick, and J. Sun, “Faster r-cnn: Towards real-time object detection with region proposal networks,” Advances in neural information processing systems, vol. 28, 2015.
  * [9] M. Tan, R. Pang, and Q. V. Le, “Efficientdet: Scalable and efficient object detection,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 10781–10790, 2020.
  * [10] S. Kumar, E. Yaghoubi, A. Das, B. Harish, and H. Proença, “The p-destre: a fully annotated dataset for pedestrian detection, tracking, re-identification and search from aerial devices,” arXiv preprint arXiv:2004.02782, 2020.
  * [11] D. Li, Z. Zhang, X. Chen, H. Ling, and K. Huang, “A richly annotated dataset for pedestrian attribute recognition,” arXiv preprint arXiv:1603.07054, 2016.
  * [12] J. Lambert, L. Liang, L. Y. Morales, N. Akai, A. Carballo, E. Takeuchi, P. Narksri, S. Seiya, and K. Takeda, “Tsukuba challenge 2017 dynamic object tracks dataset for pedestrian behavior analysis,” Journal of Robotics and Mechatronics, vol. 30, no. 4, pp. 598–612, 2018.
  * [13] X. Song, J. Sun, J. Liu, K. Chen, and H. Xie, “Openptds dataset: Pedestrian trajectories in crowded scenarios,” in Asian Simulation Conference, pp. 296–305, Springer, 2018.
  * [14] P. Dollár, C. Wojek, B. Schiele, and P. Perona, “Pedestrian detection: A benchmark,” in 2009 IEEE conference on computer vision and pattern recognition, pp. 304–311, IEEE, 2009.
  * [15] S. Zhang, R. Benenson, and B. Schiele, “Citypersons: A diverse dataset for pedestrian detection,” in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3213–3221, 2017.
  * [16] S. Hwang, J. Park, N. Kim, Y. Choi, and I. So Kweon, “Multispectral pedestrian detection: Benchmark dataset and baseline,” in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1037–1045, 2015.
  * [17] H. Song, I. K. Choi, M. S. Ko, J. Bae, S. Kwak, and J. Yoo, “Vulnerable pedestrian detection and tracking using deep learning,” in 2018 International Conference on Electronics, Information, and Communication (ICEIC), pp. 1–2, IEEE, 2018.
  * [18] Z. Liu, L. Pu, Z. Meng, X. Yang, K. Zhu, and L. Zhang, “Pofs: A novel pedestrian-oriented forewarning system for vulnerable pedestrian safety,” in 2015 International Conference on Connected Vehicles and Expo (ICCVE), pp. 100–105, IEEE, 2015.
  * [19] G. Overett, L. Petersson, N. Brewer, L. Andersson, and N. Pettersson, “A new pedestrian dataset for supervised learning,” in 2008 IEEE Intelligent Vehicles Symposium, pp. 373–378, IEEE, 2008.
  * [20] D. Yang, L. Li, K. Redmill, and Ü. Özgüner, “Top-view trajectories: A pedestrian dataset of vehicle-crowd interaction from controlled experiments and crowded campus,” in 2019 IEEE Intelligent Vehicles Symposium (IV), pp. 899–904, IEEE, 2019.
  * [21] J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You only look once: Unified, real-time object detection,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016.
  * [22] J. Redmon and A. Farhadi, “Yolo9000: better, faster, stronger,” in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 7263–7271, 2017.
  * [23] J. Redmon and A. Farhadi, “Yolov3: An incremental improvement,” arXiv preprint arXiv:1804.02767, 2018.
  * [24] M. Tan and Q. Le, “Efficientnet: Rethinking model scaling for convolutional neural networks,” in International conference on machine learning, pp. 6105–6114, PMLR, 2019.
  * [25] R. Girshick, “Fast r-cnn,” in Proceedings of the IEEE international conference on computer vision, pp. 1440–1448, 2015.


index and generate samples via a combination of local moves and mode-jumping
moves. Their mode-jumping moves are based on the Metropolis-Hastings
algorithm, whereas in our algorithm mode-jumping is achieved via Warp-U
transformations. Tak et al. (2018) proposed a repelling–attracting Metropolis-
Hastings (RAM) algorithm for exploring multi-modal distributions; it works by
constructing a proposal distribution which combines a move to low density and
a move back to high density.

Perhaps the best known general strategy for sampling from multi-modal
densities is parallel tempering (Geyer, 1991). Recent studies (Syed et al.,
2021, 2022; Surjanovic et al., 2022) proposed more efficient version parallel
tempering, focusing on its integration with a variational reference
distribution, the implementation of flexible annealing paths, and the adoption
of non-reversible communication schemes. Since parallel tempering is a
sampling algorithm, it needs to be combined with an estimation strategy, e.g.,
bridge sampling. In our simulation studies, we find that our proposed strategy
is computationally more efficient than parallel tempering for two reasons:
firstly, more of the computation for the estimation step is performed by our
Warp-U sampler than by parallel tempering, and secondly, the Warp-U sampler
involves only one chain and it always accepts inter-mode proposals.

Of course, it would be foolish to suggest Warp-U sampler dominates parallel
tempering (and other methods) for all cases, but the evidence we have
accumulated so far does suggest that it is a viable alternative in terms of
both computational cost and implementation effort. The experiments in Section
4.2 also indicate that the Warp-U sampler has the potential to surpass
parallel tempering in efficiency, particularly when dealing with target
distributions that are high-dimensional and feature modes with different
variances.

###  6.2 Limitations and Further Work

Whereas our overall approach is generally simple to implement because fitting
mixture distributions is typically straightforward (and easy to interpret), a
limitation is that the number of components in the mixture, $K$, has to be
chosen (or inferred), and further research is needed on optimally determining
this number without unduly increasing computational cost. Nevertheless, in our
limited study so far, we find that our approach is rather robust to $K$ as
long as it is not too small. Thus, with adequate computational resources, one
can avoid tuning by setting the number of components to be high initially.
Specifically, in many settings, starting with a moderate number of components,
e.g., $K=20$, is practical. If the target density has fewer than $K$ modes,
this will not substantially negatively impact the performance of the sampling
method, and the main price of specifying too many components is the
computational cost of updating the Gaussian mixture approximation
$\phi_{\text{mix}}$. It may be worthwhile to use a model selection criterion,
e.g., BIC, to determine $K$, and we encourage interested readers to explore
and investigate the performance of our methods as they adopt different methods
for choosing $K$.

A more serious limitation of our adaptive Warp-U MCMC sampler is that it is
likely inefficient in the case of very isolated modes, partly because it may
take time to identify the modes, and partly because its advantages over a
Metropolis-Hastings (MH) algorithm with a mixture density proposal rely on the
overlap between modes, and is diminished in the case of isolated modes. On the
other hand, sampling from isolated modes with unknown locations is universally
challenging. A possible strategy is to combine several methods to take
advantage of their different strengths, similar to the interweaving strategy
in Yu and Meng (2011). For example, one could consider implementing the Warp-U
sampler within each chain of PT, or even consider using some warp
transformation parameters themselves as temperature parameters to be
adaptively varied as the overall interwoven algorithm iterates. Such
interwoven algorithms would undoubtedly lose the simplicity of the Warp-U
sampler or of PT, but with the payoff of overcoming challenges that may defeat
the individual algorithms. Of course, much research is needed to explore such
ideas.

The experiment in Section 4.5 also reveals limitations of the proposed
adaptive Warp-U sampler, especially when applied to intricate, high-
dimensional target densities. However, these limitations can possibly be
mitigated by initializing a high-quality $\phi_{\rm mix}$ using variational
inference (Arenz et al., 2020; Yan et al., 2023). The experiment further
underscores that, even when substituting the random walk Metropolis step with
a Hamiltonian step, the algorithm might still exhibit inefficiencies in high-
dimensional local exploration. This is primarily attributed to the target
density’s irregular contour and heavy tail. The prevalent literature offers
solutions to the challenge posed by irregular density contours, primarily
through density transformation (Hoffman et al., 2019) or adaptation based on
local geometry (Girolami and Calderhead, 2011). The Warp-U transformation
again provides a promising alternative: instead of directly sampling from the
original target $\pi$ to explore locally, one could employ the Hamiltonian
step to draw from the Warp-U transformed $\tilde{\pi}$, which resembles a
Gaussian distribution and exhibits more regularity. Subsequently, the inverse
Warp-U transformation can be applied to get samples from $\pi$. This is
similar to the approach proposed in Algorithm 1 of Parno and Marzouk (2018).
On the other hand, the challenge posed by heavy-tailed densities could
possibly be solved by more sophisticated local proposals, e.g., local
proposals with jumps (Şimşekli, 2017).

We also want to point out that in stochastic Warp-U bridge sampling
estimation, when allocating the samples to different components in Steps 3–4
of Algorithm 5, there may be some components that have very few samples, which
will lead to high variance of the bridge sampling estimator for those
components. To address this problem, future work could develop a restriction
to ensure that each component has a minimum number of samples, or by
adaptively merging components to ensure sufficient mass for each component.
However, such merges may decrease the quality of the fit of the mixture model,
leading to (practical) higher bias, at least when the chains are still in the
exploratory stage. Therefore, much research is needed to investigate this
issue and to provide theoretically sound practical guidelines on how to deal
with such a bias-variance trade-off.

In the current work, we separately address two distinct estimators: the
stochastic Warp-U bridge estimator for estimating the normalizing constant,
and Warp-U sampling with an unbiased estimator for estimating
$\mathbb{E}_{\pi}h(\boldsymbol{\theta})$ for a general function $h$. A
potential avenue for future research could involve integrating these two
estimators. For example, the unbiased strategy can be applied to estimate the
denominator of the bridge estimator (see (20)).

The current unbiased estimation strategy is limited by its incurred
computational cost. Before the two chains meet, the computational cost is more
than double that of a single chain, because we need to compute the joint
transition kernel and draw samples from it. The additional cost can be
significant for a complex $q$ and a large number of mixture component $K$.
Moreover, in Section 2.4, we use optimal transport coupling to find the joint
transition kernel for the Warp-U transformation step and its inverse step.
This strategy minimizes the expected distance between the two chains and is
similar to Papp and Sherlock (2022). Understanding the limiting behaviour of
this coupling in high dimension and developing other coupling is also a
possible direction for future research.

Last but not the least, much needs to be done to address the biggest
theoretical gap in our current article. That is, the theoretical comparisons
we made about asymptotic variance or precision per CPU second are under the
assumption of i.i.d. draws, which is clearly violated by our algorithm and
other MCMC methods we used in this article. Whereas more painstaking efforts
to extend the results to dependent draws are certainly possible, the more
ideal approach is to implement Warp-U sampler perfectly (which would produce
i.i.d. draws), in the sense of the perfect sampling made possible by the
seminal work of Propp and Wilson (1996). Unfortunately, implementing perfect
sampling is typically a daunting task, and often impractical; see Craiu and
Meng (2011) for an overview and discussion. However, for our Warp-U sampler,
the mixture component index $\phi$ is discrete, which opens the door for
inducing practically feasible coalescence that is critical for implementing
perfect sampling; see for example, Stein and Meng (2013), who implemented
perfect sampling for the common Dirichlet-multinomial model by utilising the
discreteness of the mixture index. This is definitely a worthwhile direction
for further research; it will require hard work, especially for the adaptive
version of the Warp-U sampler, but the potential reward is large.

## Acknowledgements

We thank Yves Atchade and Pierre Jacob for very helpful comments, and US NSF
for partial financial support (to XLM).

## Supplementary Material

Supplementary Materials are available online, containing technical proofs, the
details of the unbiased Warp-U estimator and additional numerical results.

## References

  * Ali and Silvey (1966) S. M. Ali and S. D. Silvey.  A general class of coefficients of divergence of one distribution from another.  _Journal of the Royal Statistical Society: Series B (Methodological)_ , 28(1):131–142, 1966.
  * Andrieu and Robert (2001) C. Andrieu and C. P. Robert.  _Controlled MCMC for optimal sampling_.  INSEE, 2001.
  * Arenz et al. (2020) O. Arenz, M. Zhong, and G. Neumann.  Trust-region variational inference with gaussian mixture models.  _The Journal of Machine Learning Research_ , 21(1):6534–6593, 2020.
  * Atchadé (2006) Y. F. Atchadé.  An adaptive version for the metropolis adjusted langevin algorithm with a truncated drift.  _Methodology and Computing in applied Probability_ , 8(2):235–254, 2006.
  * Atchadé et al. (2011) Y. F. Atchadé, G. O. Roberts, and J. S. Rosenthal.  Towards optimal scaling of metropolis-coupled markov chain monte carlo.  _Statistics and Computing_ , 21(4):555–568, 2011\.
  * Azzalini (2013) A. Azzalini.  _The skew-normal and related families_ , volume 3.  Cambridge University Press, 2013.
  * Bennett (1976) C. H. Bennett.  Efficient estimation of free energy differences from monte carlo data.  _Journal of Computational Physics_ , 22(2):245–268, 1976.
  * Berg and Neuhaus (1991) B. A. Berg and T. Neuhaus.  Multicanonical algorithms for first order phase transitions.  _Physics Letters B_ , 267(2):249–253, 1991.
  * Betancourt and Girolami (2015) M. Betancourt and M. Girolami.  Hamiltonian monte carlo for hierarchical models.  _Current trends in Bayesian methodology with applications_ , 79(30):2–4, 2015.
  * Biswas et al. (2019) N. Biswas, P. E. Jacob, and P. Vanetti.  Estimating convergence of markov chains with l-lag couplings.  In _Advances in Neural Information Processing Systems_ , pages 7389–7399, 2019.
  * Bornn et al. (2013) L. Bornn, P. E. Jacob, P. Del Moral, and A. Doucet.  An adaptive interacting Wang–Landau algorithm for automatic density exploration.  _Journal of Computational and Graphical Statistics_ , 22(3):749–773, 2013.
  * Chimisov et al. (2018) C. Chimisov, K. Latuszynski, and G. Roberts.  Air markov chain monte carlo.  _arXiv preprint arXiv:1801.09309_ , 2018.
  * Craiu and Meng (2011) R. V. Craiu and X.-L. Meng.  Perfection within reach: exact mcmc sampling.  _Handbook of Markov Chain Monte Carlo_ , pages 199–226, 2011.
  * Craiu and Meng (2022) R. V. Craiu and X.-L. Meng.  Double happiness: Enhancing the coupled gains of l-lag coupling via control variates.  _Statistica Sinica_ , 32:1–22, 2022.
  * Danby (1988) J. M. Danby.  _Fundamentals of celestial mechanics, Willmann-Bell_ , volume 2.  1988\.
  * Geyer (1991) C. J. Geyer.  Markov chain monte carlo maximum likelihood.  _Computing science and statistics: Proceedings of 23rd Symposium on the Interface, Fairfax Station, 1991_ , pages 156–163, 1991.
  * Girolami and Calderhead (2011) M. Girolami and B. Calderhead.  Riemann manifold langevin and hamiltonian monte carlo methods.  _Journal of the Royal Statistical Society Series B: Statistical Methodology_ , 73(2):123–214, 2011.
  * Gronau et al. (2019) Q. F. Gronau, E.-J. Wagenmakers, D. W. Heck, and D. Matzke.  A simple method for comparing complex models: Bayesian model comparison for hierarchical multinomial processing tree models using warp-iii bridge sampling.  _psychometrika_ , 84(1):261–284, 2019.
  * Gronau et al. (2020) Q. F. Gronau, A. Heathcote, and D. Matzke.  Computing bayes factors for evidence-accumulation models using warp-iii bridge sampling.  _Behavior research methods_ , 52(2):918–937, 2020\.
  * Gupta (2003) A. K. Gupta.  Multivariate skew t-distribution.  _Statistics: A Journal of Theoretical and Applied Statistics_ , 37(4):359–363, 2003.
  * Heng and Jacob (2019) J. Heng and P. E. Jacob.  Unbiased hamiltonian monte carlo with couplings.  _Biometrika_ , 106(2):287–302, 2019.
  * Hoffman et al. (2019) M. Hoffman, P. Sountsov, J. V. Dillon, I. Langmore, D. Tran, and S. Vasudevan.  Neutra-lizing bad geometry in hamiltonian monte carlo using neural transport.  _arXiv preprint arXiv:1903.03704_ , 2019.
  * Jacob et al. (2020) P. Jacob, J. O’Leary, and Y. Atchadé.  Unbiased markov chain monte carlo with couplings (with discussion).  _JR Statist. Soc. Ser. B_ , 82:543–600, 2020.
  * Jacob et al. (2014) P. E. Jacob, R. J. Ryder, et al.  The Wang–Landau algorithm reaches the flat histogram criterion in finite time.  _The Annals of Applied Probability_ , 24(1):34–53, 2014.
  * Jarner and Hansen (2000) S. F. Jarner and E. Hansen.  Geometric ergodicity of metropolis algorithms.  _Stochastic processes and their applications_ , 85(2):341–361, 2000.
  * Jones et al. (2022) D. E. Jones, D. C. Stenning, E. B. Ford, R. L. Wolpert, T. J. Loredo, C. Gilbertson, and X. Dumusque.  Improving exoplanet detection power: Multivariate gaussian process models for stellar activity.  _The Annals of Applied Statistics_ , 16(2):652–679, 2022.
  * Kong et al. (2003) A. Kong, P. McCullagh, X.-L. Meng, D. Nicolae, and Z. Tan.  A theory of statistical models for Monte Carlo integration (with Discussions).  _Journal of the Royal Statistical Society: Series B (Statistical Methodology)_ , 65(3):585–604, 2003.
  * Liang (2002) F. Liang.  Dynamically weighted importance sampling in monte carlo computation.  _Journal of the American Statistical Association_ , 97(459):807–821, 2002.
  * Liang (2005) F. Liang.  A generalized Wang–Landau algorithm for Monte Carlo computation.  _Journal of the American Statistical Association_ , 100(472):1311–1327, 2005.
  * Liang et al. (2007) F. Liang, C. Liu, and R. J. Carroll.  Stochastic approximation in monte carlo computation.  _Journal of the American Statistical Association_ , 102(477):305–320, 2007.
  * Livingstone et al. (2019) S. Livingstone, M. Betancourt, S. Byrne, and M. Girolami.  On the geometric ergodicity of Hamiltonian Monte Carlo.  _Bernoulli_ , 25(4A):3109 – 3138, 2019.  10.3150/18-BEJ1083.  URL https://doi.org/10.3150/18-BEJ1083.
  * Loredo et al. (2012) T. J. Loredo, J. O. Berger, D. F. Chernoff, M. A. Clyde, and B. Liu.  Bayesian methods for analysis and adaptive scheduling of exoplanet observations.  _Statistical Methodology_ , 9(1-2):101–114, 2012\.
  * Meng and Schilling (2002) X.-L. Meng and S. Schilling.  Warp bridge sampling.  _Journal of Computational and Graphical Statistics_ , 11(3):552–586, 2002.
  * Meng and Wong (1996) X.-L. Meng and W. H. Wong.  Simulating ratios of normalizing constants via a simple identity: a theoretical exploration.  _Statistica Sinica_ , pages 831–860, 1996.
  * Neal et al. (2011) R. M. Neal et al.  Mcmc using hamiltonian dynamics.  _Handbook of markov chain monte carlo_ , 2(11):2, 2011.
  * Nelson et al. (2020) B. E. Nelson, E. B. Ford, J. Buchner, R. Cloutier, R. F. Díaz, J. P. Faria, N. C. Hara, V. M. Rajpaul, and S. Rukdee.  Quantifying the bayesian evidence for a planet in radial velocity data.  _The Astronomical Journal_ , 159(2):73, 2020.
  * Nielsen and Nock (2013) F. Nielsen and R. Nock.  On the chi square and higher-order chi distances for approximating f-divergences.  _IEEE Signal Processing Letters_ , 21(1):10–13, 2013.
  * Papp and Sherlock (2022) T. P. Papp and C. Sherlock.  A new and asymptotically optimally contracting coupling for the random walk metropolis.  _arXiv preprint arXiv:2211.12585_ , 2022.
  * Parno and Marzouk (2018) M. D. Parno and Y. M. Marzouk.  Transport map accelerated markov chain monte carlo.  _SIAM/ASA Journal on Uncertainty Quantification_ , 6(2):645–682, 2018.
  * Pompe et al. (2020) E. Pompe, C. Holmes, K. Łatuszyński, et al.  A framework for adaptive mcmc targeting multimodal distributions.  _Annals of Statistics_ , 48(5):2930–2952, 2020\.
  * Propp and Wilson (1996) J. G. Propp and D. B. Wilson.  Exact sampling with coupled markov chains and applications to statistical mechanics.  _Random Structures & Algorithms_, 9(1-2):223–252, 1996.
  * Pullen and Morris (2014) N. Pullen and R. J. Morris.  Bayesian model comparison and parameter inference in systems biology using nested sampling.  _PloS one_ , 9(2):e88419, 2014.
  * Rajpaul et al. (2015) V. Rajpaul, S. Aigrain, M. A. Osborne, S. Reece, and S. Roberts.  A gaussian process framework for modelling stellar activity signals in radial velocity data.  _Monthly Notices of the Royal Astronomical Society_ , 452(3):2269–2291, 2015.
  * Robbins and Monro (1951) H. Robbins and S. Monro.  A stochastic approximation method.  _The annals of mathematical statistics_ , pages 400–407, 1951.
  * Roberts and Rosenthal (2007) G. O. Roberts and J. S. Rosenthal.  Coupling and ergodicity of adaptive markov chain monte carlo algorithms.  _Journal of applied probability_ , 44(2):458–475, 2007.
  * Şimşekli (2017) U. Şimşekli.  Fractional langevin monte carlo: Exploring lévy driven stochastic differential equations for markov chain monte carlo.  In _International Conference on Machine Learning_ , pages 3200–3209. PMLR, 2017.
  * Stan Development Team (2019) Stan Development Team.  _Stan Modeling Language User’s Guide and Reference Manual, Version 2.28_.  2019\.  URL http://mc-stan.org/.
  * Stan Development Team (2020) Stan Development Team.  RStan: the R interface to Stan, 2020.  URL http://mc-stan.org/.  R package version 2.21.2.
  * Stein and Meng (2013) N. M. Stein and X.-L. Meng.  Practical perfect sampling using composite bounding chains: the dirichlet-multinomial model.  _Biometrika_ , 100(4):817–830, 2013.
  * Surjanovic et al. (2022) N. Surjanovic, S. Syed, A. Bouchard-Côté, and T. Campbell.  Parallel tempering with a variational reference.  _Advances in Neural Information Processing Systems_ , 35:565–577, 2022.
  * Syed et al. (2021) S. Syed, V. Romaniello, T. Campbell, and A. Bouchard-Côté.  Parallel tempering on optimized paths.  In _International Conference on Machine Learning_ , pages 10033–10042. PMLR, 2021.
  * Syed et al. (2022) S. Syed, A. Bouchard-Côté, G. Deligiannidis, and A. Doucet.  Non-reversible parallel tempering: a scalable highly parallel mcmc scheme.  _Journal of the Royal Statistical Society Series B: Statistical Methodology_ , 84(2):321–350, 2022.
  * Tak et al. (2018) H. Tak, X.-L. Meng, and D. A. van Dyk.  A repelling–attracting metropolis algorithm for multimodality.  _Journal of Computational and Graphical Statistics_ , 27(3):479–490, 2018.
  * Villani (2003) C. Villani.  _Topics in optimal transportation_.  Number 58. American Mathematical Soc., 2003.
  * Wang and Landau (2001) F. Wang and D. P. Landau.  Efficient, multiple-range random walk algorithm to calculate the density of states.  _Physical review letters_ , 86(10):2050, 2001\.
  * Wang et al. (2022) L. Wang, D. E. Jones, and X.-L. Meng.  Warp bridge sampling: The next generation.  _Journal of the American Statistical Association_ , 117(538):835–851, 2022.
  * Woodard et al. (2009) D. Woodard, S. Schmidler, and M. Huber.  Sufficient Conditions for Torpid Mixing of Parallel and Simulated Tempering.  _Electronic Journal of Probability_ , 14(none):780 – 804, 2009.  10.1214/EJP.v14-638.  URL https://doi.org/10.1214/EJP.v14-638.
  * Yan et al. (2023) Y. Yan, K. Wang, and P. Rigollet.  Learning gaussian mixtures using the wasserstein-fisher-rao gradient flow.  _arXiv preprint arXiv:2301.01766_ , 2023.
  * Yu and Meng (2011) Y. Yu and X.-L. Meng.  To center or not to center: That is not the question—an ancillarity–sufficiency interweaving strategy (asis) for boosting mcmc efficiency (with discussion).  _Journal of Computational and Graphical Statistics_ , 20(3):531–615, 2011.


# Ever-Evolving Memory by Blending and Refining the Past

Seo Hyun Kim1, Keummin Ka1,
Yohan Jo2, Seung-won Hwang2, Dongha Lee1, Jinyoung Yeo1

1Yonsei University, 2Seoul National University
1{shkimsally, kummin0429, donalee<EMAIL_ADDRESS>
2{yohan.jo<EMAIL_ADDRESS>
###### Abstract

For a human-like chatbot, constructing a long-term memory is crucial. However,
current large language models often lack this capability, leading to instances
of missing important user information or redundantly asking for the same
information, thereby diminishing conversation quality. To effectively
construct memory, it is crucial to seamlessly connect past and present
information, while also possessing the ability to forget obstructive
information. To address these challenges, we propose CREEM, a novel memory
system for long-term conversation. Improving upon existing approaches that
construct memory based solely on current sessions, CREEM blends past memories
during memory formation. Additionally, we introduce a refining process to
handle redundant or outdated information. Unlike traditional paradigms, we
view responding and memory construction as inseparable tasks. The blending
process, which creates new memories, also serves as a reasoning step for
response generation by informing the connection between past and present.
Through evaluation, we demonstrate that CREEM enhances both memory and
response qualities in multi-session personalized dialogues.

##  1 Introduction

To enhance the conversational experience with human-like chatbots, chatbots
should possess long-term memory and utilize it to engage users in
conversations. While today’s large language models (LLMs) have facilitated
interactions with humans, they often lack long-term memory. This limitation
leads to instances of forgetting important user information or redundantly
exchanging information, ultimately diminishing the quality of conversation
(Zhong et al., 2024). To bridge this gap and provide a more authentic
conversational experience, constructing and integrating long-term memory into
chatbot is imperative.

In long-term conversations, the user’s situation and preference can continue
to change as the number of sessions increases. For instance, one who was into
classical music might turn into a passionate fan of heavy metal, or one who
was suffering from severe illness could now be healthy enough to go jogging
every morning. In order to create a chatbot that is sensitive to these
changes, it is crucial that the changes are immediately reflected in its long-
term memory (Adiwardana et al., 2020).

When constructing a memory system, the approach of simply storing the
summarized dialogue or extracting persona may seem intuitive (Jang et al.,
2023; Xu et al., 2021). However, these approaches do not effectively reflect
changes in the conversation. They focus solely on the current session when
extracting information to save, resulting in a listing of isolated information
without capturing changes between sessions. Furthermore, as memory accumulates
without a refining process, outdated or contradictory information may be
intermixed (Bae et al., 2022). This hinders the retrieval of correct
information, potentially leading to inappropriate responses as shown in Case 1
of Figure 1. Therefore, any memories that become useless or obstructive should
be deliberately forgotten. While the refinement of memory is essential,
research in this area remains insufficiently explored. To address both aspects
effectively, we focus on the following question: How can we store past and new
information without causing confusion or interference between them?

To this end, we propose a novel memory framework called Contextualized
Refinement based Ever-Evolving Memory, CREEM. CREEM blends past and present
information to generate insights, which are then added as new memories.
Subsequently, refining operations are employed to forget outdated or redundant
past memories. In contrast to traditional paradigms, we do not view response
generation and memory construction as separate tasks. Our blending process,
where new insights are created, serves to link the past and present. The
insights inferred in this process are then infused to the response generation
model, assisting in properly integrating the updated information. Therefore,
the blending process serves as both a step for memory construction and
reasoning for response generation.

To demonstrate the effectiveness of CREEM, we begin by conducting memory
evaluation using G-Eval along with introducing new criteria. Additionally, we
closely examine the impact of refining by assessing the level of contradiction
using a natural language inference (NLI) model, and through questions that
address changes in content across sessions. For response generation
evaluation, we conduct both reference-based and reference-free evaluations. We
observe that even without fine-tuning, our model displayed strong performance
in memory augmented response generation.

In summary, our main contributions are as follows:

  * •

We introduce CREEM, a novel memory constructing framework to tackle the issues
of fragmented and contradictory memories.

  * •

We enhance memory adapted responding ability by infusing the blended insight
to the response generation model.

  * •

We establish new criteria for what constitutes a good memory and evaluate
based on them.

  * •

We demonstrate how the memory quality aligns with the response quality.

Figure 1: Motivating examples where our memory (Blend + Refine) is needed
compared to not refining (Case 1) and deleting all contradictories (Case 2).

##  2 Related Work

###  2.1 Long-term Memory Systems

The long-term memory system has been an ongoing subject of research. Xu et al.
(2021) extracts persona by summarizing the utterances for each speakers and
use them as a memory for generating response in ongoing conversation. They
introduce Multi-Session Chat (MSC), a dataset created by human annotators
through expanding dialogues from Persona-Chat (Zhang et al., 2018) In Jang et
al. (2023), memory is constructed by summarizing the each session in order to
remember the content of the previous session and have a conversation in the
next session. They introduce a new dataset, Conversational Chronicles (CC),
which is generated by ChatGPT given a relationship, time interval, and a
storyline employing the narratives from SODA (Kim et al., 2022). These methods
of building memory by summarizing conversations creates one-dimensional
information, which may prevent high-level conversation or conversation
appropriate to the situation. Meanwhile, our work creates new memories by
integrating relevant past memories, rather than simply summarizing the current
conversation. Research on more advanced versions of memory systems has also
been conducted. Majumder et al. (2020) expands the user’s persona with the
commonsense knowledge graph (Kim et al., 2023). In Park et al. (2023), its
generative agent conducts high-level reflections based on observations of the
world, and then add them to the memory. Motivated by this approach, we use
GPT-3.5-turbo (OpenAI, 2023) to extract insights from the current dialogue to
get high-level information.

###  2.2 Forgetting in Long-term Memory

When constructing long term memory, as sessions become longer, unnecessary or
conflicting information may accumulate. Therefore, there is a need for a
function in memory to forget information accordingly. Bae et al. (2022)
propose a long-term memory management mechanism with memory refining
operations. The operations remove or add memory by determining the
relationship between past memory and the summary created from the current
session. Among them, the ’delete’ operation deletes both memory when they are
no longer needed to be remembered and the summary contains any content that is
no longer true. Case 2 in Figure 1 shows how the deleting operation can affect
the memory capability. We implement its mechanism to compare with our method
through further evaluations. Zhong et al. (2023) introduce a memory updating
mechanism, mirroring the Ebbinghaus Forgetting Curve theory. It selectively
forgets and strengthens memories based on time elapsed and importance. However
it does not handle contradiction in memory.

Figure 2: Overall memory update process of CREEM.

##  3 Proposed Approach: CREEM

In this section, we explain the details of each step that is executed every
turn from constructing memory to generating responses. The overall method flow
is shown in Figure 2 and Algorithm 1.

###  3.1 Contextual Search

The initial step involves contextual searching to gather a robust pool of past
memories, which serve as a valuable source for blending when forming new ones.
Typically, a dialogue context is utilized as a query to retrieve pertinent
memories. We take a step further by generating a question that aligns with the
current dialogue context and using it as an additional query. This approach
enables us to retrieve a broader range of memories that are implicitly
related, extending beyond those directly resembling the ongoing dialogue. We
utilize GPT-3.5-turbo (OpenAI, 2023)(for short ChatGPT) for this process.

###  3.2 Blend and Refine

##### Blending.

When creating a new memory, past memories are taken into consideration to
ensure coherence and relevance. The Blend process uses the top-k information
that were previously retrieved to generate $n$ memories through integration
and high-level inference. Prior works that construct memory with the
information extracted from a single session can be expressed as equation 1
where $\mathrm{m}_{s}$ and $\mathbf{D}^{s}$ are new memories and dialogue at
current time $s$ respectively, and $f$ is the memory expanding function. On
the contrary, our method utilizes past memories while constructing the memory
itself as equation 2 where $\mathbf{M}_{[1,s-1]}$ is the set of past memories
accumulated.

$\displaystyle\mathrm{m}_{s}=f(\mathbf{D}^{s})$ (1)
$\displaystyle\mathrm{m}_{s}=f(\mathbf{D}^{s},\mathbf{M}_{[1,s-1]})$ (2)

##### Refining.

When adding new memory, past memory should be refined to match the overall
consistency. During the Refine step, the relationship between new and past
memories is evaluated and categorized into three types: New if it introduces
entirely novel information, Redundant if it overlaps with existing memories,
and Updated if past information has been updated. Subsequently, based on the
type of insight, appropriate actions are taken regarding past information. In
the case of Redundant or Updated, LLM identifies which past memories to
remove. After deletion, the new insights are added to the memory stream.

###  3.3 Insight Infused Response Generation

To generate responses, we provide the ongoing dialogue and the retrieved past
memories that have undergone the refining process, along with newly created
insights. The generated insights reflect changes or supplemented information
based on the current utterance, thereby encapsulating connections between past
and present. Through those insights, we enable the response generation model
to comprehend the connections that embody changes between past and present.
This ensures that responses accurately reflect the changes without any
inconsistencies.

1: Definition: $\mathbf{M}=\textit{set of entire memories}$,
$\mathbf{D}=\textit{set of dialogue utterances}$,

2: $s=\textit{session number}$, $t=\textit{turn number}$,

3: $\mathbf{M}_{t}^{s}=\textit{set of memories created at turn t of session
s}$, $\mathbf{M}_{remove}=\textit{set of memories to remove}$

4: Input: M, D , s , t

5: $query\leftarrow GenerateQuery\left(\mathbf{D}^{s}_{[1,t-1]}\right)$

6: $\mathbf{[\textit{m'},...]}\leftarrow
Retrieval\left(\mathbf{M}_{[1,s-1]}|query,\mathbf{D}^{s}_{[t-2,t-1]}\right)$

7: $\mathbf{M}_{t}^{s},type,\mathbf{M}_{remove}\leftarrow
Blend\&Refine(\mathbf{[\textit{m'},...]},\mathbf{D}^{s}_{[1,t-1]})$

8:$Response\leftarrow
GenerateResponse\left(\mathbf{M}_{t}^{s},\mathbf{[\textit{m'},...]}_{refined},\mathbf{D}^{s}_{[1,t-1]}\right)$

Algorithm 1 Memory Updating Process

##  4 Experiments

###  4.1 Datasets

We evaluate on two long-term chat datasets, Multi-Session Chat(MSC) (Xu et
al., 2021) and Conversation Chronicles (CC) (Jang et al., 2023). While MSC is
annotated by humans, CC is generated by ChatGPT. Both have 5 sessions of
dialogue in each episode with various time intervals and persona. For testing,
we use about 5 utterances per session across 50 episodes from each datasets,
yielding approximately 250 sessions and 1250 utterances for each dataset.

| $\text{SumMem}_{\text{MSC}}$ | $\textsc{ReBot}_{\text{CC}}$ | $\text{GPT}_{\text{CareCall}}$ | CREEM
---|---|---|---|---
What | present | present | present |  present + past
When | per turn | per session | per session | per turn
Model |  supervised encoder-decoder |  distilled T5-base |  ChatGPT |  ChatGPT
Fine-tuned |  $\checkmark$ |  $\checkmark$ |  $\times$ |  $\times$

Table 1: Comparison of memory constructing methods.

###  4.2 Models and Settings

We compare our memory approach with the methods introduced with the datasets
and another method reflecting the refining process of Bae et al. (2022).

##### $\text{SumMem}_{\text{MSC}}$

For the MSC dataset, they employ its own encoder-decoder abstractive
summarizer, which we will call as $\text{SumMem}_{\text{MSC}}$, to extract
persona for each speaker’s utterance.

##### $\textsc{ReBoT}_{\text{CC}}$

For the CC dataset, $\textsc{ReBoT}_{\text{CC}}$ summarizes each session’s
dialogue with a distilled T5-base model (Raffel et al., 2020), using these
summaries as long-term memory for both speakers. The T5-base model is trained
with the summaries of 80K sessions generated by ChatGPT.

We set the two baseline methods to receive the entire memory as input for
response generation, following the original $\textsc{ReBoT}_{\text{CC}}$
approach.

##### $\text{GPT}_{\text{CareCall}}$

To evaluate our refining method, we experiment with the memory management
mechanism proposed in Bae et al. (2022). Since there is no publicly available
model, we implemented each operation using ChatGPT which we call
$\text{GPT}_{\text{CareCall}}$. We implemented the methods using ChatGPT and
Contriever to replicate the same model settings with our method for a fair
comparison in the experiments. $\text{GPT}_{\text{CareCall}}$ performs
pairwise processing of the existing memory and the summary generated at the
end of the session, one by one; when memory already contains information of
summary (PASS); when memory and summary are inconsistent, or when summary
contains information of memory (REPLACE); when memory and summary are not
related (APPEND); when memory is no longer true and summary is no longer
needed to be remembered (DELETE).

##### CREEM

For CREEM, Contriever (Izacard et al., 2021) is used as the retriever to
extract 5 information pieces each for question query and dialogue context
query. Before each turn to respond, we instruct ChatGPT to generate 2 new
insights to be added to the memory. We set the temperature parameter to 0.7 in
order to create diverse insights. We provide detailed prompts in Appendix A
for reproducibility.

Table 1 provides a detailed comparison. For fair comparison regarding the
memory systems, we use ChatGPT as the response generation model for all
methods.

###  4.3 Evaluation Metrics

####  4.3.1 Memory Evaluation

Before assessing memory performance, it is essential to establish the criteria
for what defines an effective memory. A good memory should consistently offer
useful information aligned with past conversations. Similar to human memory,
it should evolve, keeping track of changes and resolving conflicts between old
and updated information. We establish three criteria for evaluating the
qualities as a next-generation memory, under the evolutionary aspects:

  * •

Integration: accurate connection and reflection of changes across sessions in
details

  * •

Consistency: absence of contradictory or conflicting information

  * •

Sophistication: advancement beyond one-dimensional, direct information

We set additional three criteria to assess the fundamental aspects of a
memory:

  * •

Relevance: closely related to the speakers

  * •

Concreteness: includes specific and precise information

  * •

Diversity: variety of topics, perspectives, or insights

We conduct G-Eval (Liu et al., 2023) customized to evaluate the memory given
each of the criteria, instruction steps, and the conversation of all sessions.

Dataset | Model | Evolutionary | Fundamental
---|---|---|---
Integration | Consistency | Sophistication | Relevance | Concreteness | Diversity
MSC | $\text{SumMem}_{\text{MSC}}$ | 3.44 | 3.95 | 3.02 | 4.09 | 3.82 | 3.72
$\textsc{ReBot}_{\text{CC}}$ | 3.86 | 4.11 | 3.52 | 4.23 | 3.97 | 4.00
$\text{GPT}_{\text{CareCall}}$ | 3.86 | 4.69 | 3.33 | 4.14 | 3.52 | 3.44
CREEM | 4.13 | 4.72 | 3.72 | 4.46 | 4.19 | 4.22
CC | $\text{SumMem}_{\text{MSC}}$ | 3.16 | 4.11 | 2.59 | 3.85 | 3.10 | 3.24
$\textsc{ReBot}_{\text{CC}}$ | 4.01 | 4.50 | 3.57 | 4.51 | 4.01 | 4.00
$\text{GPT}_{\text{CareCall}}$ | 3.88 | 4.87 | 3.67 | 4.30 | 3.59 | 3.21
CREEM | 4.17 | 4.94 | 3.83 | 4.61 | 4.19 | 4.03

Table 2: G-Eval of memory based on 6 criteria with CC and MSC dataset.

####  4.3.2 Contradiction Level Evaluation

To conduct a more comprehensive evaluation focusing on the contradictory
aspect, we assess the contradiction level by computing the average probability
of contradiction for all pairs within the memory. This evaluation is performed
using RoBERTa (Liu et al., 2019), which is fine-tuned on the MultiNLI(MNLI)
dataset (Williams et al., 2017).

####  4.3.3 Question-answering Evaluation

To conduct a more robust evaluation of the refining process, we performed a
question-and-answering(QA) based evaluation for CREEM and
$\text{GPT}_{\text{CareCall}}$.

##### Question Generation

There have been studies of evaluating abstractive summarization through QA-
based evaluation (Scialom et al., 2019; Wang et al., 2020). We create
questions specifically designed to assess the memory system’s ability to
remember altered information throughout the session. Utilizing the Chain-of-
Thoughts (Wei et al., 2022) and form-filling technique, we instruct ChatGPT to
generated multiple-choice questions. Given the entire session dialogue,
ChatGPT identifies the changed information denoted as [Before] and [After].
Then, it creats questions based on this captured information. This process
ensures that shifts in the speaker’s status to be accurately reflected within
the problem. Additionally, a ”No Answer” option is included for cases where
the question cannot be answered based on its memory. Please refer to the
Appendix B.2 for detailed prompts.

##### Answering

The quality of memory is evaluated based on how well an answering model
utilized the given memory to answer questions, measured by the accuracy rate
of its responses. We utilize ChatGPT as the answering model, providing each
memory as the sole external information.

####  4.3.4 Response Generation Evaluation

It is necessary to verify whether the quality of memory aligns with the
quality of responses. To evaluate response quality, we conduct reference
evaluations using BLEU, ROUGE-L, and BERTScore (Zhang et al., 2019; Papineni
et al., 2002; Lin, 2004). However, a high score in these metrics does not
necessarily imply that the model generated a superior response. Given the
potential diversity in responses even within the same context, deviation from
the reference response should not be interpreted as an incorrect answer.

Hence, alongside the automated evaluations, we perform a reference-free pair-
wise evaluation using ChatGPT. Since we are focusing on the memory’s impact
rather than the response generation performance itself, we established 2
criteria that can most effectively show the role of the memory:

  * •

Consistency: maintains consistency without contradictions from past sessions

  * •

Memorability: effectively recalls past events correctly by retaining
information from previous sessions

Dataset | Model | Session
---|---|---
2 | 3 | 4 | 5
MSC | $\text{SumMem}_{\text{MSC}}$ | 0.3475 | 0.3514 | 0.363 | 0.3658
$\textsc{ReBot}_{\text{CC}}$ | 0.198 | 0.2065 | 0.2157 | 0.2228
$\text{GPT}_{\text{CareCall}}$ | 0.3054 | 0.2653 | 0.2916 | 0.2678
CREEM | 0.1493 | 0.1476 | 0.1539 | 0.1613
CC | $\text{SumMem}_{\text{MSC}}$ | 0.1476 | 0.1656 | 0.1743 | 0.1775
$\textsc{ReBot}_{\text{CC}}$ | 0.1717 | 0.177 | 0.2054 | 0.2006
$\text{GPT}_{\text{CareCall}}$ | 0.1176 | 0.1036 | 0.1583 | 0.1241
CREEM | 0.11 | 0.1454 | 0.1872 | 0.1915

Table 3: Degree of Contradiction.

Figure 3: QA Accuracy Rate.

##  5 Results

###  5.1 Main Results

#####  CREEM makes high quality memory.

Table 2 displays scores of G-eval over memory quality. CREEM outperforms in
all three evolutionary aspects, indicating its superior ability to integrate,
maintain consistency, achieve sophistication. The Blend process appears to
play a crucial role in integrating information across sessions, creating
sophisticated memories by expanding the memory. Additionally, the Refine
process contributes to maintaining a consistent and well-organized memory by
continually removing redundant or outdated information. Even in fundamental
aspects, CREEM excelled in all criteria.

#####  CREEM’s refining operations are effective in maintaining memory
consistency.

Table 3 presents the degree of contradiction within the memory predicted by
the NLI model. Overall, CREEM exhibited a low level of contradiction. However,
concerning the CC dataset, $\text{GPT}_{\text{CareCall}}$ showed even lower
levels of contradiction. While CREEM merge contradictory information into one,
$\text{GPT}_{\text{CareCall}}$ remove both or one of the contradictory
information through its operations, resulting in a higher removal of
contradictory data. This approach may lead to excessive information loss, as
evidenced by QA evaluation results in Figure 3. CREEM shows higher accuracy
rates in both datasets. The lower accuracy rates for
$\text{GPT}_{\text{CareCall}}$ suggest that it may have missed significant
information during the refinement process. These results show that CREEM’s
Blend and Refine process effectively incorporates both past and future
information, maintaining better memory retention while reducing contradiction.

Model | MSC | CC
---|---|---
BLEU | ROUGE-L | BERTScore | BLEU | ROUGE-L | BERTScore
$\text{SumMem}_{\text{MSC}}$ | 1.392 | 0.143 | 0.866 | 4.053 | 0.175 | 0.876
$\textsc{ReBot}_{\text{CC}}$ | 1.416 | 0.148 | 0.863 | 3.861 | 0.195 | 0.877
$\text{GPT}_{\text{CareCall}}$ | 1.485 | 0.139 | 0.862 | 3.987 | 0.171 | 0.873
CREEM | 1.610 | 0.150 | 0.865 | 4.667 | 0.198 | 0.880
Table 4: Performance in response generation by BLEU, ROUGE-L, and BERTScore

Dataset | Criteria |  CREEM vs.
---|---|---
$\textsc{ReBoT}_{\text{CC}}$ | $\text{SumMem}_{\text{MSC}}$ | $\text{GPT}_{\text{CareCall}}$ |  $-$ Question |  $-$ Refine |  $-$ Insight Infuse
MSC | Consistency |  81.52 % |  60.09 % |  76.45 % |  62.67 % | 72.88 % | 72.37 %
Memorability |  71.01 % |  50.00 % |  68.12 % |  73.63 % | 70.34 % | 69.30 %
CC | Consistency |  76.45 % |  69.12 % |  81.29 % |  66.21 % |  59.26 % |  67.53 %
Memorability |  54.52 % |  56.87 % |  72.58 % | 49.68 % |  58.02 % | 54.32 %

Table 5: Comparison of generated response quality. We report CREEM’s winning
rate.

#####  CREEM’s memory improves response generation.

Table 4 presents the reference-based evaluation of the quality of responses
generated by each model across different datasets. Under the BERTScore metric,
the MSC data shows slightly better results with its own fine-tuned model,
$\text{SumMem}_{\text{MSC}}$. Aside from that, it shows that CREEM generates
better responses even without fine-tuning. Table 5 shows the winning rate of
CREEM. Overall, ChatGPT generates better answers when utilizing CREEM’s
memory, compared to three other memories. Particularly in terms of
consistency, ChatGPT receives better evaluations, indicating that the Refine
process effectively generates consistent responses aligned with past
conversations. We can conclude that the performance of the response generation
is positively influenced by the quality of the memory.

###  5.2 Ablation Study

##### Contextual searching using generated questions is effective for response
quality.

Table 5 shows the performance difference between CREEM and its variations,
excluding question generation and only using the current dialogue as the query
in the Contextual Search process. Except for the memorablity in CC data, the
original version of including both question and dialogue as query tends to
make better responses. This show that searching with the generated question
retrieves a wider range of information, which can contribute to creating a
better memory.

##### Refinement has a crucial effect both in memory and response quality.

Table 5 shows excluding the Refine process degrades the response quality. It
can be observed that refining leads to better memory retention and consistent
responses compared to simply accumulating all information without refinement.

##### Pre-response blending helps generate better response.

Table 5 shows the performance difference between CREEM and its variants
excluding the Insight Infusing process right before generating a response.
This setting only gives memories accumulated in the past session, while
CREEM’s original method also provides new memories created in the ongoing
session until its responding turn. Overall, CREEM consistently produces more
consistent and memory-enhanced responses.

Figure 4: Example episode of how past memory is blended and utilized. Change
in Speaker A’s hair color choice is reflected in the later sessions.

##  6 Conclusion

We create an ever-evolving memory scheme to properly integrate both present
and past memories, actively refining outdated or contradictory information.
Despite using a maximum 2-shot LLM prompting approach, our method, CREEM,
outperforms other fine-tuned methods, $\text{SumMem}_{\text{MSC}}$ and
$\textsc{ReBot}_{\text{CC}}$, in terms of memory alignment and response
generation. Even when compared with with the $\text{GPT}_{\text{CareCall}}$
which applies different refining methods within the same settings with CREEM
using ChatGPT, we have demonstrated the effectiveness of our refining method
in improving memory retention and consequently enhancing the quality of
responses. Through CREEM, we aim to create a more human-like memory system,
fostering further evolution of chat-bot memory systems.

##  7 Limitation

Our study has the following limitations: (1) Apart from the scheme of CREEM,
the results can be influenced by the performance of the retriever. As future
work, we plan to develop a retriever model specialized in memory and
conversation data; (2) During the Blend & Refine process, CREEM operates
within the top-k retrieved memories. However, it may overlook removing
insights or updated relationships when the disappearing past memories are less
than or equal to k. Currently, our method doesn’t handle unretrieved memories,
but incorporating graph relationships among memories in the future could
enhance memory retrieval and management; (3) There is not enough publicly open
high-quality long-term datasets. We expect CREEM to have an advantage over
other models as the conversations become very long through the blend and
refine process. We plan to experiment by developing a longer dataset for
better memory evaluation.

## References

  * Adiwardana et al. (2020) Daniel Adiwardana, Minh-Thang Luong, David R So, Jamie Hall, Noah Fiedel, Romal Thoppilan, Zi Yang, Apoorv Kulshreshtha, Gaurav Nemade, Yifeng Lu, et al.  Towards a human-like open-domain chatbot.  _arXiv preprint arXiv:2001.09977_ , 2020.
  * Bae et al. (2022) Sanghwan Bae, Donghyun Kwak, Soyoung Kang, Min Young Lee, Sungdong Kim, Yuin Jeong, Hyeri Kim, Sang-Woo Lee, Woomyoung Park, and Nako Sung.  Keep me updated! memory management in long-term conversations.  _arXiv preprint arXiv:2210.08750_ , 2022.
  * Izacard et al. (2021) Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bojanowski, Armand Joulin, and Edouard Grave.  Unsupervised dense information retrieval with contrastive learning.  _arXiv preprint arXiv:2112.09118_ , 2021.
  * Jang et al. (2023) Jihyoung Jang, Minseong Boo, and Hyounghun Kim.  Conversation chronicles: Towards diverse temporal and relational dynamics in multi-session conversations.  _arXiv preprint arXiv:2310.13420_ , 2023.
  * Kim et al. (2023) Donghyun Kim, Youbin Ahn, Wongyu Kim, Chanhee Lee, Kyungchan Lee, Kyong-Ho Lee, Jeonguk Kim, Donghoon Shin, and Yeonsoo Lee.  Persona expansion with commonsense knowledge for diverse and consistent response generation.  In _Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics_ , pp. 1131–1141, 2023.
  * Kim et al. (2022) Hyunwoo Kim, Jack Hessel, Liwei Jiang, Peter West, Ximing Lu, Youngjae Yu, Pei Zhou, Ronan Le Bras, Malihe Alikhani, Gunhee Kim, et al.  Soda: Million-scale dialogue distillation with social commonsense contextualization.  _arXiv preprint arXiv:2212.10465_ , 2022.
  * Lin (2004) Chin-Yew Lin.  Rouge: A package for automatic evaluation of summaries.  In _Text summarization branches out_ , pp. 74–81, 2004.
  * Liu et al. (2023) Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu.  Gpteval: Nlg evaluation using gpt-4 with better human alignment.  _arXiv preprint arXiv:2303.16634_ , 2023.
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov.  Roberta: A robustly optimized bert pretraining approach.  _arXiv preprint arXiv:1907.11692_ , 2019.
  * Majumder et al. (2020) Bodhisattwa Prasad Majumder, Harsh Jhamtani, Taylor Berg-Kirkpatrick, and Julian McAuley.  Like hiking? you probably enjoy nature: Persona-grounded dialog with commonsense expansions.  _arXiv preprint arXiv:2010.03205_ , 2020.
  * OpenAI (2023) OpenAI.  Chatgpt, 2023.  URL https://openai.com/blog/chatgpt.
  * Ouyang et al. (2022) Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al.  Training language models to follow instructions with human feedback.  _Advances in Neural Information Processing Systems_ , 35:27730–27744, 2022.
  * Papineni et al. (2002) Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu.  Bleu: a method for automatic evaluation of machine translation.  In _Proceedings of the 40th annual meeting of the Association for Computational Linguistics_ , pp. 311–318, 2002.
  * Park et al. (2023) Joon Sung Park, Joseph O’Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein.  Generative agents: Interactive simulacra of human behavior.  In _Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology_ , pp. 1–22, 2023.
  * Raffel et al. (2020) Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu.  Exploring the limits of transfer learning with a unified text-to-text transformer.  _The Journal of Machine Learning Research_ , 21(1):5485–5551, 2020.
  * Scialom et al. (2019) Thomas Scialom, Sylvain Lamprier, Benjamin Piwowarski, and Jacopo Staiano.  Answers unite! unsupervised metrics for reinforced summarization models.  _arXiv preprint arXiv:1909.01610_ , 2019.
  * Wang et al. (2020) Alex Wang, Kyunghyun Cho, and Mike Lewis.  Asking and answering questions to evaluate the factual consistency of summaries.  _arXiv preprint arXiv:2004.04228_ , 2020.
  * Wei et al. (2022) Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al.  Chain-of-thought prompting elicits reasoning in large language models.  _Advances in neural information processing systems_ , 35:24824–24837, 2022.
  * Williams et al. (2017) Adina Williams, Nikita Nangia, and Samuel R Bowman.  A broad-coverage challenge corpus for sentence understanding through inference.  _arXiv preprint arXiv:1704.05426_ , 2017.
  * Xu et al. (2021) Jing Xu, Arthur Szlam, and Jason Weston.  Beyond goldfish memory: Long-term open-domain conversation.  _arXiv preprint arXiv:2107.07567_ , 2021.
  * Zhang et al. (2018) Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston.  Personalizing dialogue agents: I have a dog, do you have pets too?  _arXiv preprint arXiv:1801.07243_ , 2018.
  * Zhang et al. (2019) Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi.  Bertscore: Evaluating text generation with bert.  _arXiv preprint arXiv:1904.09675_ , 2019.
  * Zhong et al. (2023) Wanjun Zhong, Lianghong Guo, Qiqi Gao, and Yanlin Wang.  Memorybank: Enhancing large language models with long-term memory.  _arXiv preprint arXiv:2305.10250_ , 2023.
  * Zhong et al. (2024) Wanjun Zhong, Lianghong Guo, Qiqi Gao, He Ye, and Yanlin Wang.  Memorybank: Enhancing large language models with long-term memory.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 38, pp. 19724–19731, 2024.

##  Appendix A Method Prompts

###  A.1 Query Prompt

We use ChatGPT to create an appropriate query that could be generated for all
conversations from the current session. We use a two-shot prompt for this
task, but we don’t include all examples due to the length. The prompt for this
task is as follows:
You will act as a model that generates a query relevant to the given dialogue.
Given the dialogue, what is the most salient high-level question we can answer
about speaker, focusing mainly on the last utterance?
…
Dialogue:{text}
Question about {speaker}:

###  A.2 Blend for first session Prompt

We use ChatGPT to generate two insights for the first session. For this task,
a one-shot prompt is used, and examples are omitted for length. The prompt is
as follows:
Create insights by extracting detailed information from the ¡Current
Dialogue¿.
Here is an example that can be helpful.
…
[INPUT]
¡Current Dialogue¿
{current_dialogue}

[OUTPUT]

###  A.3 Blend and Refine Prompt

We use ChatGPT to generate two insights related to insights and utterances
that have been retrieved for queries in a second or later session. For this
task, a two-shot prompt is used, and examples are omitted for length. The
prompt is as follows:
Create insights by extracting detailed information from the LAST TWO
utterences of the ¡Current Dialogue¿ and reflecting the ¡Past Memories¿.
You can make three types of insights. In one insight, include information
about only one speaker.
1\. [Updated]
If any information from the ¡Past Memories¿ is changed in the ¡Current
Dialogue¿, create an Insight reflecting the changes.
Only update the memory with the corresponding speaker’s new information.
Then, indicate which past memory has been changed to remove.
2\. [Redundant]
If ¡Current Dialogue¿ contains redundant information of the ¡Past Memories¿,
merge them including the details.
Then, indicate which past memory is redundant to remove.
3\. [New]
If there is new information that is not related to the ¡Past Memories¿, create
a new Insight.
Then, write ”None” and do not remove anything.

Here are some an examples.
…
[INPUT]
¡Past Memories¿
{retrieved_memories}

¡Current Dialogue¿:
{current_dialogue}

[OUTPUT]

###  A.4 Respond Prompt

After the Refine stage, ChatGPT generate an appropriate response to the
previous utterance based on the newly established memory. Three guidelines are
given to create an appropriate response. The prompt is as follows:
You will act as a response generator. Considering the given [Memory], continue
the given conversation and create an appropriate response for the last
utterance.
Consider the following factors when generating:
\- Do not make the response too long.
\- Do not contain any of the speaker’s name explicitly.
\- Reflect the [Memory] in the generated response.
…
[Memory of {speaker}]:
{memory_stream}:
[Conversation]:
{current_dialogue}: [Response] {speaker}:

##  Appendix B Evaluation Prompts

###  B.1 Memory Evaluation Prompt

We compare which of the memories created by $\text{SumMem}_{\text{MSC}}$,
$\textsc{ReBoT}_{\text{CC}}$, $\text{GPT}_{\text{CareCall}}$, and CREEM had
better performance through ChatGPT using the CC dataset and the MSC dataset.
There are a total of 6 criteria used in this evaluation, and each prompt
contains different evaluation steps for each criteria so that we can evaluate
which memory better reflects the conversation by looking at the total dialogue
and the memory created for the dialogue. Considering that the results are
slightly different each time ChatGPT generates, a total of 20 results are
generated at a time and the average is taken as the result. The evaluation
step is omitted due to length. The seven criteria are as follows:

  * •

Integration (1-5): This criterion evaluates whether the memory effectively
connects information across sessions and accurately reflects changes in
details about {speaker} over time. The goal is to prioritize memories that
present a cohesive organization of content across sessions.

  * •

Consistency (1-5): This criterion evaluates whether there are any
contradictory or conflicting pieces of information about {speaker} within the
memory. The goal is to ensure that the information presented about {speaker}
remains coherent and free from contradictions throughout the conversation.

  * •

Sophistication (1-5): This criterion assesses whether the content of the
memory is advanced and goes beyond one-dimensional, direct information about
{speaker} from the conversation.

  * •

Relevance (1-5): This criterion evaluates whether the memory is closely
related to the {speaker} of the conversation. It ensures that the content
contributes meaningfully to the ongoing conversation.

  * •

Concreteness (1-5): This criterion evaluates whether the memory contains
substantial and specific information about {speaker}. It assesses the level of
detail and precision in the representation of {speaker}.

  * •

Diversity (1-5): This criterion assesses whether the content of the memory has
diverse information about {speaker}.

###  B.2 Question Generating Prompt

We use ChatGPT to view the entire conversation session and create multiple-
choice questions that captured the changes. The prompts used to create the
problem are as follows, and examples of prompts are omitted for length: Your
task is to create three questions multiple choices that can be answered from
the given dialogue. Actively reference the contents of the dialogue in the
options, but try to use different expressions.
You will make several questions asking only about the target speaker. The
subject of the question should be the target speaker.
Focus on the changed status of the target speaker and use that as a source for
making questions.
To make the question difficult, you should follow these steps:
1\. Find any information that has changed about the target speaker during the
conversation.
The format should be: ”[Before] past information [After] newly updated
information”
2\. Generate questions that asks about the changed information.
3\. Next step is to make multiple answer options that is confusing. Make a
correct choice from the newly updated information mentioned after the [After]
token, and wrong choices from past outdated information mentioned after the
[Before] token.
4\. Write the correct answer with a brief explanation.

Here is an example question.
…
Now generate 3 questions about the target speaker, {target_speaker}.
Target Speaker: {target_speaker}
[Dialogue]
{dialogue}
[Information about {target_speaker}]

###  B.3 Question Answering Prompt

We use ChatGPT to solve the problems created through the above prompt through
the memories of each model. The prompt to solve the problem using memories is
as follows: Based on the given memory, solve the three questions by choosing
one of the options to answer the following question. Also choose the sentences
that support your answer.
Make sure to answer based on ONLY the given memory, without prior knowledge.
The answer might not be directly addressed in the memory so try hard to infer
the most reasonable answer among choices A, B, C. But if you still cannot find
any answers using the memory, choose ”D) No answer”.

The answer format should be:

[Answers]:
Answer 1:
A) She is a very active and lively.

Supporting sentences:
Sentence 2) She has three kids and her mom is pediatrician.
Sentence 5) She commutes by boat every day to work and lives in amsterdam with
canals.

Now solve the questions.

[Memory]:

{memory}

[Questions about Co-worker B]

{questions}

[Answers]:

###  B.4 Response Evaluation Prompt

We use ChatGPT to pairwise compare the responses generated by CREEM with the
responses generated by $\text{SumMem}_{\text{MSC}}$,
$\textsc{ReBoT}_{\text{CC}}$ and $\text{GPT}_{\text{CareCall}}$. In conducting
this work, two criteria are used: consistency and memorability. An example of
the format is given and the criteria were specified. The criteria according to
the evaluation criteria are as follows.

  * •

Consistency: The response should maintain consistency without contradictions
from past sessions. The response aligns seamlessly with what has been
discussed in previous sessions, following the individual’s characteristic and
current status.

  * •

Memorability: The response should effectively recalls past events correctly by
retaining information from previous sessions. The response actively uses
memory from the past dialogue to continue the conversation.

Considering that the results are slightly different each time ChatGPT
generates, a total of 19 results were generated at once, and the one selected
with more was taken as the result. Among the two prompts, the following is one
that evaluates consistency. Format examples have been omitted due to length:
You will be given a conversation between two individuals. You will then be
given two response options for the next turn in the conversation.
Your task is to choose the better response based on one metric with a brief
explanation.
Please make sure you read and understand these instructions carefully. Please
keep this document open while reviewing, and refer to it as needed.

Evaluation Criteria:

Consistency - The response should maintain consistency without contradictions
from past sessions. The response aligns seamlessly with what has been
discussed in previous sessions, following the individual’s characteristic and
current status.
…
Now choose the response that has better consistency given the past and current
session dialogue.

Past Sessions:
{past_dialogue}

Current Session:
{current_dialogue}

Response Options:
{response}

Better Reponse:

##  Appendix C Examples of Memory construction and Response generation

Table 7 shows an example of building a memory and generating an answer using
the methods of $\text{SumMem}_{\text{MSC}}$, $\textsc{ReBoT}_{\text{CC}}$, and
CREEM for one episode from a random MSC dataset.

| Redundant | New | Updated
---|---|---|---
MSC dataset | 24.61 % | 55.81 % | 19.59 %
CC dataset | 30.35 % | 53.06 % | 16.59 %

Table 6: Ratio of Redundant, New, Updated during constructing memory by CREEM

Dialogue Context (msc-episode-40-s4-t5)
---
Speaker A: Well, my hair has been dyed!
Speaker B: How did it come out? I bet your grandkids were thrilled.lol
Speaker A: Oh my youngest did so good! You can tell that being a
cosmetologist is her passion, she was an absolute natural with keeping my
nerves at bay.
Speaker B: At least it’s less permanent than a drastic hair cut would be.
You’ve got extra flavor now. Did you stick with just a streak or did you go
bigger?
Speaker A: We decided to do a streak on each side, for balance. And they’re
not just out in the open but under a layer of hair, so it pops out when I wear
a ponytail. My granddaughter suggested that, how creative!
$\text{SumMem}_{\text{MSC}}$ |
Memory | I am excited for Easter Weekend. I have blue hair. I have a new Easter dress.
| I am 29 years old. I am going on spring break. My mom just gave me a new
car. I drive.
| I dyed my hair with henna. I love grey hair. I want to dye my hair grey. My
dress is turquoise.
| My hair is blue. I just got my first new car. I’ve been driving pre owned
cars before.
| I have red hair. I have a friend. I have a VW Tiguan. I fill up my car once
a month.
| I don’t drive a lot. I work as a librarian at the beach.
Response | That sounds like a fun and unique way to add some color to your hair!
| It’s great that your granddaughter was able to offer a creative suggestion.
I’m sure it looks amazing
| when you wear a ponytail.
$\textsc{ReBoT}_{\text{CC}}$ |
Memory | A retired librarian plans to leave for spring break due to her mother’s new car
| and quiet nature. She enjoys nature, but is retired due to retiring.
| A woman dyes her hair red with henna, which is good for red hair. She also
suggests
| doing a semi-permanent dye and having fun with her grandkids if allowed .
| Turquoise is her favorite color, but her new dress is turquoise.
| A grandkid dyed her hair turquoise, which she thinks will look great on her
youngest
| grandson’s favorite color. They plan to touch up their hair for the summer
months due to
| the sun and pool. They also have recommendations for a day visit at the
beach.”
Response | That sounds like a fun and unique way to add some color to your hair! I’m glad your
| granddaughter had such a creative idea. It must be exciting to have a little
pop of
| color that shows when you wear a ponytail.
$\text{GPT}_{\text{CareCall}}$ |
Memory | Speaker A has asked their youngest grandkid to help them dye their hair turquoise,
| as they want to get into hairdressing as a profession.
| Speaker A jokes that nature will eventually give Speaker B grey hair.
| Speaker A recommended the book B̈eautiful Ruinsb̈y Jess Walter to Speaker B
| which is a fiction book about Hollywood and actors/actresses.”
| Speaker B recently bought a VW Tiguan in a red color.
Response | And speaking of hair, I remembered you just got that red VW Tiguan.
| So now we both have a pop of color in our lives!
CREEM |
Memory | Speaker B is open to experimenting with different hair colors and is specifically interested
| in dyeing their hair grey in the future. Speaker B highly values the
opinions and
| expertise of their grandkids when it comes to hair dyeing decisions.
Response | Your grandkids really have a good eye for hair styling! I remember you mentioning that
| you were interested in dyeing your hair grey in the future. Maybe you can
consult
| them for some ideas when the time comes.

Table 7: Comparison of memory and response generation between CREEM,
$\text{SumMem}_{\text{MSC}}$, $\textsc{ReBoT}_{\text{CC}}$ for MSC dataset

##  Appendix D Analysis of Refine

Table 6 shows the extent to which Redundant, New, and Updated processes occur
when creating memory with CREEM through 50 MSC datasets and 50 CC datasets.
When using the MSC dataset, the rate of Redundant and Updated occurrences is
about 44.2 %, and when using the CC dataset, it is about 46.94 %. Thus, we
confirm that the process of checking and refining whether there are redundant
or updated memories among past memories was performed properly.

##  Appendix E Implementation details

###  E.1 Contriever

In our implementation into CREEM, We use Contriever Izacard et al. (2021) to
retrieve relevant past insights or utterances for queries created after the
previous utterance. Contriever, trained through unsupervised contrastive
learning, serves as a dense information retriever. Remarkably, even in the
absence of explicit supervision, it exhibits outstanding proficiency in
information retrieval tasks.

###  E.2 Large Language Model

In this study, we use ChatGPT for implementing Query Generation, Blend and
Refine and Response Generation. ChatGPT, a Large Language Model (LLM) with 175
billion parameters, is built upon Instruct-GPT Ouyang et al. (2022). It is
specifically trained to adhere to user instructions and provide requested
information in a conversational style. To interact with the OpenAI API, we
utilize LangChain.

##  Appendix F License and Terms

For our implementation and evaluation, we use Huggingface
library111https://huggingface.co/. This library licensed under Apache License,
Version 2.0. We have confirmed that all of the artifacts used in this paper
are available for non-commercial scientific use.