# RF Properties and Their Variations in a 3D printed Klystron Circuit and
Cavities

Charlotte Wehner, Julian Merrick, Bradley Shirley, Brandon Weatherford,
Garrett Mathesen, and Emilio Nanni This research has been supported by the
U.S. Department of Energy (DOE) under Contract No. DE-AC02-76SF00515.The
authors are with the Technology Innovation Directorate, SLAC National
Accelerator Laboratory, Menlo Park, CA, 94025 USA

###### Abstract

Presently, the manufacturing of active RF devices like klystrons is dominated
by expensive and time consuming cycles of machining and brazing. In this
article we characterize the RF properties of X-band klsytron cavities and an
integrated circuit manufactured with a novel additive manufacturing process.
Parts are 3D printed in 316L stainless steel with direct metal laser
sintering, electroplated in copper, and brazed in one simple braze cycle.
Standalone test cavities and integrated circuit cavities were measured
throughout the manufacturing process. Un-tuned cavity frequency varies by less
than 5% of intended frequency, and Q factors reach above 1200. A tuning study
was performed, and unoptimized tuning pins achieved a tuning range of 138 MHz
without compromising Q. Klystron system performance was simulated with as-
built cavity parameters and realistic tuning. Together, these results show
promise that this process can be used to cheaply and quickly manufacture a new
generation of highly integrated high power vacuum devices.

###### Index Terms:

3D-printing, additive manufacturing, direct metal laser sintering (DMLS),
klystron, X-band

##  I Introduction

Additive manufacturing (AM) continues to see growing interest for the
manufacturing and development of RF structures and components. Once confined
to prototyping, this family of manufacturing techniques is increasingly used
to produce devices previously impossible or impractical to manufacture [1].
Many different 3D printing technologies are being explored for RF
applications. Common technologies fall into 4 categories.

Figure 1: Diagram of DMLS printer.

Fused Deposition Modeling (FDM), or Fused Filament Fabrication (FFF), involves
extruding a thermoplastic through a heated nozzle. The nozzle is precisely
moved though the build volume to deposit rapidly solidifying plastic onto the
previous layer. While invaluable for rapid prototyping or small scale
production due to the very low cost of many FDM 3D printers, the parts this
process produces are usually weaker and limited to specific thermoplastics
[2]. FDM has been used to manufacture a C-band microwave isolator with a non-
printed ferrite bead [3]. However, the limited material selection and poor
resolution and accuracy largely prevent FDM from use in the manufacture of RF
components.

Stereolithography (SLA), sometimes known as vat polymerization, uses an LCD
and backlight or laser with a galvanometer to selectively expose a thin layer
of photosensitive resin. The part is withdrawn from the vat of photoresist as
the next layer is exposed through the bottom of the vat. While also limited to
certain plastics, this technology produces strong, accurate, and high-
resolution parts while remaining very cost effective [4]. Due to these
advantages, significant previous work has demonstrated manufacturing RF
devices with SLA. Metal-plated resin waveguides and antennas have been
rigorously studied and demonstrate promise for higher frequency devices [5,
6].

Figure 2: Complete klystron circuit with input and output waveguides, pump-out
tubes, mounting for an electron gun and collector, and cavity tuning pins.
Figure 3: Diagram of conventional klystron circuit manufacturing process

Binder jetting uses a liquid binder to selectively bind together powdered
material. Binder is dispensed onto the powder bed by an inkjet to form a 2D
layer. Another layer of powder is then deposited. Binder jetting can use a
wide variety of powdered materials like metals, ceramics, or polymers.
Usually, the binder material is a polymer resin. After printing, the parts
often must be post-processed, usually involving baking out the binder and
sintering [7]. Binder jetting can produce large, cost effective parts, but so
far little work has been done using binder jetting for RF components.

Direct Metal Laser Sintering (DMLS) involves using a high powered laser to
selectively sinter metal powder. Many similar technologies like selective
laser sintering (SLS) and selective laser melting (SLM) exist. A schematic is
presented in Fig. 1. Typically, a laser is directed across the surface of a
bed of metal powder. This selectively sinters the powder, forming a layer and
adhering it to the previously sintered layer below. Then, a roller deposits
another layer of powder onto the bed and the process repeats. Once complete,
excess powder must be removed from the part. Often, post-processing techniques
like hot isostatic pressing or bead-blasting are performed to improve the
mechanical properties and surface quality [8]. Since DMLS can produce metal
parts with greater than 99% density in a variety of materials and alloys [9],
it has seen significant use for both prototyping and full-scale production in
many industries, including for RF components. For example, waveguides,
filters, Tees, and antennas have been demonstrated across many frequencies
[10-13].

In this article, we present the motivation and process for manufacturing a
klystron circuit using DMLS. We also characterize the RF properties of
cavities produced with this method. A complete klystron circuit (shown in Fig.
2) and 20 test cavities were manufactured using this process. The RF
properties of the test cavities and klystron circuit cavities were measured at
multiple stages of the manufacturing process and compared to simulated
properties. Also, a tuning study was conducted, and the klystron system
performance was simulated with realistic cavity tunings.

##  II Motivation

A diagram depicting the conventional manufacturing process of a klystron
circuit is shown in Fig. 3. Due to the limitations of conventional subtractive
manufacturing techniques like CNC Machining, complex structures like reentrant
cavities must be split into smaller, simpler parts. These parts must be
machined, polished, and cleaned separately before being brazed into
subassemblies. These subassemblies may require additional machining and post
processing steps. Next, larger subassemblies are brazed together, followed by
additional brazing stages to add additional components like supports, tuning
pins, and pumpout tubes. Complex assemblies require many stages of machining
and brazing, requiring many hours by skilled technicians and effective
coordination between manufacturing departments. This labor-intensive process
contributes significantly to the high manufacturing cost of such an RF system.
Additionally, every braze joint added to the circuit presents an opportunity
for vacuum leaks or water leaks from cooling systems. By offloading complexity
of manufacturing to automated 3D printing systems, the effective complexity
and therefore cost of a system could be significantly reduced. Understanding
the RF properties of AM cavities and the properties’ variations is vital to
validating this process.

##  III Manufactured Device and Process

Figure 4: Diagram of proposed manufacturing process. (a) Printing, (b)
Plating, (c) Face Machining, (d) Brazing

The proposed manufacturing process leverages the ability of 3D printing to
produce extremely detailed, complex parts that would be either completely
impossible or prohibitively expensive with conventional methods. The process
for a single cavity is depicted in Fig. 4. In DMLS, the cost is driven by the
volume of printed material as well as the overall dimensions of the part.
Provided that the designer understands the geometric limitations of DMLS, like
overhangs and curling, complexity is no longer a limitation. This allows many
cavities and other structures to be combined into only two parts. Printing the
entire structure as a single part is possible, though impractical due to a
reduction of the surface quality of overhanging regions [14] and shadowing
effects during electropolishing and plating.

Presently, commercially available DMLS materials do not have favorable
properties for directly printing RF cavities [15]. To achieve desirable
material electrical properties, the parts are printed in 316 stainless steel,
then electroplated with copper. The thickness of the copper plating is not
crucial, provided that it sufficiently exceeds the skin depth at the target
frequency. DMLS exhibits significantly higher surface roughness than
conventional milling, so the parts are electropolished before Cu plating.

The native surface of the DMLS parts is too rough for brazing with thin,
laser-cut shims, so the braze surfaces are first faced flat via conventional
milling. Though this is a manual machining process, it consists of only one
simple operation per part. Laser-cut 0.002” thick shims of a 25%/75% Au/Cu
braze alloy are placed between the two flat surfaces of the part. Printed-in
holes provide alignment between the two halves. The assembly is held together
with stiff wire and brazed in a hydrogen braze furnace. Again, though this is
a manual process, it is considerably simpler than the brazing in conventional
manufacturing since it only includes two complex parts with well-defined
alignment features. Also, reducing the number of braze joints decreases the
likelihood of leaks.

Figure 5: Complete brazed test block containing 10 cavities.

Two sets of ten test cavities (shown in Fig. 5) were manufactured. These test
cavities all had the same design geometry, identical to cavity 2 of the full
circuit. In addition to the test cavities, an existing X-band klystron circuit
design currently in development at SLAC has been adapted for additive
manufacturing. The circuit is designed to output 300 kW at 11.424 GHz. The
final device shown in Fig. 6 contains four cavities approximately 20 mm in
diameter, the beampipe, input and output waveguides, mounting features for a
collector and electron gun, as well as integrated tuning pins.

Figure 6: Diagram of features included in klystron circuit. (a) Unbrazed half-
piece. (b) Full, brazed circuit.

##  IV Simulation and Characterization

An eigenmode simulation using ANSYS HFSS was performed to establish the
expected cavity performance. The cavity cross-section and E-field is shown in
Fig. 7. The expected values for cavity Q and frequency are tabulated in Table
1. Additionally, the entire klystron is simulated using CERN’s KlyC code, a
1.5-D large signal simulation code for klystrons to establish expected
performance of the system [16].

Figure 7: Normalized E-field of cavity 3 simulated in HFSS TABLE I: Cavity Resonance and Design Properties Cavity | 1 | 2 | 3 | 4
---|---|---|---|---
Frequency (GHz) | 11.424 | 11.44 | 11.495 | 11.41
Quality Factor | 200 | 2000 | 2000 | 125

Cavities were measured using two E-field probes on an Agilent N5241A network
analyzer. Each probe was positioned near the irises of each successive cavity
using manual slide stages. The peak resonance of each cavity was determined by
inserting both probes completely into each cavity, then slowly removing them
while monitoring the reflection and transmission coefficient curves. Once it
was obvious that removing each probe further did not affect the resonant
frequency (i.e., the cavities are not being detuned) a measurement was taken.
Since the metal body of the probes extended through the entire beampipe,
except in the cavity under test, other cavities are shorted. A waveguide load
was placed on both the input and output waveguides during testing.

Every cavity of the full klystron circuit was measured both before and after
brazing. The Q factor and frequency was calculated from the peak in the S12
transmission coefficient. To characterize the variation inherent in every
manufacturing step, the 20 test cavities were characterized immediately after
printing, after electropolishing, after plating, after face machining, and
after brazing.

##  V Test Cavity Results

The as-printed resonance for all 20 test cavities is shown in Fig. 8 and Fig.
9. Note that the simulated frequency and Q accounts for the additional
material which is to be removed during the face-machining step. There is a
significant spread in both frequency and Q, however, observe that Q factor is
uniformly lower, and frequency uniformly higher than the design intent. These
trends, once characterized, would allow the designer to compensate by
designing the cavities to have a higher Q factor and lower frequency than
intended.

Figure 8: Frequency of all test cavities measured after printing. Simulated
frequency is shown as horizontal grey line. Figure 9: Q factor of all test
cavities measured after printing. Simulated Q factor is shown as horizontal
grey line. TABLE II: Test Cavity Properties Throughout Manufacturing Process

| As Printed | Polished | Plated |  Face Machined*
---|---|---|---|---
Mean Frequency (GHz) | 11.178 | 11.378 | 11.380 |  11.565
Frequency (GHz) | 0.115 | 0.181 | 0.125 |  0.107
Mean Q Factor | 246 | 313 | 1961 |  1317
Q Factor CV | 0.228 | 0.265 | 0.243 |  0.48

* Data only from 10 test cavities. Other data from all 20.

Figure 10: Transmission coefficient of 10 test cavities measured after
brazing.

The cavity parameters before and after each major manufacturing step are shown
in Table 2. After polishing, the frequency and frequency standard deviation
rise. The Q also rises, but this is expected given that the lowered surface
roughness would contribute to a lower effective surface resistance. After
plating, Q rises significantly since the cavity surface is copper. Standard
deviation for both frequency and Q decreases. After machining, the frequency
rises significantly as the cavity dimensions are brought to their final
values. The Q factor decreases, and the Q factor standard deviation rises
significantly, though this is not a direct comparison as the face machining
data is only from ten out of twenty test cavities.

The final brazed cavity resonance of ten cavities is shown in Fig. 10. There
is significant variation in both frequency and Q factor. However, most of the
cavities have a frequency within 150 MHz of each other. During the printing
process, the 3D model provided to the printer must be sliced into layers, then
converted into a series of instructions for the printer to execute to create
the part. Combined with phenomena inherent to the physical process of
sintering powder, these factors mean that the actual printed part geometry is
not identical to the model provided to the printer. Two fundamental types of
variations are identified.

Firstly, there are consistent variations which similarly affect all parts. For
example, it has been shown that small holes in DMLS parts are consistently
printed smaller than the model [14]. These variation work to offset the mean
frequency and mean quality factor from the theoretical values. Once these
variations are characterized, the designer can compensate. For example, the
mean frequency of the final cavities as shown in Table 2 is 141 MHz higher
than the theoretical frequency. To compensate, the designer could target a
frequency of 11.283 GHz. Assuming the offset remains consistent, the actual
mean frequency of those cavities would be much closer to desired frequency of
11.424 GHz.

Secondly, inconsistent variations which may differently affect each
theoretically identical part. These inconsistent errors contribute to the
imprecision of the part and work to increase the standard deviation observed
in the cavities’ frequency and Q factor. Since these variations are inherently
not predictable, they provide a more difficult challenge to creating accurate
cavities. Work must go into both limiting these variations and compensating
for them via tuning.

##  VI Tuning Study

Tuning conventionally-manufactured copper cavities via mechanical deformation
is relatively well-understood. However it is expected that the different
mechanical properties of stainless steel, as well as the single-piece design
of print-in-place tuning pins, will behave differently during tuning. To
quantify this behavior, a preliminary tuning study was performed by impacting
one of the completed, brazed test cavities multiple times. Between each
impact, the cavity resonance was measured using the standard technique. The
results of this study are shown in Fig. 11. Through 5 rounds of tuning, the
cavity frequency was raised more than 150 MHz. These traces also indicate that
the Q factor was compromised.

Figure 11: Transmission coefficient of test cavity measured after successive
rounds of tuning.

Additionally, as can be seen in Fig. 12, the tuning pin is significantly
deformed. In a different tuning test, the pin broke completely through the
cavity wall, as shown in Fig. 13.

Figure 12: Deformed tuning pin after tuning test Figure 13: Damaged cavity
after failed tuning test. (a) Tuning pin head on exterior of cavity. Undamaged
pin is shown on the left. (b) Interior of damaged cavity with pin protruding.
Undamaged cavity is shown on the left. TABLE III: Test Cavity Tuning Results

| Before | After |  |
---|---|---|---|---
Cavity | Frequency (GHz) | Q | Frequency (GHz) | Q | Frequency Change (MHz) | Deformation (µm)
1 | 11.671 | 1284 | 11.778 | 1220 | 107 | 516
2 | 11.897 | 1250 | 11.84 | 1275 | -57 | -347
3 | 11.725 | 1407 | 11.819 | 1195 | 94 | 511
4 | 11.871 | 1180 | 11.988 | 1095 | 117 | 438
5 | 11.608 | 1195 | 11.575 | 1197 | -33 | -297
6 | 11.49 | 967 | 11.457 | 962 | -33 | -320
7 | 11.583 | 566 | 12.029 | 1101 | 446* | 648
8 | 11.655 | 906 | 11.619 | 1030 | -36 | -350
| Average increase (MHz): | 106 | Average Decrease (MHz): | -32

* Excluded from average due to questionably large shift

An additional tuning study was conducted on 8 of the test cavities. The
results are presented in Table 3. For this study, a tuning tool was
fabricated. The tool grasps the head of each tuning pin and can smoothly push
or pull the pin with a screw. Four cavities each were tuned upwards and
downwards to the maximum extent permitted by the tuning tool. The resonance of
each cavity was measured before and after tuning. After tuning, the cavities
were cut open and the cavity wall acted on by the tuning pin was measured on a
laser confocal microscope. The deformation in the cavity wall caused by tuning
was measured. These results establish an expected tuning bandwidth of 138 MHz
with current tuning structures. As expected, the distance the cavity wall is
deformed correlates to the frequency shift. Further, Q factor does not appear
to be compromised through tuning, with some cavities even increasing in
quality factor after tuning. Together, these results suggest that even with
current tuning pin geometry, any cavity frequency spread less than 138 MHz
will allow cavities to be tuned to an exact frequency. Note that these tuning
structures were completely unoptimized, and by affecting a larger portion of
the cavity wall and therefore displacing more cavity volume, it is likely that
significantly more effective tuning structures can be designed.

##  VII Klystron Circuit Results

The klystron circuit underwent a helium leak check which showed a good braze
joint with no leaks. Further, a DMLS sample was placed in vacuum to assess the
material’s suitability under vacuum. No significant outgassing was observed.

Figure 14: Measured transmission coefficients of cavities 3 and 4 measured
after brazing.

The measured resonant peaks for circuit cavities 3 and 4 are shown in Fig. 14.
Both cavities show significant resonance. Cavity 3 is very clear and shows
relatively high Q. Cavity 4 is less clear, likely due to coupling into the
connected output waveguide. Determining the peak on cavity 4 was difficult due
to the proximity to the end of the beampipe. The resonance of all circuit
cavities both before and after brazing are shown in Fig. 15 and 16. There is
significant variation both between the cavities and between each cavity and
its intended frequency. Cavity 2 exhibits the largest difference between
intended and actual frequency at 4.6%.

Figure 15: Resonant frequency of klystron circuit cavities as simulated,
before brazing, and after brazing.  Figure 16: Quality factor of klystron
circuit cavities as simulated, before brazing, and after brazing.

The measured Q factor shows significant promise. First, the final post-braze
resonance is very nearly half of the design Q for every cavity. Second, with a
Q factor of around 1000, cavities 2 and 3 demonstrate that this manufacturing
process can indeed create cavities with Q factors on the order necessary for
klystrons.

To inform further testing and determine expected performance, the klystron was
simulated, and an optimization was performed. Two potential tuning solutions
are presented. In both cases, the operational frequency of the klystron has
been changed from 11.424 GHz. Allowing flexibility in this parameter enabled
considerably higher output power for this test device. In a production
klystron, the designer could target a specific operational frequency by
offsetting the designed frequency of the cavities. In these simulations, it is
assumed that tuning does not affect the Q factor.

TABLE IV: Klystron Tuning Solution 1 Cavity | Frequency (GHz) | Tuning Needed (MHz)
---|---|---
1 | 11.955 | 369
2 | 11.971 | 0
3 | 12.000 | 370
4 | 11.941 | 542

Solution 1 is shown in Table 4. Due to the large frequency offset of cavity 2,
the rest of the cavities must be tuned by nearly 400 MHz, outside the
established tuning range. This would require refined tuning structures which
permit a larger tuning range. However, this solution allows an output power of
257 kW at a frequency of 11.955 GHz.

TABLE V: Klystron Tuning Solution 2 Cavity | Frequency (GHz) | Tuning Needed (MHz)
---|---|---
1 | 11.620 | 34
2 | 11.971 | 0
3 | 11.630 | 0
4 | 11.620 | 131

Solution 2 is shown in Table 5. In this solution, cavity 2 is untuned, so the
large frequency offset means the cavity does not significantly participate in
the klystron operation. Because of this, the remaining cavities need to be
tuned significantly less, even less than the tuning range achievable with
current tuning structures. However, effectively removing cavity 2 reduces the
power output to 55 kW at 11.620 GHz.

##  VIII Discussion

Although early in development, the measured RF properties and klystron
simulations show promise that the manufactured klystron circuit can operate as
a viable RF source. Additionally, the properties of the RF cavities indicate
that only incremental progress is necessary to create consistent and accurate
AM cavities as part of a tightly integrated and cost-effective device.
However, further testing is necessary to prove vacuum tubes manufactured with
this process can be functional.

The manufactured device should be pumped down to determine whether the
structure can achieve a low enough base pressure. The slightly porous
structure of the DMLS stainless steel as well as brazing in a hydrogen
atmosphere may introduce additional outgassing and adsorption.

Due to the observed inconsistencies, further investigation and reduction of
the large variations present in cavity frequency and quality is critical to
the successes of this manufacturing process. These inconsistencies likely
arise through multiple phenomena.

Firstly, the surface roughness introduces variation into the surface
resistance of cavity walls, as well as small-scale geometry which could impact
RF behavior. Multiple routes for further development exist. DMLS was carried
out by a commercial vendor and was likely optimized for acceptable and
repeatable performance across many different metrics relevant to many
different industries. Refining DMLS process parameters specifically for this
application could yield lower-roughness parts more suitable for RF structures.
Additionally, post-processing steps like electropolishing or bead blasting
could be investigated in more detail to reduce roughness after printing.

Second, DMLS has been shown to introduce bulk geometry errors, such as small
holes being smaller than expected. Further study should carefully measure the
as-printed geometry and characterize these errors. It would be especially
valuable to know if and how the designer could compensate for these errors
before printing.

With only incremental improvement in the variation of RF properties and tuning
bandwidth, X-band AM cavities can be cost-effectively produced to target a
specific frequency and Q factor. AM provides the flexibility to quickly
achieve this incremental improvement and integrate the resulting cavities into
the next generation of klystrons and other high power X-band devices.

## References

  * [1] B. Zhang, Y.-X. Guo, H. Zirath, and Y. P. Zhang, “Investigation on 3-D-printing technologies for millimeter- wave and terahertz applications,” Proceedings of the IEEE, vol. 105, no. 4, pp. 723–736, 2017.
  * [2] D. Popescu, A. Zapciu, C. Amza, F. Baciu, and R. Marinescu, “FDM process parameters influence over the mechanical properties of polymer specimens: A Review,” Polymer Testing, vol. 69, pp. 157–166, 2018.
  * [3] V. Laur, J. P. Gouavogui, and B. Balde, “c-band hybrid 3-D-printed microwave isolator,” IEEE Transactions on Microwave Theory and Techniques, vol. 69, no. 3, pp. 1579–1585, 2021.
  * [4] J. Huang, Q. Qin, and J. Wang, “A review of stereolithography: Processes and systems,” Processes, vol. 8, no. 9, p. 1138, 2020.
  * [5] S. Verploegh, M. Coffey, E. Grossman, and Z. Popovic, “Properties of 50–110-GHz waveguide components fabricated by metal additive manufacturing,” IEEE Transactions on Microwave Theory and Techniques, vol. 65, no. 12, pp. 5144–5153, 2017.
  * [6] E. Geterud, P. Bergmark, and J. Yang, “7th European Conference on Antennas and Propagation,” in Lightweight Waveguide and Antenna Components Using Plating on Plastics, 2013, pp. 1812–1815.
  * [7] M. Ziaee and N. B. Crane, “Binder jetting: A review of process, materials, and methods,” Additive Manufacturing, vol. 28, pp. 781–801, 2019.
  * [8] M. Shellabear and O. Nyrhilla, “DMLS – DEVELOPMENT HISTORY AND STATE OF THE ART,” LANE 2004 Conference, Sep. 2004.
  * [9] P. Bajaj, A. Hariharan, A. Kini, P. Kürnsteiner, D. Raabe, and E. A. Jägle, “Steels in Additive Manufacturing: A review of their microstructure and properties,” Materials Science and Engineering: A, vol. 772, p. 138633, 2020
  * [10] B. Zhang, Z. Zhan, Y. Cao, H. Gulan, P. Linner, J. Sun, T. Zwick, and H. Zirath, “Metallic 3-D printed antennas for millimeter- and submillimeter wave applications,” IEEE Transactions on Terahertz Science and Technology, vol. 6, no. 4, pp. 592–600, 2016.
  * [11] D. Miek, S. Simmich, F. Kamrath, and M. Hoft, “Additive manufacturing of E-plane cut dual-mode X-band waveguide filters with mixed topologies,” IEEE Transactions on Microwave Theory and Techniques, vol. 68, no. 6, pp. 2097–2107, 2020.
  * [12] J. Wu, C. Wang, and Y. Guo, “Ridged waveguide magic tees based on 3-D printing technology,” IEEE Transactions on Microwave Theory and Techniques, vol. 68, no. 10, pp. 4267–4275, 2020.
  * [13] S. Verploegh, M. Coffey, E. Grossman, and Z. Popovic, “Properties of 50–110-GHz waveguide components fabricated by metal additive manufacturing,” IEEE Transactions on Microwave Theory and Techniques, vol. 65, no. 12, pp. 5144–5153, 2017.
  * [14] L. G. Gallant, A. Hsiao, and G. McSorley, “Benchmark physical and mechanical property chararacterization of 316L stainless steel DMLS prints,” Progress in Canadian Mechanical Engineering. Volume 4, 2021.
  * [15] “Metal 3D printing service for custom parts: Online quoting,” Metal 3D Printing Service for Custom Parts. [Online]. Available: https://www.protolabs.com/services/3d-printing/direct-metal-laser-sintering/#Alloys. [Accessed: 22-Jun-2022].
  * [16] ”Cai, J. and Syratchev, I., ”KlyC: 1.5-D Lage Signal Simulation Code for Klystrons,” IEEE Trans. Plasma Sci., Vol. 47, No. 4, 1734 (2019)


# ChatHaruhi: Reviving Anime Character in Reality
via Large Language Model

Cheng Li Ziang Leng
Chenxi Yan, Junyi Shen, Hao Wang, Weishi MI, Yaying Fei, Xiaoyang Feng
Song Yan, HaoSheng Wang, Linkang Zhan, Yaokai Jia, Pingyu Wu, Haozhen Sun

<EMAIL_ADDRESS>
https://github.com/LC1332/Chat-Haruhi-Suzumiya

###### Abstract

Role-playing chatbots built on large language models have drawn interest, but
better techniques are needed to enable mimicking specific fictional
characters. We propose an algorithm that controls language models via an
improved prompt and memories of the character extracted from scripts. We
construct ChatHaruhi, a dataset covering 32 Chinese / English TV / anime
characters with over 54k simulated dialogues. Both automatic and human
evaluations show our approach improves role-playing ability over baselines.
Code and data are available at https://github.com/LC1332/Chat-Haruhi-Suzumiya.

##  1 Introduction111 This is an open source work, and the original
affiliations of all authors can be found in the Contributors sec 8.1 . 222
This technique report was first written in Chinese then translate into English
via Claude with manually proofreading.

With the release of ChatGPT by OpenAI OpenAI (2023), large language models and
their applications have received widespread attention. Role-playing is a novel
and active application area. Users found that large language models have the
ability to act as specific characters, and communities for sharing prompts
have even emerged (e.g. AIPRM AIPRM (2023) ). Many companies have also
released role-playing products based on language models, such as Glow,
Character.AI, etc Character.AI (2023). These applications and experiments with
getting language models to role-play have garnered great interest, and have
potential applications in many areas like games, creative industries, etc.

Figure 1: Our algorithm is role-playing Haruhi Suzumiya. Note that the user’s
questions are related but not identical to the original plot, while Chat
Haruhi Suzumiya’s answers can largely quote the original plot.

In open-source role-playing implementations, developers or users have employed
similar prompts, inputting them continuously into ChatGPT or as a system
whisper into the language model:

I want you to act like {character} from {series}. I want you to respond and
answer like {character} using the tone, manner and vocabulary {character}
would use. Do not write any explanations. Only answer like {character}. You
must know all of the knowledge of {character}. My first sentence is "Hi
{character}."

Figure 2: The statistics of the ChatHaruhi-54K dataset, showing 32 characters
and 54,726 dialogues. The opaque bars indicate the original script data, while
the translucent bars present simulated dialogues generated by models like
Alpaca.

With the intelligence exhibited by larger language models like ChatGPT or
ClaudeAnthropic (2023), trained on many stories, users found that models can
demonstrate a certain capability for role-playing under such prompts. However,
while simple, such implementations have the following drawbacks: 1. They rely
heavily on the language model’s existing memories. If the language model’s own
memories about the work are fuzzy, it cannot mimic specific characters well.
2. The "know all of the knowledge of {character}" is vaguely defined, and does
not guard well against hallucinations. 3. Even with such prompts, the
chatbot’s conversational style is still heavily influenced by the underlying
language model. Adjusting the prompt may alleviate this, but finely tuning the
prompt is needed for each character. These drawbacks clearly limit the utility
of such role-playing chatbots.

Another simple idea is to fine-tune the model on the character’s dialogues.
With sufficient data, language models can capture a character’s tone, but this
also introduces new problems. In a preliminary experiment, we found that fine-
tuned ChatBots produced more hallucinations. Also, for many minor characters,
it is difficult to obtain enough data for fine-tuning. In summary, better
enabling language models to role-play and mimic character classics remains an
unsolved issue.

The main goal of this project is to study whether natural language models can
play real characters from anime, TV or other works during a conversation. In
this process, we believe a virtual character consists of three core
components:

Knowledge and background: Each virtual character exists in their own
background. Characters in Harry Potter exist in the magical world of Harry
Potter. Haruhi Suzumiya is situated in a Japanese high school. Other anime
characters also have their own worldbuilding. Therefore, in constructing the
chatbot, we hope it can understand the setting of the corresponding story.
This poses a major test of the language model’s memory, often requiring
external knowledge bases.

Personality: The personality of a character is also a very important part of
anime, TV and even game works. The personality must remain consistent
throughout the work. Some literary works even define the personality first
before writing the rest. Therefore, we hope the chatbot reflects the original
personality.

Linguistic habits: Language habits are easiest for language models to mimic.
With large models in recent years, given suitable examples in context,
language models can produce mimicking outputs. Here we hope that fans
interacting with the chatbot can ’reproduce’ classic excerpts, providing them
with a better experience.

The key idea of this project is to extract as much of the original script as
possible to form a memory database for the character. When users ask new
questions, the system searches for relevant classic plots. Combined with
prompts about the character’s setting, we attempt to better mimic the
character by controlling the language model. Meanwhile, inspired by CAMELLi et
al. (2023) and BaizeXu et al. (2023), we designed a system to automatically
generate dialogues fitting the character’s personality, even for characters
with fewer original dialogues. This allows us to generate sufficient data for
fine-tuning a local model.

Figure 3: A blueprint of the complete ChatHaruhi system is given. Dialogues
are first extracted from novels, TV shows etc. as reference exchanges $D$ for
each character, forming the core chatbot. Simulated dialogues are further
generated by Alpaca-like models, training a 7B model. Thus, large models like
ChatGPT and Claude can be used, or fine-tuned 7B models.

The main contributions of this paper can be summarized as follows:

1\. Based on large language models, we propose a complete role-playing
algorithm system. This algorithm can effectively organize a character’s
memories, allowing language models to mimic the tone and knowledge of specific
anime/TV characters during a conversation. This system can use pretrained
model like ChatGPT and Claude, or a smaller 7B-size model.

2\. We construct a role-playing dataset covering 32 different Chinese/English
TV/anime characters. By collecting and structurally extracting dialogues from
movies, novels, scripts, we have collected over 22,000 conversational
exchanges. This data can be used to train and evaluate role-playing language
models. Using our proposed algorithm, aided by GPT-3 and GPT-4, we
additionally simulated over 31,000 dialogues for these characters. Combined,
this forms the ChatHaruhi-54k dataset.

3\. To evaluate and compare different role-playing chatbots, we use both
automatic and human evaluations. For automatic evaluation, we test whether the
chatbot can respond to classic plot points with similar answers to the
original script. For human evaluation, we propose two metrics for raters to
assess: Alignment: Whether the chatbot’s answer aligns with the character’s
original setting. Response quality: Whether the chatbot’s responses have good
linguistic quality. Results show that given the same base language model, our
algorithm yields improved role-playing performance.

To support further research, all data and code are available at
https://github.com/LC1332/Chat-Haruhi-Suzumiya. We are also working to further
modularize the project code for ease of use (see Appendix: API Design).

##  2 Related Work

##### In-context Learning

In the development of ChatGPT, starting from GPT2 Brown et al. (2020), it was
proposed to eliminate special extraction tokens in language models and adopt
the form of instructions + examples to enhance the natural language model’s
ability to handle various tasks. Since its introduction, In Context Learning
has been a focal point of research. Previous work has proposed better methods
of posing questions Zhao et al. (2021); Holtzman et al. (2021), better
selection of token examples for demonstration Liu et al. (2021); Lu et al.
(2022); Rubin et al. (2022), meta-training with explicit contextual learning
objectives Chen et al. (2022), and a variant of context learning that follows
instructionsMishra et al. (2022); Efrat et al. (2021); Wei et al. (2022); Sanh
et al. (2022) . At the same time, some studies have reported issues of
vulnerability and over-sensitivity in context learning Lu et al. (2022); Zhao
et al. (2021); Mishra et al. (2022).

In our work, in-context learning is primarily used to generate user questions
for our chatbot. Given a character’s background and prior memories, our
approach produces the subsequent dialogues responding to each question in-
context. Compared to general conversational agents, our system focuses on
tailoring the dialogues to a specific persona based on its given settings and
history. The generated question-answer pairs provide valuable data for
analyzing and learning the behaviors of a persona-based agent.

##### Automatic Dialogue Generation

Recent advances in large language models (LLMs) have shown impressive
capabilities in open-domain dialogues. Models like Meena (Adiwardana et al.,
2020), LaMDA Touvron et al. (2023)) and ChatGPT OpenAI (2022) are pretrained
on massive amounts of conversational data and can conduct human-like chitchat.
Concurrently, there have been attempts to replicate such models with open-
source LLMs. Alpaca Taori et al. (2023)uses self-instruction to collect data
from a proprietary LLM, and fine-tunes LLaMATouvron et al. (2023). Vicuña (Ahn
et al., 2023) trains LLaMA on dialogues from ChatGPT.

More related to our work, BaizeXu et al. (2023) proposes a pipeline to
automatically generate multi-turn dialogues by making ChatGPT converse with
itself. The data is used to fine-tune LLaMA into Baby Baize. CAMELLi et al.
(2023) explores facilitating cooperation between chat agents using inception
prompting, guiding them to complete tasks while maintaining human intentions.

Our work similarly leverages large conversational models like ChatGPT to
automatically generate dialogues between agents. However, different from prior
work, we focus on dialogue generation for a specific character that the user
wants to role-play. Our system incorporates substantial prompts about the
character’s background, personality and prior conversations, in order to
produce in-character dialogues. The generated exchanges provide valuable data
for learning the behaviors of a specific persona.

Figure 4: The core dialogue system of ChatHaruhi, comprising the system
prompt, character memories $D(q,R)$ retrieved for the user query $q$, and the
dialogue history $H$.

##  3 ChatBot Design

Given a specific character $R$ and a query question $q$, we want to be able to
generate an answer $a$ based on the character’s knowledge background,
personality and language habits:

$a=\mathrm{argmax}_{a^{\prime}}P(a^{\prime}|R,q,\Theta)$ (1)

where $\Theta$ represents the language model parameters which are static
during inference. After the release of ChatGPT, users found that they can
specify a certain system prompt $s_{R}=$ ‘I want you to act like character
from series..‘ So:

$a=\mathrm{argmax}_{a^{\prime}}P(a^{\prime}|s_{R},q,\Theta)$ (2)

This shows the language model has some role-playing ability. However, the
character’s memory completely relies on the parameters $\Theta$. If the
model’s knowledge is limited and does not even contain the desired character
$R$, it often fails to achieve the ideal effect.

Inspired by in-context learning, in addition to $\Theta$ and $s_{R}$ , we can
introduce a sequence of the character’s previous dialogues, i.e.,

$D(q,R)={(u_{1},v(u_{1};R)),...,(u_{M},v(u_{M};R))}$ (3)

where $u_{m}$ is any question raised by characters other than $R$, and
$v(u_{m};R)$ is character $R$’s reply to the question. We hope that by
inputting the classic dialogues of the character into the context, the model
will have a better ability to play the role of character $R$, i.e.,

$a=\mathrm{argmax}_{a^{\prime}}P(a^{\prime}|s_{R},D(q,R),q,\Theta)$ (4)

For characters with a larger worldview, in order to make the content of
$D(q,R)$ more relevant to the content of $q$, we use sentence embeddings here
to search for the $M$ most relevant Q&As from a larger memory bank $U$. Here
$U$ is the set of all sentences where other characters interact with $R$
throughout the novel/movie.

Of course, in practice, we also need to additionally record a dialogue history
$H$ to ensure conversational continuity, since the context of previous
dialogues needs to be considered as well.

$a=\mathrm{argmax}_{a^{\prime}}P(a^{\prime}|s_{R},D(q,R),q,H,\Theta)$ (5)

The overall construction of the chatbot is shown in the fig 4. In the other
subsections of this chapter, we will introduce the details of the system
prompt $s_{R}$, the classic dialogues $D$ from the story, and the searching
mechanism $u_{m}(q)$.

###  3.1 System Prompt

Actually, the system prompt mentioned in the introduction can already achieve
basic functionality when using ChatGPT as the base model. After initial
experiments, we find two aspects of this system prompt that need improvement:

Won’t repeat lines: For models like ChatGPT and LLaMA2 that have gone through
a lot of reinforcement learning from human feedback (RLHF), since these
language models often face tasks like "Give me m different options", "Generate
m titles", etc., the output of such language models tends to not repeat
content from the context. We also observed this phenomenon in preliminary
experiments. Therefore, our proposed method is to emphasize in the prompt
$s_{R}$ that the model is cosplaying a specific character. And emphasize that
the language model can reuse classic lines from the novel or movie.

Character emphasis not prominent enough: Due to RLHF, each language model has
its own specific language preferences. Even when given $D(q,R)$ to imitate,
the model’s output is still influenced by the language model itself. We find
that supplementing the personality of the character at the end of $s_{R}$
yields better results in this case.

Based on the above two points, the character setting prompt template $s_{R}$
we commonly use is as follows:

I want you to act like {character} from {series}.
You are now cosplay {character}
If others’ questions are related with the novel, please try to reuse the
original lines from the novel.
I want you to respond and answer like {character} using the tone, manner and
vocabulary {character} would use.
You must know all of the knowledge of {character}.
{Supplementary explanation of the character’s personality}

Note that we have strengthened the requirement for the language model to reuse
sentences from the story. We find the final output of the language model is
quite sensitive to the effects of the supplementary explanation. Including
adding certain verbal tics in the supplementary explanation will also be
reflected in the final output.

###  3.2 Dialogues from each Character

In order to better reproduce the character’s behavior in novels/TV
shows/movies, we have included a large number of classic script excerpts in
$D$. It should be noted here that except for a few characters (such as
crosstalk performer Yu Qian), not all dialogues are in a good question-answer
format. Here, the actual $D$ we use is in story form, as shown in the figure,
i.e.,

$D(q,R)=\\{d_{1},d_{2},...,d_{M}\\}$ (6)

We ensure that in $d_{m}$ there is at least one dialogue pair in the form of
$(u_{m},v(u_{m};R))$. Between the information of $u$ and $v$, there may be
narration, or more dialogues from other characters, or action information of a
character. We relax this condition so that each story $d_{m}$ can better
preserve the plot of the dialogue. Sometimes the narration and actions around
the dialogues are inevitable. Also, relaxing the condition is conducive to
preparing more script data, which will be mentioned in the later novel text
extraction.

###  3.3 Original Dialogue Searching

In practice, the total number of tokens summed over all stories for a
character $R$ often far exceeds the mature scope of the language model. Here
we use a search method to reduce the number of Original Dialogues input each
time.

For a query $q$, we will use a sentence embedding model $f()$ to extract
embeddings $f(d)$ for all $d\in D$. After similarly extracting $f(q)$ for the
query $q$, we extract the $M$ samples from $D$ that are closest (in cosine
similarity) to $f(q)$. This forms the reference context $D(q,R)$ for this
dialogue.

For the number of original dialogue excerpts cited per dialogue $M$, we will
actually adjust dynamically based on the number of tokens searched. In the
specific implementation, if using OpenAI’s turbo-3.5 model, we will limit the
total number of tokens in $D$ to within 1500.

Therefore, when building the dialogue memory bank, we suggest that the length
of each story should not be too long, so as not to occupy the space of other
stories during search.

For the embedding model, we use OpenAI’s text-embedding-ada-002 model
Neelakantan et al. (2022). At the same time, for Chinese questions, we use
Luotuo-Bert-Medium Siyi et al. (2023), because the latter is a distilled model
from the former, with the same distribution. This allows us to seamlessly use
Luotuo-Bert even when the stories are in English, enabling cross-lingual
chatbots, which will be explained in the cross-lingual experiments section.

It should be noted that we have noticed many other embedding models such as
instructor-large Su et al. (2022), M3E Wang Yuxin (2023) or BGE Shitao Xiao
(2023) models, etc. In the reconstruction of ChatHaruhi 2. 0 A when there is
sufficient time, we will replace the embedding model for experiments.

###  3.4 Chat Memory

For the memory $H$, we record each user query $q$ and the chatbot’s response
$a$, forming a sequence $H$:

$H=\\{(q_{1},a_{1}),...,(q_{T},a_{T})\\}$

The information in $H$ is also input to the language model to ensure
conversational coherence. In the actual implementation, starting from $T$, we
count the total number of tokens forward. And limit the dialogue history input
to the language model to within 1200 tokens.

Therefore, in this work, we do not care about the character’s long-term
memory. 1200 tokens can accommodate about 6-10 rounds of dialogue. As language
models become able to contain longer contexts, how to encode and summarize a
long-term memory will also be a more interesting problem.

Figure 5: ChatHaruhi-54K covers 32 different Chinese and English characters.

##  4  Character Dataset Building

For our project, we need classic stories $D(q,R)$ related to character $R$ as
input when generating dialogues involving that character. Therefore,
constructing original dialogues $D$ for each character is crucial. In
addition, we need more data than just the original dialogues $\|D\|$ to train
local models. In this section, we first introduce how to build $D$ for each
character.

###  4.1  Characters

In the current version of our project, we selected 32 characters to form the
dataset.

Haruhi Suzumiya: When choosing the first character, we wanted someone who
satisfies: 1. ChatGPT has some knowledge about them. 2. The character’s
fictional world is not too large for the first character. 3. The character has
a distinctive personality. So we chose the character Haruhi Suzumiya, a famous
anime character who represents the transition from light novels to animation.
Many subsequent school-based light novel animes pay homage to Haruhi Suzumiya.

Li Yunlong: (from Drawing Sword) is the first character ChatGPT knows little
about that we added. We found that with appropriate memory dialogues, ChatBot
can also effectively mimic Commender Li’s behaviors. Here we use the TV
version of Drawing Sword, which has extensive dialogue writing compared to the
novel, sculpting a very three-dimensional and vivid military character.

Harry Potter (novel, 8 characters): After initial qualitative tests of the
first few characters were successful, we started trying to build stories with
larger fictional worlds. Harry Potter is a good choice with a large audience,
and if we want to incorporate multimodal data in the future, there are also
suitable datasets we can reference. For the Harry Potter novels, we used the
novel extraction tool described later to extract passages. We then compiled
the story collection $D_{R}$ for each character separately.

Big Bang Theory (TV show, 3 characters): Big Bang Theory is also a TV show
that researchers enjoy using to form datasets. The show itself depicts the
stories of several Caltech researchers. From Big Bang Theory, we extracted
Sheldon, Penny and Raj for experiments. As with Harry Potter, Big Bang Theory
enables potential future incorporation of other multimodal datasets for
multimodal research.

Demi-Gods and Semi-Devils (novel, 7 characters): Demi-Gods and Semi-Devils is
a grand wuxia novel by Jin Yong. With its multi-threaded narrative centered
around the three protagonists Duan Yu, Qiao Feng and Xu Zhu, the intricate
story unfolds. Having been adapted into TV dramas multiple times, the novel
holds a special place in Chinese readers’ hearts. We extracted Duan Yu, Xu
Zhu, Qiao Feng, Xiao Feng, Jiumozhi, Murong Fu, Wang Yuyan - noting Xiao Feng
is Qiao Feng’s name after finding out he is Khitan. Treating them as two
separate characters allows us to observe any different behaviors in response
to the same questions.

Wei Xiaobao: is the protagonist in another novel by Jin Yong, The Deer and the
Cauldron. He is a clever and cunning character, holding positions in the Qing
government, anti-Qing organizations and the jianghu underworld, while also
popular with female characters.

My Own Swordsman" (TV show, 3 characters): My Own Swordsman" is a popular
episodic comedy in China. We extracted Tong Xiangyu, Bai ZhanTang and Guo
Furong from it.

Genshin Impact (wiki, 5 characters): Genshin Impact is an open world RPG
developed by miHoYo that is commercially successful with players worldwide.
Its intricate story is set in the fictional world of Teyvat. Many characters
are beloved by players - we extracted Akaya, Raiden Shogun, Zhongli, Hutao and
The Wanderer.

Wang Duoyu: is the protagonist of the movie Hello Mr. Billionaire, a remake of
the 1985 American movie Brewster’s Millions. The script can also be found
online. Interestingly, Wang Duoyu is supposed to keep the secret of "spending
100 million in a month". The current ChatBot cannot keep this secret well, so
improving via constructed reasoning chains or constitutional additions could
be an interesting future direction.

Counsellor Tang: Counsellor Tang(Tang-shiye) is a frequently appearing minor
character in the movie Let the Bullets Fly directed by Jiang Wen. The latter
is a hugely popular movie among Bilibili users, and its script can be found
online. Thus we attempted constructing the character Counsellor Tang.

Yu Qian: Yu Qian is the witty stock character in Guo Degang’s comic crosstalk
routines. Because witty stock characters have very consistent speaking styles
in Chinese crosstalk, and the Crosstalk-Generation Zhang (2020) project has a
large corpus of Yu Qian’s material, we also included him in our project.

###  4.2 Original Script Extraction

In all cases, a character’s dialogues do not naturally organize into the form
shown in Figure 1. For this, we constructed different extraction tools for TV
shows, movies or novels:

####  4.2.1  Quotes Data

The scripts for Counsellor Tang from Let the Bullets Fly, Wang Duoyu from
Hello Mr. Billionaire, and Yu Qian can be directly found online. For the first
two characters, we manually split the scripts into segments and formatted them
into the defined "character: ’dialogue’" form.

Yu Qian’s corpus in Crosstalk-Generation contains over 6,000 dialogue
exchanges. All dialogues are in good question-answer form. We checked the
length of each utterance $u$ and $v$, and split before a sentence if it had
locally maximal length in a continuous crosstalk routine.

For Genshin Impact and The Big Bang Theory, enthusiastic netizens have posted
quotes and plots on wikis and websites. Preliminary organization of these
resources yields the target format. Here the Big Bang Theory data was split
using the same finite state machine as for the novel extraction.

####  4.2.2  Extract from TV Series

For characters like Haruhi Suzumiya and Li Yunlong, TV scriptwriters often
create original dialogues. Comparing the Drawing Sword’s novel and TV drama
for example, the latter has more conversational information and three-
dimensional characterization. Extracting character dialogues from the TV drama
is needed in such cases.

We first perform speech recognition using Whisper or directly utilize original
subtitles. We further identify the speaker of each line using a 192-dim ECAPA-
TDNN speaker verification embedding, trained on some labeled character
dialogues.

Finally, recognition errors are manually corrected and scripts split. Since
manual organization is time-consuming (often needing to rewatch the show), we
only processed Haruhi Suzumiya and Li Yunlong this way. We hope more
enthusiasts can build characters after open-sourcing the full TV processing
toolkit.

####  4.2.3  Extract from Novel

Figure 6: The 22k original exchanges for fine-tuning are extracted from movie
scripts, skit scripts, TV shows and novels.

Thanks to progress in large language models, we can also use general language
models to batch process novels. For Demi-Gods and Semi-Devils, The Deer and
the Cauldron, and Harry Potter, we utilize Kor’s extraction mechanism (an in-
context learning information extraction library) to extract ‘character-action-
dialogue‘ information sentence by sentence.

In the extraction prompt, if a sentence contains dialogue, we want the
language model to record the action as ‘dialogue‘ and the dialogue content.

If a sentence does not contain dialogue, we want the language model to
summarize the character’s actions in the action field. The language model also
has some ability to infer who is speaking in each dialogue from context.

After bulk novel extraction, we use a finite state machine to split the
dialogues. For each protagonist, we look for a segment of appropriate length,
preferably with limited length, few characters, and minimal non-dialogue
sentences. The automatic extraction is implemented with such a state machine.

Extraction statistics are shown in the table.

##  5  Dialogue Synthesizing

Given the system prompt $s_{R}$ and corresponding classic animation $D(q,R)$
for each character, we find the character can already respond to user
questions $q$ in a certain style. However, at this point we need to leverage
ChatGPT or Claude APIs to model $p(a|s_{R},D(q,R),q)$.

If we want to transfer the functionality of ChatHaruhi to a local model, we
need to build a suitable $(R,q,a)$ dataset. In this section we discuss how to
augment dialogue data for characters with limited data.

###  5.1 Generate Dialogue from Question

Note that the collected $D$ data is not in strict $(q,a)$ form. This means we
cannot simply fine-tune a language model to learn all $\\{D_{R}\\}$ data. To
address this, for any $d\in D_{R}$, we take all dialogues before the
protagonist $R$’s turn as $q$, hoping to take this $q$ as the first question
$q_{1}$ to generate a dialogue.

Figure 7: Massive simulated queries are generated with Alpaca-like. Detailed
prompts given in Appendix.

In practice, we find the language model can sometimes output multiple
dialogues, i.e. after giving one answer $a_{1}$, it generates a new question
$q_{2}$ and subsequent replies. Moreover, all $a$ in such generated dialogues
conform to character $R$’s profile. Hence, we want to modify our ChatBot to
further utilize this trait to produce more dialogues. For each dialogue $d$,
we locate the first utterance by character $R$. Before this utterance, $d$ is
split into left and right parts $d^{L}$ and $d^{R}$. We insert User Message
special tokens at both ends of $d^{L}$ and AI Message special tokens at both
ends of $d^{R}$. We do this for all $M$ stories, so that based on these $M$
examples, given $q_{1}$, the language model can simulate generating the
corresponding dialogue $d^{\prime}$. The resulting $d^{\prime}$ becomes fine-
tuning data for the language model. That is,

$d^{\prime}(q_{1})=\mathrm{LLM}(s_{R},(d^{1}_{M},d^{R}_{M}),\ldots,(d^{L}_{M},d^{R}_{M}),q_{1})$
(7)

Often this method generates $d^{\prime}$ with only one sentence. But it can
also generate multi-sentence dialogues with close to 50% probability. When the
given $q$ overlaps with the original text, due to our $s_{R}$ prompt design,
the model tends to output according to the protagonist’s original lines.

###  5.2  Question Generating

Note some characters have very limited data, insufficient for fine-tuning
language models. Thus we need to augment questions $q$ using existing data for
each character. Fortunately, in a recent study, R. Taori et al. augmented from
less than 200 instructions to 54K questions. Here we adapt their prompt (see
Appendix for details).

Figure 8: LLM generates full dialogues conditioned on the first utterance and
history.

When using augmentation methods like Alpaca, we need to provide a clear
$(q,a)$ pair, based on which the model generates around 10 heuristic outputs.
Here we only keep $q_{s}$, then leverage the aforementioned technique to
regenerate training dialogues in the character’s ChatBot. We use a mix of
ChatGPT, GPT4 and Claude for question generation, with the latter two
producing questions more relevant to the character, at a higher cost.
Statistics of final generation for each character are shown in Figure 2. Note
in the first version, we used Alpaca-like approaches to generate around 27k
data. Alpaca-generated questions are influenced by our examples, i.e. they
tend to relate to the original scripts. We hope to further filter for real
user questions in later versions for testing.

We collected 22,752 original dialogues ($D_{R}$) and additionally simulated
31,974 questions with corresponding dialogues for the ChatHaruhi-v1 dataset.
Note that each dialogue does not necessarily consist of only one QA pair. In
total, we collected 54,726 dialogues.

##  6  Experiments

Previous work often evaluates the quality of dialogues in role-playing
chatbots by conducting pairwise human comparisons of dialogue outputs from
different language models, and analyzing the results with TrueSkill or Elo
rating. This approach has high cost due to the need for human evaluation, and
different human raters may yield inconsistent results. Moreover, for role-
playing, dialogue quality alone is insufficient for evaluation. For example,
prompts like "Li Yunlong’s language style is crude" or "Bai Zhantang’s
language has a Jianghu flavor" would significantly reduce dialogue quality
despite accurately reflecting the role. Thus, human evaluations should judge
both ’role consistency’ and ’dialogue quality’ separately.

Figure 9: Experiments are conducted on three characters using: a) ChatGPT with
the prompt, b) full ChatHaruhi + ChatGPT, c) ChatGLM2 with the prompt, d) full
ChatHaruhi + ChatGLM2, and e) full ChatHaruhi + fine-tuned ChatGLM2. More
characters are available on our hugging face demo.

###  6.1  Metrics for Automatic Evaluation

Due to the limited dialogues and lack of continuity for the 5 Genshin Impact
roles, we only consider the remaining 27 roles for evaluation. For each of the
27 roles, we select 30 stories containing long dialogues $\hat{a}$ spoken by
character $R$ from their classic narratives $D$. We test whether a model can
produce a plausible response $a$ to the previous utterance $q$ before
$\hat{a}$. We judge plausibility by the similarity of $\hat{a}$ and $a$ using
sentence embeddings. Specifically, we compute cosine similarity between
$\hat{a}$ and $a$ for each role. We use OpenAI’s Text-Embedding-Ada-002, a
multilingual sentence embedding model, for evaluation.

###  6.2  Metrics for User Study

The user study is still in progress and will be included in a future version
of this report.

###  6.3  Language Model Tuning

With the complete 54K ChatHaruhi dataset, we can fine-tune local language
models. Approximately 15K dialogues are in English, with the remainder in
Chinese. We fine-tune the ChatGLM2-6B model [cite]. Inputs follow the
$s$-$R$-$D$-$H$-$q$ format described earlier, with $a$ as the target for GPT
loss calculation. We obtain three models:

  * •

model-A, Fine-tuned on the 22,752 original dialogues.

  * •

model-B, Fine-tuned on the full 54K dataset with original and simulated
dialogues.

  * •

model-C, Alternatively, we can fine-tune on original character utterances
rather than ChatBot-generated dialogues.

All models were fine-tuned for 3 epochs on 4-A100 GPUs. We will release
versions A and B with this report, and version C in a future update.

###  6.4  Qualitative Results

We qualitatively compare five models:

  1. 1.

GPT Turbo 3.5 given just the system prompt $s_{R}$.

  2. 2.

Turbo 3.5 given the full $s$-$R$-$D$-$H$-$q$ prompt.

  3. 3.

ChatGLM2 given just the system prompt.

  4. 4.

ChatGLM2 given the full prompt.

  5. 5.

ChatGLM2 fine-tuned on ChatHaruhi with the full prompt.

With classic dialogues and an improved prompt, models like ChatGPT can
effectively adopt the speaking style of specific characters. Fine-tuning the
7B model also helps it internalize the full prompt.(Fig 9)

###  6.5  Quantitative Results

Quantitative experiments are still in progress and will be included in a
future version of this report.

###  6.6  User Study

The user study is still in progress and will be included in a future update.

##  7  Conclusion, Discussion & Future Work

In this tech report, we present an attempt at constructing a system capable of
role-playing dialogues as different virtual characters. Leveraging the in-
context learning abilities of language models and the growth of larger models,
we show the possibility of mimicking distinctive conversational styles by
providing appropriate system prompts and example passages of classic scenes
featuring each character. We generate a dataset of 54K simulated dialogues and
demonstrate the feasibility of fine-tuning multiple roles into a single 7B
parameter local model.

As our first attempted character is the vividly characterized Haruhi Suzumiya,
we name the project ChatHaruhi and the dataset Haruhi-54K. Accompanying this
report, we release model A trained on 23K original transcripts and model B
trained on the full 54K dataset, demos on HuggingFace, and the complete
ChatHaruhi-54K dataset.

In future iterations, we will refine the ChatHaruhi interface for easier
reusability (see Appendix A for ChatHaruhi 2.0 draft), and supplement the
quantitative evaluations.

##  8  Acknowledgments

This is an open source project originating from the June 2023 study group of
the DataWhale community, where we tested an early Haruhi-only version of the
chatbot and received highly enthusiastic feedback. We then recruited
volunteers from the community (e.g. Yan Chenxi, Feng Xiaoyang) for
collaborative development. DataWhale nominated the project for the ModelScope
Hackathon in early July, bringing greater exposure and drawing more
developers. We sincerely thank DataWhale and ModelScope for their support
during this project.

ChatHaruhi is also a sub-project of the open source community Luotuo, which
has benefited from various donations including funding, computing resources,
and OpenAI API credits. We thank Luotuo’s sponsors for their supports.

###  8.1 Contributors

This project is an open source work, with all personnel contributing and
developing in their spare time. The developers of this project may belong to
other institutions or be independent developers. Here we list the main
contributions of each developer, as well as their affiliations.

Cheng Li@SenseTime purposed the entire project and designed and implemented
most of the functionality.

Ziang Leng@SenseTime designed and implemented the overall training, data
generation and backend architecture of ChatHaruhi1.0.

Chenxi Yan@Chengdu University of Information Technology implemented and
maintained the backend of ChatHaruhi1.0 version.

Junyi Shen@Zhejiang University implemented the training code and participated
in the generation of training dataset.

Hao Wang collected script data from My Own Swordsman and participated in the
generation of augmented data.

Weishi MI@Tsinghua University participated in the generation of augmented
data.

Yaying Fei@Beijing University of Technology implemented the ASR function of
the script tool and participated in the Openness-Aware Personality paper sub-
project.

Xiaoyang Feng@Nanjing Agricultural University integrated the functions of the
script recognition tool and participated in the Openness-Aware Personality
paper sub-project.

Song Yan collected data from The Big Bang Theory. Implemented script format
conversion functionality.

HaoSheng Wang implemented voiceprint recognition function in script tool, and
tts-vits speech synthesis function.

Linkang Zhan@Case Western Reserve University collected system prompt and story
data from Genshin Impact.

Yaokai Jia implemented the Vue version of the front end, and practiced GPU
extraction of Bert in the psychology project.

Pingyu Wu@Juncai Shuyun helped deploy the first version of the training code.

Haozhen Sun@Tianjin University drew the mosaic of ChatHaruhi characters.

If you have any suggestions for the project, such as the interface design of
ChatHaruhi2.0, or want to add references to the future version of this report,
please go to our project https://github.com/LC1332/Chat-Haruhi-Suzumiya to
submit an issue.

## References

  * AIPRM (2023) AIPRM. 2023.  Complete small business tasks with prompt management & 1-click prompts for ChatGPT.  https://www.aiprm.com/.  Accessed: 2023-02-18.
  * Anthropic (2023) Anthropic. 2023.  Introducing claude.  https://www.anthropic.com/index/introducing-claude.  Accessed: 2023-2-18.
  * Brown et al. (2020) Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020.  Language models are few-shot learners.  _Advances in neural information processing systems_ , 33:1877–1901.
  * Character.AI (2023) Character.AI. 2023.  Character ai.  https://beta.character.ai/help.  Accessed: 2023-08-18.
  * Chen et al. (2022) Mingda Chen, Jingfei Du, Ramakanth Pasunuru, Todor Mihaylov, Srini Iyer, Veselin Stoyanov, and Zornitsa Kozareva. 2022.  Improving in-context few-shot learning via self-supervised training.  In _Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 3558–3573, Seattle, United States. Association for Computational Linguistics.
  * Efrat et al. (2021) Avia Efrat, Uri Shaham, Dan Kilman, and Omer Levy. 2021.  Cryptonite: A cryptic crossword benchmark for extreme ambiguity in language.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 4186–4192, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Holtzman et al. (2021) Ari Holtzman, Peter West, Vered Shwartz, Yejin Choi, and Luke Zettlemoyer. 2021\.  Surface form competition: Why the highest probability answer isn’t always right.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 7038–7051, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Li et al. (2023) Guohao Li, Hasan Abed Al Kader Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. 2023.  Camel: Communicative agents for "mind" exploration of large scale language model society.
  * Liu et al. (2021) Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. 2021.  Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing.
  * Lu et al. (2022) Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, and Pontus Stenetorp. 2022\.  Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity.
  * Mishra et al. (2022) Swaroop Mishra, Daniel Khashabi, Chitta Baral, and Hannaneh Hajishirzi. 2022.  Cross-task generalization via natural language crowdsourcing instructions.
  * Neelakantan et al. (2022) Arvind Neelakantan, Tao Xu, Raul Puri, Alec Radford, Jesse Michael Han, Jerry Tworek, Qiming Yuan, Nikolas Tezak, Jong Wook Kim, Chris Hallacy, et al. 2022\.  Text and code embeddings by contrastive pre-training.  _arXiv preprint arXiv:2201.10005_.
  * OpenAI (2022) OpenAI. 2022.  Introducing chatgpt.  _Open AI Blog_.
  * OpenAI (2023) OpenAI. 2023.  Chatgpt: Optimizing language models for dialogue.
  * Rubin et al. (2022) Ohad Rubin, Jonathan Herzig, and Jonathan Berant. 2022.  Learning to retrieve prompts for in-context learning.  In _Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 2655–2671, Seattle, United States. Association for Computational Linguistics.
  * Sanh et al. (2022) Victor Sanh, Albert Webson, Colin Raffel, Stephen H. Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Teven Le Scao, Arun Raja, Manan Dey, M Saiful Bari, Canwen Xu, Urmish Thakker, Shanya Sharma Sharma, Eliza Szczechla, Taewoon Kim, Gunjan Chhablani, Nihal Nayak, Debajyoti Datta, Jonathan Chang, Mike Tian-Jian Jiang, Han Wang, Matteo Manica, Sheng Shen, Zheng Xin Yong, Harshit Pandey, Rachel Bawden, Thomas Wang, Trishala Neeraj, Jos Rozen, Abheesht Sharma, Andrea Santilli, Thibault Fevry, Jason Alan Fries, Ryan Teehan, Tali Bers, Stella Biderman, Leo Gao, Thomas Wolf, and Alexander M. Rush. 2022.  Multitask prompted training enables zero-shot task generalization.
  * Shitao Xiao (2023) Zheng Liu Shitao Xiao. 2023.  Baai general embedding.  https://github.com/FlagOpen/FlagEmbedding.
  * Siyi et al. (2023) Liu Siyi, Leng Ziang, Huang Hongsen, Chen Shunian, Hu Jing, Sun Ao, Chen Qiyuan, and Cheng Li. 2023.  Luotuo embedding: Generative text embedding model distilled from openai api.  https://github.com/LC1332/Luotuo-Text-Embedding.
  * Su et al. (2022) Hongjin Su, Jungo Kasai, Yizhong Wang, Yushi Hu, Mari Ostendorf, Wen-tau Yih, Noah A Smith, Luke Zettlemoyer, Tao Yu, et al. 2022.  One embedder, any task: Instruction-finetuned text embeddings.  _arXiv preprint arXiv:2212.09741_.
  * Taori et al. (2023) Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023.  Stanford alpaca: An instruction-following llama model.  https://github.com/tatsu-lab/stanford_alpaca.
  * Touvron et al. (2023) Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023.  Llama: Open and efficient foundation language models.  _arXiv preprint arXiv:2302.13971_.
  * Wang Yuxin (2023) He sicheng Wang Yuxin, Sun Qingxuan. 2023.  M3e: Moka massive mixed embedding model.
  * Wei et al. (2022) Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V. Le. 2022.  Finetuned language models are zero-shot learners.
  * Xu et al. (2023) Canwen Xu, Daya Guo, Nan Duan, and Julian McAuley. 2023.  Baize: An open-source chat model with parameter-efficient tuning on self-chat data.
  * Zhang (2020) Zuobai Zhang. 2020.  GitHub - Oxer11/Crosstalk-Generation: Perform crosstalk with Qian Yu — github.com.
  * Zhao et al. (2021) Tony Z. Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. 2021.  Calibrate before use: Improving few-shot performance of language models.

##  Appendix A  Appendix-A: ChatHaruhi 2.0 Design

ChatHaruhi is an open-source conversational agent project. Initially developed
for many hackathons, it incorporated multimodal features like images and
speech. The project can now be run via a Gradio demo in the source code.
However, this design hampers research on multiple chatbots, e.g. adding
characters, studying interactions, and upgrading the memory. We will refactor
ChatHaruhi after this arXiv paper published, with a tentative API as follows:

⬇

1from ChatHaruhi import ChatHaruhi

2

3chatbot = ChatHaruhi(system_prompt=’prompt.txt’,

4

5story_db=’./story_chroma_folder’,

6llm=’openai’)

7

8response = chatbot.chat(text=’Can you introduce yourself?’,

9role=’Kyon’)

This simplifies access via a system prompt and vector database. LLM switching
also becomes possible, including local models, Claude, and Xinghuo etc.

If using the characters involved in ChatHaruhi-54K, you can directly use:

⬇

1from ChatHaruhi import ChatHaruhi

2

3chatbot = ChatHaruhi(role_name = ’Haruhi’,

4

5llm = ’openai’)

6

7response = chatbot.chat(text = ’Can you introduce youself?’,

8role = ’Kyon’)