# RLPlanner: Reinforcement Learning based Floorplanning for Chiplets with Fast
Thermal Analysis ††thanks: This work was supported by Pre-research project of
ministry foundation (Grant No.31513010501).(* Corresponding Authors: Leilai
Shao<EMAIL_ADDRESS>and Xiaolei Zhu (xl_zhu@zju.edu.cn))

Yuanyuan Duan1, Xingchen Liu1, Zhiping Yu2, Hanming Wu1, Leilai Shao3∗,
Xiaolei Zhu1∗ 1School of Micro-Nano Electronics, Zhejiang University,
Hangzhou, China  2School of Integrated Circuits, Tsinghua University, Beijing,
China  3School of Mechanical Engineering, Shanghai Jiao Tong University,
Shanghai, China

###### Abstract

Chiplet-based systems have gained significant attention in recent years due to
their low cost and competitive performance. As the complexity and compactness
of a chiplet-based system increase, careful consideration must be given to
microbump assignments, interconnect delays, and thermal limitations during the
floorplanning stage. This paper introduces RLPlanner, an efficient early-stage
floorplanning tool for chiplet-based systems with a novel fast thermal
evaluation method. RLPlanner employs advanced reinforcement learning to
jointly minimize total wirelength and temperature. To alleviate the time-
consuming thermal calculations, RLPlanner incorporates the developed fast
thermal evaluation method to expedite the iterations and optimizations.
Comprehensive experiments demonstrate that our proposed fast thermal
evaluation method achieves a mean absolute error (MAE) of ±0.25 K and delivers
over 120x speed-up compared to the open-source thermal solver HotSpot. When
integrated with our fast thermal evaluation method, RLPlanner achieves an
average improvement of 20.28% in minimizing the target objective (a
combination of wirelength and temperature), within a similar running time,
compared to the classic simulated annealing method with HotSpot.

###### Index Terms:

reinforcement learning, fast thermal evaluation, chiplet floorplanning

##  I Introduction

To address the increasing cost of large Systems-on-Chip (SoCs) on advanced
technology nodes, chiplet-based design or 2.5D integration has emerged as a
solution. However, as the complexity and compactness of chiplet-based systems
increase, it becomes critical to address issues such as microbump assignments,
interconnect delays, and thermomechanical stress during the initial
floorplanning stage.

Traditional physical floorplanning of monolithic chips primarily focuses on
reducing total wirelength and minimizing area [1], which results in compact
floorplans that may lead to potential thermal-induced failures. Recent
floorplanning works have started considering the thermal aspect in chiplet-
based systems [2, 3]. They employ simple greedy strategies or simulated
annealing (SA) to handle chiplet system floorplanning, lacking the flexibility
and transferability required for complex 2.5D integrations. Moreover, these
optimization methods are constrained by time-consuming thermal evaluations,
which hinder fast and efficient exploration of thermal-aware floorplanning.
Some attempts have been made to use convolutional neural networks (CNN) or
graph convolutional networks (GCN) to build surrogate models for accelerated
thermal evaluations [4, 5]. However, these models often rely on empirical
parameters, such as tile and window sizes, requiring domain knowledge for
determination, or show limited speed-ups ($<$3x), making them impractical for
real designs.

Figure 1: Overview of RLPlanner: The implementation of chiplet floorplanning
using reinforcement learning.

In this work, we present RLPlanner, a floorplanning tool based on
reinforcement learning (RL) for early-stage chiplet-based systems. RLPlanner
utilizes advanced RL techniques and incorporates a novel fast thermal
evaluation method to optimize both the maximum operating temperature and total
wirelength of the chiplet system.

TABLE I: Comparisons against baselines on benchmark systems

Design | Multi-GPU System[6] | CPU-DRAM System[7] | Ascend 910 System[8]
---|---|---|---
Method | Reward |  | Wirelength
---
($mm$)
| Temperature
---
(∘C)
| Runtime
---
($s$)
Reward |  | Wirelength
---
($mm$)
| Temperature
---
(∘C)
| Runtime
---
($s$)
Reward |  | Wirelength
---
($mm$)
| Temperature
---
(∘C)
| Runtime
---
($s$)
RLPlanner | -37.1263 | 97742 | 91.15 | 20910 | -44.9467 | 176246 | 92.88 | 8925 | -7.4063 | 18130 | 77.12 | 7773
RLPlanner(RND) | -40.2777 | 104636 | 91.85 | 20380 | -41.7496 | 164460 | 92.15 | 8993 | -7.4433 | 18221 | 76.84 | 9991
TAP-2.5D(HotSpot) | -42.4572 | 124639 | 91.68 | 35211 | -60.3570 | 181269 | 97.94 | 15056 | -8.7651 | 21456 | 74.94 | 15731
TAP-2.5D*(Fast Thermal Model) | -41.3358 | 111545 | 91.97 | 20782 | -50.2010 | 231859 | 92.82 | 9172 | -7.7890 | 19067 | 76.16 | 9984

  * *

takes a similar amount of time as training RLPlanner for 600 epochs.

##  II Methodology

###  II-A Overall Architecture of RLPlanner

As shown in Fig. 1, the overall architecture of RLPlanner consists of three
main parts: the floorplanning environment for chiplets, the RL-based agent,
and a thermal-aware reward calculator. The environment updates the action mask
$M_{t}$ and state $s_{t}$ based on previously placed chiplets at each time
step $t$. The agent generates the probability matrix of actions
$\pi_{\theta}(a_{t}|s_{t})$ and the expected reward $V(s_{t})$ by the policy
and value networks, correspondingly. Before sampling the action $a_{t}$ that
places chiplets sequentially, the probability of infeasible actions will set
to be ’0’ based on $M_{t}$. Once all chiplets have been placed, the reward
calculator will first perform microbump assignments to minimize the total
wirelength by allocating pin locations for each inter-chiplet connection, and
then conduct the thermal evaluation.

###  II-B Reinforcement Learning and Agent Architecture

In our agent architecture, the policy network and the value network share the
same feature encoding CNN layers and two separate fully connected layers are
used to get the probability matrix and expected reward. We employ the proximal
policy optimization (PPO) [9] algorithm to train the networks. Moreover, we
employ a random network distillation (RND) [10] bonus to encourage the agent
to explore novel states. It involves two neural networks: a fixed and randomly
initialized target network, and a predictor network trained on data collected
by the agent.

###  II-C Reward Calculations

TABLE II: Accuracy and speed comparison during thermal evaluation

Metrics | Fast Thermal Model | Hotspot
---|---|---
MSE | 0.1732 K |  Ground Truth
RMSE | 0.4162 K
MAE | 0.2523 K
MAPE | 0.0726%
| Inference
---
Speed
| 0.1012 s
---
(127$\times$)
12.8976 s

The target of chiplet floorplanning is to minimize the total wirelength and
maximum operating temperature. We customize the reward function as
$R=-\lambda\times
W-\mu\times\frac{(max(T-T_{0},0))^{\alpha}}{1+e^{-(T-T_{0})}}$ where $W$ and
$T$ are the wirelength and temperature, $\lambda$ and $\mu$ are the weights,
and $T_{0}$ is the temperature limitation and $\alpha$ is a hyperparameter to
avoid gradient non-smoothness at $T=T_{0}$. The microbump assignments and
wirelength optimization can be found in [6]. Traditional thermal analysis
simulator HotSpot [11] is CPU-intensive since it involves constructing a
thermal model for the entire system and solving large linear equations
repeatedly. To address the computational burden while maintaining accurate
thermal analysis, a physical-informed fast thermal model is proposed. It
simplifies the thermal resistance network structure by treating it as an
linear and time-invariant (LTI) system. By calculating the self-thermal and
mutual-thermal resistance in the thermal resistance network, the chiplets’
temperatures can be obtained. Thus, we characterize the self-thermal
resistance by setting a chiplet’s power to a non-zero value and run HotSpot to
create a 2D self-thermal resistance table, and characterize the mutual-thermal
resistance by a 1D table with respect to the distance between power source and
grid location. More details can be found at the github.

##  III Experimental Results

###  III-A Quantitative evaluations of the fast thermal model

A dataset comprising 2,000 synthetic chiplet systems are used to conduct
thorough comparisons between the proposed fast thermal model and Hotspot. The
results are summarized in TABLE II, where mean square error (MSE), root mean
square error (RMSE), mean absolute error (MAE) and mean absolute percentage
error (MAPE) are calculated. The comparisons demonstrate that the proposed
fast thermal model can accurately estimate the maximum temperature of a
chiplet-based system and achieve a speedup over 127x during thermal
evaluations compared to Hotspot.

TABLE III: Comparisons of reward on 5 synthetic systems

| Reward
---|---
Method | Case1 | Case2 | Case3 | Case4 | Case5
RLPlanner | -5.8288 | -6.3236 | -10.0058 | -8.4076 | -8.6193
RLPlanner(RND) | -5.1062 | -6.7848 | -9.9335 | -8.3903 | -8.2049
TAP-2.5D(HotSpot) | -6.6439 | -8.9846 | -12.3946 | -10.5525 | -10.6965
TAP-2.5D*(Fast Thermal Model) | -6.3627 | -7.1250 | -10.7151 | -9.8286 | -8.5189

  * *

takes a similar amount of time as training RLPlanner for 600 epochs.

###  III-B Performance evaluations of RLPlanner

Three open-source benchmarks and five synthetic systems are used to evaluate
the performance of RLPlanner. Comparisons and analysis between the developed
RLPlanner and TAP-2.5D [6], an SA-based thermal-aware floorplanning algorithm,
are presented in TABLE I and TABLE III. Across all eight cases, RLPlanner with
RND achieves an average improvement of 20.28% in optimization goals compared
to TAP-2.5D with HotSpot, and an average improvement of 9.25% compared to
TAP-2.5D with the fast thermal model within similar or less running times. It
confirms our intuition that by conducting end-to-end co-optimizations,
RLPlanner shows better efficiency and quality in optimizing chiplet-based
systems.

## References

  * [1] T.-C. Chen and Y.-W. Chang, “Modern floorplanning based on b/sup*/-tree and fast simulated annealing,” _IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems_ , vol. 25, no. 4, pp. 637–650, 2006.
  * [2] A. Coskun _et al._ , “A cross-layer methodology for design and optimization of networks in 2.5 d systems,” in _2018 IEEE/ACM International Conference on Computer-Aided Design (ICCAD)_. IEEE, 2018, pp. 1–8.
  * [3] F. Eris _et al._ , “Leveraging thermally-aware chiplet organization in 2.5 d systems to reclaim dark silicon,” in _2018 Design, Automation & Test in Europe Conference & Exhibition (DATE)_. IEEE, 2018, pp. 1441–1446.
  * [4] V. A. Chhabria _et al._ , “Thermal and ir drop analysis using convolutional encoder-decoder networks,” in _Proceedings of the 26th Asia and South Pacific Design Automation Conference_ , 2021, pp. 690–696.
  * [5] L. Chen _et al._ , “Fast thermal analysis for chiplet design based on graph convolution networks,” in _2022 27th Asia and South Pacific Design Automation Conference (ASP-DAC)_. IEEE, 2022, pp. 485–492.
  * [6] Y. Ma _et al._ , “Tap-2.5 d: A thermally-aware chiplet placement methodology for 2.5 d systems,” in _2021 Design, Automation & Test in Europe Conference & Exhibition (DATE)_. IEEE, 2021, pp. 1246–1251.
  * [7] A. Kannan _et al._ , “Enabling interposer-based disintegration of multi-core processors,” in _Proceedings of the 48th international symposium on Microarchitecture_ , 2015, pp. 546–558.
  * [8] Huawei ascend 910 provides an nVidia AI training alternative. [Online]. Available: https://www.servethehome.com/huawei-ascend-910-provides-a-nvidia-aitraining-alternative/
  * [9] J. Schulman _et al._ , “Proximal policy optimization algorithms,” _arXiv preprint arXiv:1707.06347_ , 2017.
  * [10] Y. Burda _et al._ , “Exploration by random network distillation,” _arXiv preprint arXiv:1810.12894_ , 2018.
  * [11] W. Huang _et al._ , “Hotspot: A compact thermal modeling methodology for early-stage vlsi design,” _IEEE Transactions on very large scale integration (VLSI) systems_ , vol. 14, no. 5, pp. 501–513, 2006.


# The NPC AI of The Last of Us: A case study

Harsh Panwar
School of ELectronic Engineering and Computer Science
Queen Mary University of London
London
<EMAIL_ADDRESS>

###### Abstract

Abstract: The Last of Us is game focused on stealth, companionship and
strategy. The game is based in a lonely world after the pandemic and thus it
needs AI companions to gain the interest of players. There are three main NPCs
the game has - Infected, Human enemy and Buddy AIs. This case study talks
about the challenges in front of the developers to create AI for these NPCs
and the AI techniques they used to solve them. It also compares the challenges
and approach with similar industry-leading games.

_K_ eywords NPC, GameAI, Infected, Ellie, PS4

##  1 Introduction

The last of us is a third-person shooter (TPS) action-adventure made by
Naughty Dog and distributed by Sony Computer Entertainment developed majorly
for PlayStation 3 and later on remastered for PlayStation 4 in 2014 [1]. Since
it’s release the game has received amazing reviews by game developer critics
[2] [3] [4] as well as by the gaming community and is considered as the best
game of the decade as per Metacritic [5].

The game is set in post-apocalyptic America after the parasitic Cordyceps
fungus [6] has wiped out majority of the humanity as we know it and divided
the entire world into the infected and the survivors. In the nature this
species of fungus [7] can be seen attacking on the ants and taking control of
their brains [8] forcing the ants to lose control and become a useless
creature with only one job left - become host for the fungus, generating a
massive sprout which eventually shoots out of their head and infect others
eventually. Inspired by this natural phenomena, the creators of The last of Us
thought of a scenario where a similar fungus affected the human body.

This paper is further organized into various sections where Section
2:Challenges summarizes the various challenges faced by developers while
creating the NPC AI. Section 3:Skills and Behaviours discusses the Skills and
Behaviours AI system used in the game, Section 4:AI Perception presents the AI
perception used and compares it with other games. Section 5:Follow Positions
presents the AI technique for follow positions. Section 6:Cover and Posts
presents the AI technique used for cover and parts. Finally, Section
7:Conclusion concludes this paper.

##  2 Challenges

There are mainly two types of people left after the pandemic has affected the
entire world in The Last of Us \- The Infected and the Survivors. The pandemic
in The Last of Us has left most of the population disfigured and aggressive
and those who have been overcome by the fungus are declared as Infected. The
infected lacks any humanly characteristics such as compassion or self-
preservation and are mainly driven by the instincts of the fungus. Then there
are the survivors who survived the pandemic and now lives in quarantine zones
and are at constant threats of the Infected and other predatory survivors
known as Hunters. And there are also half a dozen of survivors which are
companions of Joel, the main character, and help him travel in the lonely
world. The developers had to majorly work on these three groups - The
Infected, The Human Enemy and the Buddy companions. The developers chose to
develop them using AI as such a system creates an enviornment closest to the
real world. The players want the non-player characters (NPCs) in the game to
be intelligent and just a simple glitch like an NPC running into a wall or
trying to hide in plain sight can result in player’s disinterest in the game.
In a study done by Tremblay et. al. [9] they found that the adaptive AI
companion has more influence over the player’s experience and it is a step
forward in developing meaningful and engaging games. But each of these groups
have there own challenges when it comes to developing the AI for them and they
are discussed below in brief:

(a) Ellie, a 14-year old teenage girl who accompanies Joel in most part of the
game and is dependent on him.

(b) Elizabeth, who reveals special abilities and has a complex background
story.

Figure 1: Screenshot from the games showing the companion AIs in (a) The Last
of Us and (b) Bioshock: Infinite

###  2.1 The Infected

One basic understanding of AI in Games is to make the NPCs intelligent and
smarter. This simply can’t be done for the Infected AI as they have been made
dumb by the fungus. This was one of the biggest challenges for the AI
Developers in The Lat of Us as they had to make them seem chaotic and alien
unlike Hunters which work in groups and coordinate with each other. Our
understanding of intelligence is more closely related to humanly behaviours
such as togetherness, coordination, expressions etc. which were not allowed to
be use for the Infected.

Figure 2: infected-canvass behaviour

The Infected also because of the fungus were left to be sightless and had to
rely mostly on hearing. This made it even more difficult to make them seem
more real as otherwise it’ll become very easy to tackle them and the player
could lose interest in the game. Just like in the actual world if you become
blind or lose one of the 5 senses then the other senses for example hearing
becomes stronger [10], the developers had to make sure that their AI is really
smart when it comes to actions and reactions using the ability of hearing.

###  2.2 Human enemy

The primary challenge while creating the Human AI was how to make an NPC so
impactful that killing that NPC will have a effect on the player which will
make the game interesting. Unlike in Alien:Isolation where the Xenomorph,
primary antagonist, has two main AIs - main control itself using behavior
trees which is very common in AAA games and an AI-director just like Left 4
Dead [11] which mainly manages the pacing of the game and the main focus was
on Sense, Menace and Speed [12]. The Human Enemy AI in The Last of Us had to
be focused mainly on Cover, Stealth and Flanking to make them look more like
real humans and work in groups to give a feel of fighing against actual
humans.

###  2.3 Buddy companions

The companion AI is one of the key factor in making a game like The Last of Us
interesting since in a post-apocalyptic world which is full of Infected
alienated creatures it’s essential that the only friend we have is intelligent
and also compassionate. In an online survey done by Emmerich et. al. [13] on
players they found out that most players prefer to have a companion and the
most important charateristics they look in the companion are - Personality
$(M=3.82)$, Context Sensitivity $(M=3.53)$ and Story Significance $(M=3.70)$
on a Mean scale: $0-4$. Elizabeth is a companion AI in Bioshock: Infinte [14]
but is very different from Ellie in a way that Elizabeth has special abilities
to help the player while Ellie is more dependent on the player. Since in The
Last of Us the characters lacks any superpower or mysterious abilities, Ellie
who’s just a 14-year old teenager is very difficult to be developed as someone
who the player will find useful and not a liability who has to be escorted.
Making a character that we as a player cared for was the key to the success of
the game.

The pandemic in The Last of Us has left most of the population disfigured and
aggressive and those who have been overcome by the fungus are declared as
Infected. Although the AI for the Infected and the Hunters are similar, the
developers have made the architecture modular which helps to make changes to
the decision-making logic which makes the Infected feel fundamentally
different than the Hunters [15].

##  3 Skills and Behaviours

The AI System used in The Last of Us mainly comprises of high-level decision
logic known as the skills and low-level capabilities known as behaviours. The
skills are used to decide what the character should do while the behaviours
are used to implement those decisions. This sort of model helps in re-using
the same low-level behaviours for multiple high-level characteristics. The
skills are stacked in a priority queue which tells us which skills can
interrupt other skills. The examples for skills for the infected are - chase,
search, follow, sleep, wander, etc. and for human enemy the skills are -
panic, advance, melee, gun combat, hide, investigate, scripted and flank.
Since the skills are high-level they don’t change the animations or
pathfinding systems. These changes are instead made by behaviour object using
a behaviour stack. Examples of behaviour are MoveToLocation, StandAndShoot and
TakeCover.

Most of the skills are common between the different characters for ease of
adding new characters at any stage of development. There are 4 types of
Infected - Runner, Stalker, Clicker and Bloater and only two skills are unique
to single characters - ambush skill by the stalker and throw skill by the
bloater. Now we will explain some of the major skills in detail:

###  3.1 Search Skill

The hunters are more intelligent beings and can communicate with each other to
gather knowledge about the player’s position. The infected on the other hand
lacks this and also can hardly see. So a requirement was there for a special
search skill for the infected. This skill is in low priority compared to the
chase skill and thus becomes valid when the infected loses track of the player
during chase phase. The search by the infected should look unplanned and
random while also covering a large amount of area. The search points [16] can
be anywhere on the navigation mesh and hence the search area scan be of
arbitary size and shape. To solve this infected-canvass behaviour was
developed as shown in fig 2 and the steps are explained below:

  1. 1.

With the Infected in the center we place a logical grid on the covered area by
the canvass radius.

  2. 2.

Any obstacle that comes in the way and the entire area outside is fixed as
seen and the rest of the cells needs to be checked.

  3. 3.

Multiple possible animations are given to the behaviour by the invoking
skills. Location and orientation of the Infected after these animations is
looked at.

  4. 4.

The number of unseen cells is computed for every animation.

  5. 5.

We play the most desirable animation generally by counting the number of
unseen cells and ignore the animations that were played recently.

  6. 6.

After the completion of the wedges they are marked as seen and the process is
repeated from step 3.

##  4 AI Perception

AI perception is one of the key factor in determining the success of any Game
AI. Especially in a game like The Last of Us which is heavily based on stealth
it’s important to know the position of other characters. The developers
initially decided to use Vision Cone which is a very common way of visualising
how the enemy sees the world and has been used in games like The Uncharted
[17] and Alien: Isolation. As seen in Fig. 3 using a Vision Cone is very
effective in finding player which are at a distance but fails to register
player which is at close proximity to the NPC or standing right next to the
NPC. This lead to the NPC acting very dumb and unaware of the surrounding
which made it easy to full them and lead to the players losing interest in the
game. One way to solve this is to have multiple Vision cone similar to the
ones used in the Alien: Isolation Xenomorph AI[12]. It uses 4 different view
cones namely - normal, focused, peripheral and close as seen in Fig. 4(a). The
developers of The Last of Us came up with an even better and complex form of
Vision cone with a simple rule that the angle of view is inversely
proportional to the distance between the NPC and the player as seen in the
Fig. 4(b).

Figure 3: The initial vision cone developed for The Last of Us which was
unable to look at players standing right next to the NPC.

(a) Four Vision cone used for the Xenomorph in the Alien: Isolation \- normal
(Blue), focused (Red), peripheral (Green) and close (Pink).

(b) The final complex vision cone used in The last of Us which is easily able
to see the player in close proximity.

Figure 4: Screenshot from the games showing the companion AIs in (a) The Last
of Us and (b) Bioshock: Infinite

##  5 Follow Positions

It’s essential for the Buddy AI, Ellie, to follow the player and stay close to
him to not be responsible for raising any alerts to the enemy or even if
something bad happened it can be related to the players on mistake since they
both were close to each other and the player shouldn’t have been that exposed
in the first place. To bring this in the game a follow system was developed
which generates a number of candidate follow positions which are evaluated for
quality. This generation was done by casting raycasts as seen in Fig. 5.

  * •

First the raycasts are sent from the player to the follow region to make sure
of a clear path between the player and the Buddy AI. Every raycasts that
sucessfully reaches the follow region is considered as a candidate. See fig.
5(a)

  * •

Secondly new set of rays are shooted forward from every candidate to avoid any
walls. See fig. 5(b)

  * •

And finally from the players position rays are cast to each forward position
to ensure obstacle free-movement. See fig. 5(c)

Figure 5: Pathfinding raycasts for follow positions (a) for generating
candidate positions (b) for checking forward location (c) for checking future
position Figure 6: Raycasts being generated for coverposts. Figure 7: Some of
the best cover could have a path which leads to the player before reaching the
cover. Figure 8: Ranking for each posts calculated based on AI criterion
written in LISP.

##  6 Cover and Posts

The Last of Us is a cover-based TPS [18] and the main focus is on stealth
based attacking and strategising each move. For the AI to look and feel human-
like it’s a very important decision of where to stand or take cover and is a
fairly complex problem. For establishing proper covers we first need to
recognise a set of potential locations which we’ll call posts. There are two
different kind of posts \- cover and open. The cover posts were formed around
the NPC’s location while the open posts were around the player. Each NPC would
be assigned 20 cover posts and in each frame 160 raycasts were thrown to them.
The posts where every ray was rejected out of the 4 were rejected as shown in
Fig. 6 The posts an NPC should use would be determined by different criterion
written in LISP [19] using post selectors. After the development of the game
there were 17 different post selectors with the most important one being ai-
criterion-static-pathfind-not-near-player [20]. This was used when the path to
the post required the human enemy to cross the player as shown in Fig. 7. This
wouldn’t make sense and will lead to the player easily killing the human enemy
and making the difficulty easier. Each of the criterion would give some value
in float numbers and the product of the criteria for a given post selector and
a given post in a value which would be used as the rating for that post. Then
all the posts would be ranked accordingly and the one with the highest score
would be chosen for that NPC as shown in Fig. 8.

##  7 Conclusion

The Last of Us was a complete success and received love from the players
mostly because of how close to reality the game tried to go thanks to it’s AI.
The game was in a sense first of it’s kind where the difficulty of the game
didn’t lie in making the characters hard to kill but by increasing the co-
ordination between the NPCs and thus increasing the intelligence in the game
characters. Through the AI used in Ellie the players were able to create an
actual relationship with her and each and every gesture performed by her made
a meaning to the player. This leads us to question what we consider as human-
like and can we make a companion AI in video games that will feel like an
actual human being. The level of determination that the developers of The Last
of Us went to can be seen when they talk about "her personal space" and "her
choices" in one of the papers they have written on the character. The
developers also avoided cheating in most ways with things like only using
Teleportation when extremely necessary and in a way that the player don’t
realize makes us understand that why the game feels so engaging to the
players.

## References

  * Dog [2013] Naughty Dog.  The last of us.  _Sony Computer Entertainment_ , 2013.
  * [2] hrej.cz.  The last of us.  URL https://hrej.cz/article/the-last-of-us-3.
  * Rybář [2013] Václav Rybář.  The last of us - recenze, Jun 2013.  URL https://games.tiscali.cz/recenze/the-last-of-us-recenze-225826.
  * iDNES.cz [2013] iDNES.cz.  Recenze: The last of us je, Jun 2013.  URL https://www.idnes.cz/hry/recenze/the-last-of-us-recenze.A130605_094544_bw-ps3-recenze_vdp.
  * Harradence [2020] Michael Harradence.  The last of us is metacritic users’ game of the decade, Jan 2020.
  * Stark [2022] Chelsea Stark.  The creepy, real science behind ’the last of us’, Mar 2022.  URL https://mashable.com/archive/the-last-of-us.
  * Evans and Samson [1982] Harry C Evans and Robert A Samson.  Cordyceps species and their anamorphs pathogenic on ants (formicidae) in tropical forest ecosystems i. the cephalotes (myrmicinae) complex.  _Transactions of the British Mycological Society_ , 79(3):431–453, 1982.
  * Hywel-Jones [1996] Nigel L Hywel-Jones.  Cordyceps myrmecophila-like fungi infecting ants in the leaf litter of tropical forest in thailand.  _Mycological Research_ , 100(5):613–619, 1996\.
  * Tremblay and Verbrugge [2013] Jonathan Tremblay and Clark Verbrugge.  Adaptive companions in fps games.  _FDG_ , 13:229–236, 2013.
  * Karns et al. [2012] Christina M. Karns, Mark W. Dow, and Helen J. Neville.  Altered cross-modal processing in the primary auditory cortex of congenitally deaf adults: A visual-somatosensory fmri study with a double-flash illusion.  _Journal of Neuroscience_ , 32(28):9626–9638, 2012.  ISSN 0270-6474.  doi: 10.1523/JNEUROSCI.6488-11.2012.  URL https://www.jneurosci.org/content/32/28/9626.
  * Kitson [2009] Niall Kitson.  Left 4 dead.  _Irish Gothic Journal_ , (6):76, 2009.
  * Tommy ThompsonBloggerMay 20 [2020] 2020 Tommy ThompsonBloggerMay 20.  Revisiting the ai of alien: Isolation, May 2020.  URL https://www.gamedeveloper.com/design/revisiting-the-ai-of-alien-isolation.
  * Emmerich et al. [2018] Katharina Emmerich, Patrizia Ring, and Maic Masuch.  I’m glad you are on my side: How to design compelling game companions.  In _Proceedings of the 2018 Annual Symposium on Computer-Human Interaction in Play_ , CHI PLAY ’18, page 141–152, New York, NY, USA, 2018. Association for Computing Machinery.  ISBN 9781450356244.  doi: 10.1145/3242671.3242709.  URL https://doi.org/10.1145/3242671.3242709.
  * [14] 2k.  Bioshock-infinte.  URL https://2k.com/en-US/game/bioshock-infinite/.
  * Botta [2019] Mark Botta.  Infected ai in the last of us.  In _Game AI Pro 360: Guide to Character Behavior_ , pages 1–12. CRC Press, 2019.
  * Straatman et al. [2006] R Straatman, A Beij, and W Sterren.  Dynamic tactical position evaluation.  _AI Game Programming Wisdom_ , 3(5.2):389–403, 2006.
  * Dog [2015] Naughty Dog.  _The art of the Uncharted Trilogy_.  Dark Horse Comics, 2015.
  * Dyckhoff [2019] Max Dyckhoff.  Ellie: Buddy ai in the last of us.  In _Game AI Pro 360: Guide to Character Behavior_ , pages 25–36. CRC Press, 2019.
  * Steele [1990] Guy Steele.  _Common LISP: the language_.  Elsevier, 1990.
  * McIntosh [2015] Travis McIntosh.  Human enemy ai in the last of us.  _Game AI Pro 2: Collected Wisdom of Game AI Professionals_ , page 419, 2015.