Date of current version July 6, 2023.

This work was supported by the Project Augmented Humanity
[POCI-01-0247-FEDER-046103], financed by Portugal 2020, under the
Competitiveness and Internationalization Operational Program, the Lisbon
Regional Operational Program, and by the European Regional Development Fund.
This work has been submitted to the IEEE for possible publication. Copyright
may be transferred without notice, after which this version may no longer be
accessible.

Corresponding author: Manuel Gomes (e-mail: manuelgomes@ua.pt).

# Volumetric Occupancy Detection: A Comparative Analysis of Mapping Algorithms

Manuel Gomes1,2  Miguel Oliveira1,2  and Vítor Santos1,2  Intelligent System
Associate Laboratory (LASI), Institute of Electronics and Informatics
Engineering of Aveiro (IEETA), University of Aveiro, 3810-193 Aveiro, Portugal
Department of Mechanical Engineering, University of Aveiro, 3810-193 Aveiro,
Portugal

###### Abstract

Despite the growing interest in innovative functionalities for collaborative
robotics, volumetric detection remains indispensable for ensuring basic
security. However, there is a lack of widely used volumetric detection
frameworks specifically tailored to this domain, and existing evaluation
metrics primarily focus on time and memory efficiency. To bridge this gap, the
authors present a detailed comparison using a simulation environment, ground
truth extraction, and automated evaluation metrics calculation. This enables
the evaluation of state-of-the-art volumetric mapping algorithms, including
OctoMap, SkiMap, and Voxblox, providing valuable insights and comparisons
through the impact of qualitative and quantitative analyses. The study not
only compares different frameworks but also explores various parameters within
each framework, offering additional insights into their performance.

###### Index Terms:

Volumetric Detection; Human-robot Collaboration; ROS; OctoMap; SkiMap;
Voxblox;

=-21pt

##  I Introduction

The field of human-robot collaboration is experiencing rapid advancements,
leading to the introduction of new functionalities and interfaces. One
prominent area of development involves the use of visual interfaces for
gesture recognition. Researchers have explored various approaches, including
those utilizing RGB [1, 2], Depth [3] or Stereo cameras [4] to capture and
interpret human gestures. Another avenue of exploration in human-robot
collaboration is the utilization of audio-based functionalities. Some
researchers have focused on transforming speech into text [5] or detecting
emotions from speech [6]. Combining both visual and audio cues, certain
authors have investigated the fusion of images and audio for speech [7] and
emotion recognition [8]. In addition to visual and audio interfaces,
electromyography interfaces have emerged as a promising approach for gesture
recognition in human-robot collaboration. These interfaces leverage the
measurement of muscle activity signals to detect and interpret human gestures
[9, 10, 11]. Furthermore, advanced functionalities in human-robot
collaboration encompass the handover of objects between humans and machines.
Researchers have proposed innovative strategies and techniques to facilitate
smooth and efficient object transfer [12, 13, 14].

While these functionalities and interfaces are intriguing from a research
perspective, it is important to note that they are not indispensable for
collaboration to take place. However, an essential functionality that
underpins collaboration scenarios is volumetric detection. Volumetric
detection provides a crucial security feature by enabling real-time detection
of foreign objects within a workspace, including their current pose. While 1-D
or 2-D barriers can detect the entrance of foreign objects, volumetric
detection creates a proliferate environment for collaboration, enhancing
safety and efficiency in collaborative settings.

Currently, there is a lack of widely adopted volumetric detection frameworks
specifically designed for human-robot collaboration. Mapping frameworks serve
as the foundation for most volumetric detection systems since volumetric
detection is required for mapping purposes. However, the evaluation of mapping
frameworks often lacks comprehensive and insightful metrics, with performance
assessments primarily focused on time and memory efficiency indicators [15,
16, 17].

This paper presents aims to address the aforementioned gaps in research. The
objectives of this study are to:

  * •

Create a simulation environment capable of testing the capabilities of various
volumetric detection frameworks;

  * •

Establish a methodology to extract ground truth data from the simulation
environment;

  * •

Apply evaluation metrics and automate their calculation to enable a
comprehensive evaluation;

  * •

Provide insights into the performance of well-known mapping algorithms, namely
OctoMap, SkiMap, and Voxblox, and identify optimal parameter settings for
these frameworks.

The article is organized in five sections to comprehensively address these
objectives. Introduction (section I) presents the purpose of this research and
highlights the contributions of the authors. Related Work (section II)
provides a detailed description of OctoMap, SkiMap, and Voxblox, including
their potential advantages and disadvantages. Proposed Approach (section III)
outlines the testing environment, ground truth extraction methodology, and
evaluation metrics in a comprehensive manner. Results (section IV) offers an
in-depth description of the experimental methodology employed, along with
extensive qualitative and quantitative results. Finally, Conclusions (section
V) summarizes the approach and its advantages, while also delineating avenues
for future research and development.

##  II Related Work

This section provides an overview of three open-source mapping frameworks,
namely OctoMap, SkiMap, and Voxblox. These mapping frameworks have played a
crucial role in advancing the field of 3D mapping, empowering robots and
autonomous systems to navigate and operate with efficiency and accuracy in
complex, spatial environments.

OctoMap is a mapping framework that utilizes a tree-based representation known
as octrees to provide high flexibility in terms of mapped area and resolution
[15]. Octrees are hierarchical data structures for spatial subdivision in
three dimensions [18]. Each octree consists of nodes that represent cubic
volumes of space. In OctoMap, this tree is dynamically updated in real-time
based on sensor measurements or observations. The tree begins with a single
root node representing the entire space. Then, the node is recursively divided
into eight more nodes until it reaches a predefined minimum voxel size,
thereby discretizing the three-dimensional space into a hierarchical grid. An
example of an octree representation is depicted in Figure 1.

Figure 1: An octree that stores free cells, shaded in white, and occupied
cells, colored black. The left side displays the volumetric model, while the
right side exhibits the corresponding tree representation [15].

The basic form of an octree can be used to model a Boolean property, where
each node can have two states: occupied volume or free volume. In the case of
volumetric detection, a node is initialized if a certain volume is occupied.
However, this approach creates ambiguity by failing to differentiate between
unknown and free voxels. To address this issue, OctoMap explicitly represents
free volumes in the tree by employing raycasting. This technique assumes the
volumes between the sensor and the measured endpoint as free space. A visual
depiction of raycasting is presented in Figure 2. By adopting this
representation, the framework achieves a compact environment representation by
pruning all children of the same node if they have the same state.

Figure 2: A laser scanner scans a flat surface at a shallow angle by rotating.
In the first scan (top), a cell is detected as occupied, but in the subsequent
scan (bottom) after the rotation of the sensor, the same cell is updated as
free. Occupied cells are depicted as gray boxes, while free cells are
represented in white [15].

In scenarios involving sensor noise and dynamic environments, the Boolean tree
representation is inadequate. Therefore, OctoMap incorporates a probabilistic
model to represent occupancy. In this model, each node is associated with a
probability value indicating the likelihood of a voxel being occupied. A
threshold is often applied to classify a voxel as occupied or free based on
its probability value. OctoMap adopts a holistic approach by considering the
complete unfolding of a scene, rather than relying solely on time snapshots.
Consequently, it incorporates multiple observations to determine the present
state of a voxel. For instance, when a voxel has been observed as unoccupied
in $k$ instances, a minimum of $k$ observations of occupancy is required to
transition it from an unoccupied to an occupied state. This characteristic
strengthens the framework’s resilience to sensor noise, facilitates temporal
scene analysis, and enables sensor fusion. Nevertheless, in dynamic
environments, this behavior becomes undesirable as it hinders the algorithm’s
responsiveness. To address this issue, OctoMap proposes a clamping update
policy that establishes upper and lower bounds on the number of observations
needed to change the state. In summary, OctoMap provides a powerful framework
for 3D occupancy mapping, offering efficient storage, real-time updates, and
probabilistic representation. Its octree-based structure enables accurate
modeling of the environment, making it a valuable tool for robots and
autonomous systems operating in three-dimensional spaces.

SkiMap is a mapping framework, akin to OctoMap, that employs a tree of
SkipLists as its underlying data structure [16]. SkipLists are layered
probabilistic data structures [19]. The first layer comprises an ordered
linked list, and each subsequent layer is a subset of the layer below it,
encompassing a fraction of the elements. This design reduces the computational
complexity associated with random access.

In SkiMap, the tree of SkipLists consists of multiple levels, where the first
SkipList tracks quantized x coordinates, representing depth level 1, and the
individual elements in this SkipList are referred to as xNodes. Each xNode
itself corresponds to a SkipList that tracks quantized y coordinates,
representing depth level 2, with its elements named yNodes. Finally, each
yNode corresponds to a SkipList of zNodes, representing the actual voxels and
serving as containers for user data. As a result of this data structure, voxel
coordinates can be obtained by traversing predecessors, obviating the need to
store coordinates within the containers alongside user data. This tree of
SkipLists is better visualized in Figure 3.

Figure 3: Voxel grouping process into a Tree of SkipLists. Each voxel,
depicted as a blue box, is connected to the rootNode through a yNode,
represented by a green tile, and the yNode is further linked to an xNode,
depicted as a red tile [16].

To classify voxels as occupied or free, Each voxel within the tree of
SkipLists possesses a dynamically defined weight. When a detection occurs
within a voxel, its weight increases, while it decreases when no detection is
present. If the weight surpasses a predefined threshold known as the minimum
voxel weight, the voxel is classified as occupied. The authors of SkiMap argue
that their methodology leads to reduced runtime and memory usage compared to
OctoMap.

Voxblox is a mapping framework similar to the previously discussed OctoMap and
SkiMap [17]. Originally developed for Micro Aerial Vehicles operating in
unstructured and unexplored environments, Voxblox provides a fast and flexible
local mapping solution.

This framework is based on the creation of Euclidean Signed Distance Fields
derived from Truncated Signed Distance Fields. Both ESDFs and TSDFs are data
structures built upon Signed Distance Fields, which are voxel grids that
divide the 3D space. Each voxel is assigned a projective distance label
representing the distance between the voxel and the nearest detected surface.
Positive and negative values indicate whether the voxel is located outside or
inside the surface, respectively.

In TSDFs, a truncation distance is defined to limit the storage of distances,
optimizing memory usage and processing speed by focusing on the relevant parts
of the environment. A TSDF representation is presented in Figure 4. In
contrast, ESDFs employ the Euclidean distance between the voxel and the
nearest obstacle instead of the projective distance. Consequently, ESDFs are
particularly useful for navigation purposes.

Figure 4: 2D cross-section of a general Truncated Signed Distance Field (TSDF)
computed on a grid of regular voxels, with each arrow representing the
distance vector of a voxel [20].

Voxblox calculates TSDFs by employing raycasting and averaging the newly
measured projective distances into the existing voxels. For each detection, a
weighting function is applied, which decreases quadratically with the distance
from the sensor to the surface and linearly with the distance from the voxel
to the surface. Subsequently, Voxblox derives ESDFs from TSDFs using a method
based on occupancy maps [21]. This method utilizes wavefronts, propagating
waves that update distances from the start voxel to its neighbors. Updated
voxels are added to the wavefront queue for further propagation. Voxblox
employs two wavefronts: a raise wavefront and a lower wavefront. The raise
wavefront is triggered when the new distance value of a voxel from the TSDF
exceeds the previously stored value in the corresponding ESDF voxel.
Invalidation of the voxel and its children follows, as they are added to the
raise queue. The wavefront continues propagation until no voxels with
invalidated parents remain. Conversely, the lower wavefront initiates when a
new occupied voxel enters the map or when a previously observed voxel
decreases its value. Based on the current voxel, its neighbors, and their
respective distances, the distances of neighboring voxels are updated. The
wavefront propagation terminates when there are no remaining voxels whose
distances could decrease based on their neighbors.

The authors of Voxblox argue that constructing a TSDF is faster than building
an OctoMap, and constructing an ESDF from a TSDF is faster than building it
from an occupancy map.

##  III Proposed Approach

The frameworks described in the previous section are usually assessed using
only basic metrics, such as time of integration of voxel grid or memory usage
[16, 17]. While these metrics serve as valuable indicators of efficiency, they
prove insufficient for volumetric representation. The existing body of
research lacks a comprehensive measurement approach that adequately assesses
the alignment between the outputs of frameworks and the actual objects being
detected. Consequently, the authors of this study have undertaken the
development of a tool specifically designed to carry out such evaluation. The
subsequent sections will provide a detailed description of this tool.

###  III-A Testing Environment

The development of a simulation-based testing environment by the authors was a
strategic choice, primarily driven by the objective of acquiring ground truth
values, which cannot be feasibly obtained in a real environment. Although this
decision effectively eliminated the challenges associated with hardware-
related issues encountered in testing with physical sensors, its primary
motivation remains the acquisition of accurate reference data.

Gazebo111https://gazebosim.org, an open-source robotics simulator, is used to
construct their simulation environment. It was selected due to its attributes,
including a highly accurate physics engine that faithfully replicates object
behavior. Furthermore, Gazebo has the capability to simulate multiple sensors
concurrently, allowing for the introduction of sensor noise. A noteworthy
feature of Gazebo is its capacity to animate static models, referred to as
actors, which plays a vital role in creating realistic environments. Moreover,
Gazebo’s integration with the Robot Operating System (ROS) is particularly
significant. ROS, an open-source framework, offers a comprehensive set of
software libraries and tools that are instrumental in the development,
control, and integration of robotic systems. This integration is important
because all the algorithms employed in this study are ROS-integrated. This
seamless integration significantly enhances the ease-of-use and compatibility
of the simulation environment with the algorithms, ensuring an efficient
workflow.

The simulated environment was based on LARCC, a collaborative cell equipped
with three LiDARs, one RGB-D camera, and three RGB cameras [22]. This real-
life setup can be observed in Figure 5. The environment includes four sensors
suitable for volumetric detection: the three LiDARs and the RGB-D camera. The
dimensions of the cell are 4 meters in length, 2.8 meters in width, and 2.29
meters in height.

Figure 5: LARCC, an advanced collaborative cell embedded with an extensive
array of sensory mechanisms.

The purpose of this environment is to test the ability of occupancy algorithms
to promptly detect a person entering the cell. To simulate this scenario
accurately, a Gazebo actor representing a person was spawned in the middle of
the cell. The actor moves in an elliptical pattern along the center of the
cell, with a semi-major axis of 2 meters aligned with the length of the cell
and a semi-minor axis of 1 meter aligned with the width. The actor moves in
this pattern two times, totaling into an episode of around 96 seconds. In the
center of this elliptical movement, a rectangular plate measuring 0.8 meters
in length, 0.6 meters in width, and 1 meter in height was placed. This plate
serves to evaluate the algorithms’ performance when the moving object is
partially occluded. The environment can be seen in detail in Figure 6.

Figure 6: Testing environment, representing LARCC. The cell contains 3 RGB
cameras, 3 LiDARs and 1 RGB-D, signaled by red, yellow and blue ellipses,
respectively. The actor is presented as a human with a green shirt.

###  III-B Data Filtering and Retrieval

The testing environment produces a point cloud for each range-finder sensor.
To conduct an evaluation, it is essential to obtain the ground truth data that
correspond to the actor.

The fusion of point clouds from multiple sensors is a prerequisite for
accurate evaluation. To achieve this, an approximate time filter is
implemented222https://wiki.ros.org/message_filters/ApproximateTime. This
filter organizes the ROS messages into a queue for each sensor. When all four
sensors to have a point cloud message in the queue, the filter retrieves the
latest message from each sensor and then clears the queue.

In order to enhance the efficiency of the testing framework, the authors
implemented a downsampling technique on the point cloud acquired from the
RGB-D camera. The point cloud exhibited high density, resulting in
computationally intensive operations. To mitigate this issue, a mean pooling
algorithm was employed to downsample the point cloud.

In this approach, the LinkStates message provided by Gazebo is
utilized333https://docs.ros.org/en/jade/api/gazebo_msgs/html/msg/LinkStates.html.
The LinkStates message contains real-time pose information for each rigid body
(link) present in the simulation. The actor consists of various links, such as
”Head”, ”RightForeArm”, ”LowerBack”, ”LeftUpLeg”, among others. To determine
the connections between these links, an automated analysis of the original
mesh file (.dae) of the actor is performed.

A three-dimensional volume for each connection (joint) that encapsulates the
entire link within it was created. For instance, a cylinder with a radius of
0.1 meters is defined between the shoulder and the elbow. A more complex
example involves a rectangular prism with a width of 0.2 meters and a length
of 0.3 meters between the Spine and the LowerBack. One of the prism’s axes is
aligned with the line connecting the LeftHip and the RightHip.

With these volumes established, point-in-polyhedron algorithms were
implemented to extract the actor points from the original point clouds. For a
cylinder with a radius of $r$, the algorithm verifies whether a point,
represented by the coordinate vector $\vec{q}$, lies between the top and
bottom faces of the cylinder. The centers of these faces are represented by
the coordinate vectors $\vec{p}_{1}$ and $\vec{p}_{2}$, respectively.
Additionally, the algorithm checks if the point lies within the curved surface
of the cylinder. This is done by verifying if the distance between the point
and the cylinder’s center axis is smaller than the radius $r$. These
operations are illustrated by Equation 1, Equation 2, and Equation 3.

$(\vec{q}-\vec{p}_{1})\cdot(\vec{p}_{2}-\vec{p}_{1})\geq 0$ (1)

$(\vec{q}-\vec{p}_{2})\cdot(\vec{p}_{2}-\vec{p}_{1})\leq 0$ (2)

$\frac{|(\vec{q}-\vec{p}_{1})\times(\vec{p}_{2}-\vec{p}_{1})|}{|\vec{p}_{2}-\vec{p}_{1}|}\geq
r$ (3)

For a rectangular prism with dimensions of length $l$, width $w$, and height
$h$, the normal vectors for all its faces are computed. Specifically,
$\vec{n}_{l}$, $\vec{n}_{w}$, and $\vec{n}_{h}$ represent the normal vectors
along the length, width, and height of the prism, respectively. To determine
if a point $q$ lies within the prism, Equation 4 is utilized. This equation
calculates the coordinate vector of the point with respect to the coordinate
vector of the prism’s center, denoted as $\vec{c}$. Subsequently, the dot
product of this vector with each of the normal vectors is computed. This
calculation enables an assessment of how far the point is from the center in a
particular direction. If the result is greater than half of the corresponding
dimension of the prism, the point is considered outside the prism.

$|(\vec{q}-\vec{c})\cdot\vec{n}_{\alpha}|\leq\frac{\alpha}{2},\quad\alpha\in\\{l,w,h\\}$
(4)

From this method, the actor points can be extracted, as seen in Figure 7.
However, a challenge arises due to the mismatched units of measurement between
this points and the voxels retrieved from the mapping framework. To address
this issue, a voxel grid was constructed with matching resolution and pose as
the voxel grid defined by the framework. In this new grid, occupied voxels
represent the locations of the actor points, therefore being the ground truth
data.

Figure 7: Projection of actor points, in purple, to the image of the center
camera of LARCC.

###  III-C Evaluation Metrics

The assessment of the effectiveness of the framework will be predicated upon
several performance metrics, namely Precision ($P$), Recall ($R$), and F1
Score ($F_{1}$). The inclusion of $F_{2}$ and $F_{3}$ scores are also
presented, as these scores place a higher emphasis on recall in relation to
precision. This emphasis proves valuable, particularly in contexts involving
security scenarios, such as volumetric detection in a factory setting. These
metrics are defined by Equation 5, Equation 6, and Equation 7, in which $TP$,
$FP$, and $FN$ are true positives, false positives, and false negatives,
respectively. In Equation 7, the $\beta$ is 1 in $F_{1}$ score, 2 in $F_{2}$
score and 3 in $F_{3}$ score.

$P=\frac{TP}{TP+FP}$ (5)

$R=\frac{TP}{TP+FN}$ (6)

$F_{\beta}=(1+\beta^{2})\cdot\frac{P\cdot R}{(\beta^{2}\cdot P)+R}$ (7)

To establish $TP$, $FP$ and $FN$ values within the context of this work, it is
necessary to define relevant and retrieved elements. Relevant elements refer
to the ground truth voxels that have been previously extracted, as described
in subsection III-B. Retrieved elements, on the other hand, correspond to the
output of the frameworks in the form of voxels.

Ensuring the retrieved elements are extracted after relevant elements is
crucial. To accomplish this temporal alignment, an approximate time filter,
resembling the one described in subsection III-B, is implemented. This filter
organizes the output messages from the mapping framework into a queue,
arranging them in ascending order based on their timestamps. It identifies the
most recent timestamp among the fused point cloud messages. Consequently, it
retrieves the earliest message from the mapping framework queue that is
received after point cloud messages latest timestamp.

With both the relevant and retrieved elements established, $TP$ can be
identified as voxels that are classified as occupied by both the framework’s
and the ground truth’s voxel grid. $FP$ present a greater challenge, as the
framework’s voxel grid encompasses the entire scene rather than solely
focusing on the actor. To address this, the convex hull of the actor points is
obtained, and any retrieved element within this convex hull that is not a $TP$
is classified as $FP$. Conversely, $FN$ are voxels that are identified as
occupied by the ground truth but classified as unoccupied by the frameworks.
By utilizing these definitions, the aforementioned metrics can be calculated
to quantitatively assess the performance of the framework. These definitions
can be visualized in Figure 8.

Figure 8: Projection of true positive, false positive, and false negative
voxels to the image of the center camera of LARCC. The voxels are represented
in green, blue, and red, respectively.

##  IV Results

This section presents the experimental evaluation and results of the OctoMap,
SkiMap, and Voxblox frameworks. It encompasses both qualitative and
quantitative analyses to comprehensively assess the capabilities and
limitations of these frameworks in capturing and representing the dynamic
environment. The experimental infrastructure is described, followed by the
qualitative results of the default parameter settings for each framework.
Finally, the quantitative results, including precision, recall, and F scores,
are presented and analyzed, offering valuable insights into the performance of
OctoMap, SkiMap, and Voxblox.

###  IV-A Experimental Methodology

In order to standardize the experiments, a ROS tool named bag files was
utilized444https://wiki.ros.org/Bags. These files are capable of capturing all
the ROS messages, such as sensor pointclouds, within a specific time frame.
Subsequently, they allow for the replaying of the recorded messages at the
original recording rate. By recording the environment described in subsection
III-A, it ensures consistency and minimizes variability between experiments.
Bag files offer the functionality to adjust the data reproduction speed,
enabling both deceleration and acceleration. This feature proves particularly
useful in this study, as slowing down the bag file playback allows the mapping
framework sufficient time to integrate the map. This ensures that slower
frameworks are not disadvantaged. It is important to note that the objective
of this study is not primarily focused on measuring efficiency.

The experiments conducted not only involve comparing the default versions of
the frameworks with each other but also exploring different parameters within
each framework. Some parameters exhibit variations across frameworks. These
parameters are:

  * •

Minimum Voxel Length — This parameter represents the length of the smallest
voxel into which the algorithm can divide the environment.

  * •

Hit Probability — It denotes the probability of the algorithm increasing the
occupied probability of a voxel that has a detection.

  * •

Miss Probability — Probability of the algorithm reducing the occupied
probability assigned to a voxel lacking a detection.

  * •

Minimum Voxel Weight — It indicates the minimum weight required for a voxel to
be classified as occupied.

  * •

Truncation Distance — This value represents the maximum distance considered
for TSDF calculations in Voxblox.

  * •

Constant Weight — When this boolean parameter is active, calculated weights
remain constant irrespective of the distance between the point and the sensor.

The group of parameters and their default values can be summarized in Table I.

TABLE I: Test parameters and their default values for each of the tested frameworks: OctoMap, SkiMap and Voxblox. N/A means not applicable. | OctoMap | SkiMap | Voxblox
---|---|---|---
Minimum Voxel Length (m) | 0.1 | 0.1 | 0.1
Hit Probability | 1 | N/A | N/A
Miss Probability | 0.4 | N/A | N/A
Minimum Voxel Weight | N/A | 1 | N/A
Truncation Distance (m) | N/A | N/A | 0.1
Constant Weight | N/A | N/A | False

###  IV-B Qualitative Results

In this subsection, the authors aim to provide qualitative results of the
frameworks utilizing their default parameters. All figures are consistently
captured at the same moment in time and scene position, specifically when the
actor passes for the second time between the middle plate and the right leg of
the gantry.

Commencing with OctoMap, as depicted in Figure 9, the actor’s representation
is clearly discernible, and the entire scene is well-defined. However, a
limitation arises wherein certain voxels that previously indicated occupancy
encounter difficulty in accurately depicting emptiness, particularly those
corresponding to the upper side of the actor when it traverses behind the
plate. This phenomenon arises from the properties of raycasting, as the rays
that intersect with these voxels do not encounter any objects behind them from
the perspective of the sensors. Consequently, the affected voxels cannot be
effectively cleared, leading to a disparity between their actual empty state
and their visual representation.

Figure 9: OctoMap occupied voxels representing the actor passing through
between the middle plate and the right leg of the gantry.

Figure 10 illustrates the qualitative results of SkiMap. Interestingly, the
issue observed in OctoMap is magnified in this framework, as there appears to
be minimal to no occupied voxel erasure. This occurrence can be attributed to
the fact that SkiMap was not specifically designed with volumetric detection
in mind and, consequently, is ill-equipped to handle such scenarios. The
framework’s limitations in effectively handling and updating voxel occupancy
representations in dynamic environments become more apparent, further
highlighting its lack of readiness for addressing these specific types of
situations.

Figure 10: SkiMap occupied voxels representing the actor passing through
between the middle plate and the right leg of the gantry.

Figure 11 presents the qualitative results of Voxblox. It is evident that this
framework encounters significant challenges in detecting moving actors within
the scene. This difficulty can potentially be attributed to the adoption of a
high truncation distance, which leads to the framework observing a larger
volume encompassing the actor. As a consequence, the larger volume primarily
consists of empty space, resulting in a comparatively greater distance value
assigned to the voxel and subsequently classifying it as empty. A more
comprehensive explanation of this phenomenon can be found in subsection IV-C,
where detailed quantitative results are elaborated upon.

Figure 11: Voxblox occupied voxels representing the actor passing through
between the middle plate and the right leg of the gantry.

###  IV-C Quantitative Results

In this subsection, the authors present the quantitative results of precision,
recall, and F scores obtained from evaluating different variations of OctoMap,
SkiMap, and Voxblox. These metrics provide objective measures of the
frameworks’ performance in capturing and representing the dynamic environment.
The analysis aims to provide deeper insights into the strengths and
limitations of each framework, facilitating a comprehensive understanding of
their respective capabilities.

Starting with OctoMap, the effect of the minimum voxel length can be seen in
Table II. The precision consistently improves with higher lengths, possibly
because false positives are generally situated in close proximity to true
positives. By enlarging the voxels, this proximity is now encompassed within
the true positive voxels, thereby enhancing precision. The change in recall,
however, is less pronounced and reaches its peak at 0.1 meters. Generally,
recall is smaller than precision due to occlusion in the testing environment,
where the front half of the actor obstructs the back half, resulting in a
considerable number of false negatives. Smaller lengths yield smaller recall
as a smaller volume of the actor is covered by occupied voxels. Similarly,
higher lengths also yield smaller recall because certain false negatives
persist while the overall voxel count decreases, thereby elevating the false
negatives to true positives ratio. Thus, the authors propose that an optimal
recall point is reached in this environment, estimated to be around 0.1
meters. The authors further conclude that a length of 0.150 meters excels in
applications with lower security requirements, while a length of 0.1 meter is
ideal for scenarios prioritizing security.

TABLE II: Values of precision, recall and F scores for OctoMap when altering minimum voxel length. Minimum Voxel Length (m) | Precision | Recall | F1 Score | F2 Score | F3 Score
---|---|---|---|---|---
$0.05$ | 0.347 | 0.416 | 0.379 | 0.400 | 0.408
$0.075$ | 0.519 | 0.468 | 0.492 | 0.477 | 0.473
$0.1$ | 0.650 | 0.477 | 0.550 | 0.504 | 0.490
$0.125$ | 0.750 | 0.468 | 0.577 | 0.506 | 0.487
$0.150$ | 0.833 | 0.460 | 0.593 | 0.506 | 0.482
$0.175$ | 0.883 | 0.439 | 0.586 | 0.488 | 0.462
$0.2$ | 0.940 | 0.425 | 0.586 | 0.478 | 0.450

The impact of OctoMap hit probability is demonstrated in Table III of this
study. Notably, higher probabilities yield improved precision and recall
measures, with precision values remaining constant for probabilities of 0.7
and 1. This trend can be attributed to the increased sensitivity of the
framework to changes as the probability is increased, thereby reducing the
occurrences of false negatives and false positives. Conversely, decreasing the
probability results in an inaccurate representation of the environment,
leading to an augmented number of false negatives and false positives. Based
on these findings, the authors strongly advocate for a hit probability of 1
across all scenarios.

TABLE III: Values of precision, recall and F scores for OctoMap when altering hit probability. Hit probability | Precision | Recall | F1 Score | F2 Score | F3 Score
---|---|---|---|---|---
$0.5$ | 0.414 | 0.011 | 0.021 | 0.014 | 0.012
$0.7$ | 0.675 | 0.218 | 0.330 | 0.252 | 0.234
$1$ | 0.650 | 0.477 | 0.550 | 0.504 | 0.490

The impact of OctoMap miss probability is illustrated in Table IV of this
investigation. Upon examination, it is observed that increasing the miss
probability to 0.7 leads to a reduction in both precision and recall values.
Conversely, when the probability fluctuates within the range of 0.2 to 0.4,
the results remain relatively consistent. The decline in precision and recall
can be attributed to a substantial decrease in the weight assigned to voxels,
rendering them significantly less likely to be classified as occupied, even in
the presence of detections. Consequently, this amplifies the number of false
negatives and diminishes the count of true positives. Furthermore, this
imbalance results in a larger number of false positives compared to true
positives. Hence, for scenarios resembling the one described, the authors
endorse employing a low miss percentage.

TABLE IV: Values of precision, recall and F scores for OctoMap when altering miss probability. Miss Probability | Precision | Recall | F1 Score | F2 Score | F3 Score
---|---|---|---|---|---
$0.2$ | 0.675 | 0.459 | 0.547 | 0.491 | 0.475
$0.4$ | 0.650 | 0.477 | 0.550 | 0.504 | 0.490
$0.7$ | 0.430 | 0.027 | 0.050 | 0.033 | 0.030

The impact of the minimum voxel length on SkiMap is demonstrated in Table V of
this study. The observed behavior is analogous to that observed in OctoMap,
whereby precision increases as the voxel length enlarges. Additionally, the
recall reaches its peak at approximately 0.075 meters in length. Based on
these findings, the authors further infer that a voxel length of 0.150 meters
is particularly well-suited for applications with lower security requirements.
Alternatively, voxel lengths of 0.075 or 0.1 meters are recommended for
scenarios that prioritize security considerations.

TABLE V: Values of precision, recall and F scores for SkiMap when altering minimum voxel length. Minimum Voxel Length (m) | Precision | Recall | F1 Score | F2 Score | F3 Score
---|---|---|---|---|---
$0.05$ | 0.310 | 0.566 | 0.400 | 0.485 | 0.523
$0.075$ | 0.470 | 0.657 | 0.548 | 0.608 | 0.632
$0.1$ | 0.606 | 0.629 | 0.617 | 0.624 | 0.627
$0.125$ | 0.707 | 0.581 | 0.638 | 0.603 | 0.592
$0.150$ | 0.799 | 0.544 | 0.647 | 0.581 | 0.562
$0.175$ | 0.867 | 0.500 | 0.633 | 0.545 | 0.521
$0.2$ | 0.922 | 0.466 | 0.619 | 0.517 | 0.490

The impact of the minimum voxel weight on SkiMap is depicted in Table VI of
this investigation. It is observed that as the minimum weight increases,
precision values also increase, while recall values decrease. This phenomenon
can be attributed to the fact that higher minimum weights result in detections
occurring only in voxels that exhibit a high level of certainty in terms of
occupancy. Consequently, the number of false positives is reduced, but the
number of false negatives is amplified. However, when evaluating the F scores,
it is evident that a minimum weight of 1 yields the most optimal performance
for this particular scenario.

TABLE VI: Values of precision, recall and F scores for SkiMap when altering minimum voxel weight. Minimum Voxel Weight | Precision | Recall | F1 Score | F2 Score | F3 Score
---|---|---|---|---|---
$1$ | 0.606 | 0.629 | 0.617 | 0.624 | 0.627
$5$ | 0.613 | 0.582 | 0.597 | 0.588 | 0.585
$10$ | 0.624 | 0.542 | 0.580 | 0.557 | 0.549

The impact of the minimum voxel length on Voxblox is presented in Table VII,
highlighting similarities and notable distinctions from the findings of
OctoMap and SkiMap. While some similarities in behavior are observed, a
striking difference is identified. Specifically, for voxel lengths smaller
than 0.150 meters, a significant decrease in recall is observed. This
phenomenon can be attributed to the relationship between the truncation
distance and the minimum voxel length, which will also be discussed in detail
in the discussion of Table VIII. The authors conducted an analysis and found
that achieving a ratio of voxel size per truncation distance of at least 1.5
leads to an increase in recall, despite Voxblox’s recommended ratio of 0.5.
The authors speculate that this improvement occurs because as the ratio
increases, the framework incorporates points that are farther away from the
voxel, consequently yielding a larger average distance value for that voxel.
This effect may generate distance values that fall below the occupied
threshold, resulting in false negatives.

TABLE VII: Values of precision, recall and F scores for Voxblox when altering minimum voxel length. Minimum Voxel Length (m) | Precision | Recall | F1 Score | F2 Score | F3 Score
---|---|---|---|---|---
$0.05$ | $0.313$ | 0.043 | 0.075 | 0.052 | 0.047
$0.075$ | $0.625$ | 0.068 | 0.123 | 0.083 | 0.075
$0.1$ | $0.749$ | 0.089 | 0.159 | 0.108 | 0.097
$0.125$ | $0.750$ | 0.179 | 0.289 | 0.211 | 0.194
$0.150$ | $0.770$ | 0.550 | 0.643 | 0.585 | 0.568
$0.175$ | $0.85$ | 0.521 | 0.646 | 0.565 | 0.542
$0.2$ | 0.9 | 0.485 | 0.633 | 0.535 | 0.509

The impact of Voxblox truncation distance is elaborated upon in Table VIII,
providing additional support for the authors’ previous argument regarding the
importance of achieving a ratio of at least 1.5. The table clearly
demonstrates that significantly better results are obtained when the ratio
reaches 2, in comparison to ratios of 1 or 0.5, which is the recommended
value. These findings emphasize the significance of appropriately adjusting
the truncation distance to establish a favorable balance between voxel size
and the distance of influence. By ensuring a higher ratio, the algorithm
selectively incorporates points within close proximity to the voxel,
effectively mitigating the introduction of extraneous signals originating from
distant points. Consequently, this approach engenders more precise and
dependable distance values, thereby optimizing the performance of Voxblox
within the given contextual parameters.

TABLE VIII: Values of precision, recall and F scores for Voxblox when altering the truncation distance. Truncation Distance (m) | Precision | Recall | F1 Score | F2 Score | F3 Score
---|---|---|---|---|---
$0.05$ | 0.544 | 0.698 | 0.612 | 0.661 | 0.679
$0.1$ | 0.749 | 0.089 | 0.159 | 0.108 | 0.097
$0.2$ | 0.634 | 0.024 | 0.045 | 0.029 | 0.026

The impact of the Voxblox constant weight is illustrated in Table IX. Notably,
the utilization of a constant weight leads to a decline in performance
compared to Voxblox’s default behavior. This is primarily attributed to the
fact that the constant weight approach overlooks crucial parameters that are
typically considered by Voxblox, including the distance from the sensor to the
voxel and the distance of detection to the center of the voxel. By neglecting
these important factors, the algorithm’s performance is compromised, resulting
in suboptimal outcomes. It is evident that the dynamic and context-aware
weight assignment employed by Voxblox provides more accurate and reliable
results compared to the simplified constant weight approach.

TABLE IX: Values of precision, recall and F scores for Voxblox when altering if the calculated weight of a voxel in constant. Constant Weight | Precision | Recall | F1 Score | F2 Score | F3 Score
---|---|---|---|---|---
True | 0.569 | 0.026 | 0.049 | 0.032 | 0.028
False | 0.749 | 0.089 | 0.159 | 0.108 | 0.097

##  V Conclusions

This paper presented a novel test environment designed to evaluate the
performance of volumetric detection algorithms. The proposed environment
utilizes simulation techniques to capture ground truth data and compares them
to the detections obtained, enabling the calculation of precision, recall, and
F scores.

To capture the ground truth data, the test environment leverages Gazebo and
its LinkStates messages to track the location of each actor joint in real
time. For each joint, a polyhedron that encompasses the entire joint is
defined. Subsequently, point in polyhedron algorithms are applied to calculate
the actor points inside each joint, which are then used to construct a voxel
grid correspondent to the ground truth data. This ground truth is compared to
the voxel grid generated by the frameworks under evaluation, allowing for the
determination of true positives, false positives, and false negatives.

This approach fills a gap in the field by providing a standardized method for
evaluating the precision and recall of volumetric detection algorithms.
Typically, existing approaches only compare integration time and memory usage
among frameworks, neglecting other important aspects.

The development of the innovative test environment and performance metrics
proved essential in providing previously unknown insights into the performance
of various volumetric detection frameworks. The conducted tests involved
systematically varying parameters within each framework to identify an optimal
configuration. The results were evaluated using precision, recall, and F
scores, supplemented by a qualitative assessment of each framework’s default
settings, enabling a more comprehensive analysis.

Based on these insights, it can be concluded that OctoMap, with its default
parameters, is the most suitable framework for volumetric detection in a cell
environment. While SkiMap exhibited marginally superior quantitative outcomes,
the qualitative evaluation highlighted its limitation in accurately removing
occupied voxels that are actually empty. This issue arises as a significant
portion of the voxels traversed by the actor continue to be labeled as
occupied in SkiMap.

Future work should focus on expanding the volume of analysis to encompass the
entire trajectory of the actor during the experiment. This would enhance the
compatibility between quantitative and qualitative analyses and address the
aforementioned issue with SkiMap. Additionally, further tests of the test
environment should be conducted in different scenarios and with a wider range
of frameworks to ensure comprehensive evaluation and validation.

## References

  * [1] Joel Baptista, Vítor Santos, Filipe Silva and Diogo Pinho  “Domain Adaptation with Contrastive Simultaneous Multi-Loss Training for Hand Gesture Recognition”  In _Sensors_ 23.6 Multidisciplinary Digital Publishing Institute, 2023, pp. 3332  DOI: 10.3390/s23063332
  * [2] Heike Brock et al.  “Developing a Lightweight Rock-Paper-Scissors Framework for Human-Robot Collaborative Gaming”  In _IEEE Access_ 8, 2020, pp. 202958–202968  DOI: 10.1109/ACCESS.2020.3033550
  * [3] Gongfa Li et al.  “Dynamic Gesture Recognition in the Internet of Things”  In _Ieee Access_ 7 Piscataway: Ieee-Inst Electrical Electronics Engineers Inc, 2019, pp. 23713–23724  DOI: 10.1109/ACCESS.2018.2887223
  * [4] Du Jiang et al.  “Gesture Recognition Based on Binocular Vision”  In _Cluster Computing-the Journal of Networks Software Tools and Applications_ 22 New York: Springer, 2019, pp. 13261–13271  DOI: 10.1007/s10586-018-1844-5
  * [5] Ali Bou Nassif et al.  “Speech Recognition Using Deep Neural Networks: A Systematic Review”  In _Ieee Access_ 7 Piscataway: Ieee-Inst Electrical Electronics Engineers Inc, 2019, pp. 19143–19165  DOI: 10.1109/ACCESS.2019.2896880
  * [6] Abdel Aziza Abdelhamid et al.  “Robust Speech Emotion Recognition Using CNN plus LSTM Based on Stochastic Fractal Search Optimization Algorithm”  In _Ieee Access_ 10 Piscataway: Ieee-Inst Electrical Electronics Engineers Inc, 2022, pp. 49265–49284  DOI: 10.1109/ACCESS.2022.3172954
  * [7] Triantafyllos Afouras et al.  “Deep Audio-Visual Speech Recognition”  In _Ieee Transactions on Pattern Analysis and Machine Intelligence_ 44.12 Los Alamitos: Ieee Computer Soc, 2022, pp. 8717–8727  DOI: 10.1109/TPAMI.2018.2889052
  * [8] Juanpablo Heredia et al.  “Adaptive Multimodal Emotion Detection Architecture for Social Robots”  In _Ieee Access_ 10 Piscataway: Ieee-Inst Electrical Electronics Engineers Inc, 2022, pp. 20727–20744  DOI: 10.1109/ACCESS.2022.3149214
  * [9] Ulysse Cote-Allard et al.  “Deep Learning for Electromyographic Hand Gesture Signal Classification Using Transfer Learning”  In _Ieee Transactions on Neural Systems and Rehabilitation Engineering_ 27.4 Piscataway: Ieee-Inst Electrical Electronics Engineers Inc, 2019, pp. 760–771  DOI: 10.1109/TNSRE.2019.2896269
  * [10] Jinxian Qi et al.  “Intelligent Human-Computer Interaction Based on Surface EMG Gesture Recognition”  In _Ieee Access_ 7 Piscataway: Ieee-Inst Electrical Electronics Engineers Inc, 2019, pp. 61378–61387  DOI: 10.1109/ACCESS.2019.2914728
  * [11] Miguel Simao, Nuno Mendes, Olivier Gibaru and Pedro Neto  “A Review on Electromyography Decoding and Pattern Recognition for Human-Machine Interaction”  In _Ieee Access_ 7 Piscataway: Ieee-Inst Electrical Electronics Engineers Inc, 2019, pp. 39564–39582  DOI: 10.1109/ACCESS.2019.2906584
  * [12] Afonso Castro, Joel Baptista, Filipe Silva and Vitor Santos  “Classification of Handover Interaction Primitives in a COBOT–Human Context with a Deep Neural Network”  In _Journal of Manufacturing Systems_ 68, 2023, pp. 289–302  DOI: 10.1016/j.jmsy.2023.03.010
  * [13] Anees Peringal et al.  “Design of Dynamics Invariant LSTM for Touch Based Human-UAV Interaction Detection”  In _Ieee Access_ 10 Piscataway: Ieee-Inst Electrical Electronics Engineers Inc, 2022, pp. 116045–116058  DOI: 10.1109/ACCESS.2022.3218904
  * [14] Afonso Castro, Filipe Silva and Vitor Santos  “Trends of Human-Robot Collaboration in Industry Contexts: Handover, Learning, and Metrics”  In _Sensors_ 21.12 Multidisciplinary Digital Publishing Institute, 2021, pp. 4113  DOI: 10.3390/s21124113
  * [15] Armin Hornung et al.  “OctoMap: An Efficient Probabilistic 3D Mapping Framework Based on Octrees”  In _Autonomous Robots_ 34.3, 2013, pp. 189–206  DOI: 10.1007/s10514-012-9321-0
  * [16] Daniele De Gregorio and Luigi Di Stefano  “SkiMap: An Efficient Mapping Framework for Robot Navigation”  In _2017 IEEE International Conference on Robotics and Automation (ICRA)_ , 2017, pp. 2569–2576  DOI: 10.1109/ICRA.2017.7989299
  * [17] Helen Oleynikova et al.  “Voxblox: Incremental 3D Euclidean Signed Distance Fields for on-Board MAV Planning”  In _2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_ Vancouver, BC: IEEE, 2017, pp. 1366–1373  DOI: 10.1109/IROS.2017.8202315
  * [18] Jane Wilhelms and Allen Van Gelder  “Octrees for Faster Isosurface Generation”  In _ACM Transactions on Graphics_ 11.3, 1992, pp. 201–227  DOI: 10.1145/130881.130882
  * [19] William Pugh  “Skip Lists: A Probabilistic Alternative to Balanced Trees”  In _Communications of the ACM_ 33.6, 1990, pp. 668–676  DOI: 10.1145/78973.78977
  * [20] Yuan He, Shunyi Zheng, Fengbo Zhu and Xia Huang  “Real-Time 3D Reconstruction of Thin Surface Based on Laser Line Scanner”  In _Sensors_ 20.2 Basel: Mdpi, 2020, pp. 534  DOI: 10.3390/s20020534
  * [21] Boris Lau, Christoph Sprunk and Wolfram Burgard  “Improved Updating of Euclidean Distance Maps and Voronoi Diagrams”  In _2010 IEEE/RSJ International Conference on Intelligent Robots and Systems_ , 2010, pp. 281–286  DOI: 10.1109/IROS.2010.5650794
  * [22] Daniela Rato et al.  “A Sensor-to-Pattern Calibration Framework for Multi-Modal Industrial Collaborative Cells”  In _Journal of Manufacturing Systems_ 64, 2022, pp. 497–507  DOI: 10.1016/j.jmsy.2022.07.006

|  Manuel Gomes received an Integrated Master’s Degree in Mechanical
Engineering from the University of Aveiro in 2022. Their M.Sc. dissertation
focused on sensor calibration in robotic vehicles. Currently pursuing a Ph.D.
in Mechanical Engineering at the University of Aveiro, the author is actively
involved in the AUGMANITY project, a EU project, working on collaborative
robots.
---|---
|  Miguel Oliveira received the bachelor’s and M.Sc. degrees from the
University of Aveiro, Aveiro, Portugal, in 2004 and 2007, respectively, and
the Ph.D. degree in mechanical engineering (specialization in robotics, on the
topic of autonomous driving systems), in 2013. From 2013 to 2017, he was an
Active Researcher with the Institute of Electronics and Telematics Engineering
of Aveiro, Aveiro, and with the Institute for Systems and Computer
Engineering, Technology and Science, Porto, Portugal, where he participated in
several EU-funded projects, as well as national projects. He is currently an
Assistant Professor with the Department of Mechanical Engineering, University
of Aveiro, where he was also the Director of the Masters in Automation
Engineering. He has supervised more than 20 M.Sc. students. He authored over
20 journal publications, from 2015 to 2020. His research interests include
autonomous driving, visual object recognition in open-ended domains, multi-
modal sensor fusion, computer vision, and the calibration of robotic systems.
---|---
|  Vítor Santos obtained a 5-year degree in Electronics Engineering and
Telecommunications in 1989, at the University of Aveiro, Portugal, where he
later obtained a PhD in Electrical Engineering in 1995, and the Habilitation
in Mechanical Engineering in 2018. He was awarded fellowships for research in
mobile robotics during 1990-1994 at the Joint Research Center, Italy. He is
currently Associate Professor at the University of Aveiro where he lectures
several courses on robotics, autonomous vehicles and computer vision, and has
carried out research activity on mobile robotics, autonomous driving, advanced
perception and humanoid robotics, also in public and privately funded
projects. He has supervised and co-supervised more than 100 students in
Masters, PhD and Postdoc programs, and coordinated the creation of two
University degrees in the field of Automation at the University of Aveiro. He
founded the ATLAS project for mobile robot competition that achieved 6 first
prizes in the annual Autonomous Driving competition, and has coordinated the
development of ATLASCAR, the first real car with autonomous navigation
capabilities in Portugal. He was involved in the organization of several
conferences, workshops and special sessions in national and international
events, including being the General Chair of the IEEE International Conference
on Autonomous Robot Systems and Competitions, ICARSC2021. He is one of the
founders of the Portuguese Robotics Open in 2001 and co-founder of the
Portuguese Society of Robotics in 2006.
---|---
  *[ESDFs]: Euclidean Signed Distance Field
  *[TSDFs]: Truncated Signed Distance Field
  *[TSDF]: Truncated Signed Distance Field
  *[ESDF]: Euclidean Signed Distance Field
  *[ROS]: Robot Operating System


# A Framework for Automated Measurement of Responsible AI Harms in Generative
AI Applications

Ahmed Magooda*, Alec Helyar*, Kyle Jackson*, David Sullivan, Chad Atalla,
Emily Sheng, Dan Vann, Richard Edgar, Hamid Palangi, Roman Lutz, Hongliang
Kong, Vincent Yun, Eslam Kamal, Federico Zarfati, Hanna Wallach, Sarah Bird,
Mei Chen  MicrosoftRedmondWA ahmedmagooda, alec.helyar, kyle.jackson,
dsullivan, chad.atalla, emilysheng, dan.vann<EMAIL_ADDRESS>hpalangi,
romanlutz, hongliang.kong, xi.yun, eskam, fzarfati, wallach, sarah.bird,
<EMAIL_ADDRESS>
###### Abstract.

We present a framework for the automated measurement of responsible AI (RAI)
metrics for large language models (LLMs) and associated products and services.
Our framework for automatically measuring harms from LLMs builds on existing
technical and sociotechnical expertise and leverages the capabilities of
state-of-the-art LLMs, such as GPT-4. We use this framework to run through
several case studies investigating how different LLMs may violate a range of
RAI-related principles. The framework may be employed alongside domain-
specific sociotechnical expertise to create measurements for new harm areas in
the future. By implementing this framework, we aim to enable more advanced
harm measurement efforts and further the responsible use of LLMs.111This is a
living document

* Authors contributed equally.

##  1\. Introduction

Rapid advancements in artificial intelligence (AI) and natural language
processing (NLP) have led to the development of increasingly sophisticated
large language models (LLMs) such as (GPT-4(OpenAI, 2023), LLama 2(Touvron et
al., 2023), Falcon(Penedo et al., 2023), etc.), with advanced text generation
capabilities across a wide range of task types. While these models unlock
numerous opportunities, there are also serious concerns about models causing
harm(Kumar et al., 2023). Manual detection of harms may better account for
nuances. However, as the availability and capabilities of LLMs grow, it is
increasingly necessary to develop automated frameworks for measuring harms
with a speed and scale that can match the pace of the technology’s
proliferation.

Motivated by the need for an automated harm measurement framework which is
flexible enough to align with evolving, valid, and reliable definitions of
harms, as well as the need for a measurement implementation that could be
applied across different types of products and services related to LLMs (e.g.,
chatbots, summarization systems, etc.), we propose and implement a framework
that harnesses the capabilities of LLMs to test other LLMs and assess their
potential for causing harm. While our work yields tools for automated
measurement, creating the harm-specific measurement resources (e.g., harm
measurement definitions) still requires domain-specific expertise. We would
like to preface the rest of this paper with an acknowledgment that this is not
the final, only, nor necessarily best implementation to measuring harms;
however, it is an implementation that allows for flexibility in updating
definitions and applying to various products and services. There are still
open questions about the risks of employing LLMs to perform parts of the harm
measurement process and how much of the measurement pipeline can and should be
automated—we discuss this more in Sec. 5 but mostly leave these important
questions to future work.

The core of our proposed framework comprises of two key components: (1) data
generation from templates and (2) evaluation of generated outputs. First, we
introduce a data generation component designed to assess LLM propensity for
generating specific types of potential harmful content. This component
simulates various real-world LLM products and services, such as question
answering, summarization, and conversation. Next, we introduce an evaluation
component that uses GPT-4 to assess LLM-generated content according to harm
definitions. This component evaluates AI-generated content and produces both
quantitative and qualitative outputs, yielding numerical annotations of harm
severity and written snippets about annotation reasoning. Our framework
enables automatic comparison of different LLM-based products and services
against measurement sets built by domain experts for various harms, allowing
practitioners to compare strengths and weaknesses.

##  2\. Architecture

Our measurement framework comprises of two components that are tailored for
assessing LLMs: 1) data generation from templates and parameters, and 2)
evaluation of generated outputs via annotation guidelines. The data generation
component uses templates and parameters to simulate interactions with the LLM
under test to generate data which approximates a user-AI interaction in some
product or service. The templates and parameters are separately created by
domain experts for each harm to ensure the reliability and validity of the
resulting measurements. Next, the evaluation component produces annotations of
the LLM’s output on the generated data by applying annotation guidelines. The
annotation guidelines are provided by domain experts based on the harm
definitions they create.

Figure 1. Sample Azure Machine Learning pipeline which shows the two
evaluation parts (Data Generation, and Evaluation

The evaluation process is streamlined by treating the LLM under test as a
black box which need only accept inputs and yield outputs. Additionally, the
implementation of this framework supports two different environments for
computation. The first environment involves running the evaluation on a local
machine, where prompt construction and model API coordination, model API
calling, etc. occur locally. The second environment utilizes the Azure Machine
Learning (AML) platform to automatically construct evaluation pipelines and
perform evaluations using AML compute resources. Figure 1 shows an sample AML
evaluation pipeline.

###  2.1. Data Generation

The first part of our framework focuses on simulating a hypothetical user’s
interaction with a real product or service such as question answering, chat,
and document summarization. The goal of this part of the data generation
pipeline, referred to as task simulation, is to generate interactions (between
the LLM and a hypothetical user) which cover topics or patterns associated
with a target harm. To achieve this, we use another LLM to play the role of a
hypothetical user, initiating the task and participating in the conversation
based on the templates provided by domain experts. We denote the LLM under
test as $\bm{LLM_{test}}$ and the user-simulating LLM as $\bm{LLM_{user}}$.

Figure 2. An example of a hypothetical persona template skeleton for
simulating conversation with search scenario Figure 3. An example of
hypothetical parameters to be injected into the template skeleton from Figure
2

We provide a set of templates, referred to as persona templates, which provide
guidelines for the $\bm{LLM_{user}}$ regarding how to behave and which topics
or goals to introduce in the interaction with $\bm{LLM_{test}}$. For
simplicity and generalizability, we employ Jinja-style parameterized
templates. Each template describes the basic structure and theme of the
conversations, leaving placeholders for parameters specifying specific topics,
groups of people, etc, to be incorporated. Then, each template is combined
with each set of corresponding parameters to create one or more completed
personas for the $\bm{LLM_{user}}$ to leverage in task simulation with the
blackbox $\bm{LLM_{test}}$222The templates and parameters are two pieces of
the measurement resources that are created by domain experts for each harm.
The process of how domain experts create these measurement resources will be
examined separately in future work..

Given these completed personas created by combining templates and parameters,
we run task simulation next. Each completed persona serves as instructions for
$LLM_{user}$, shaping how it interacts with $LLM_{test}$. This part injects
creativity and is critical for automating and scaling up the process, but it
also yields risks. For example, what if the $LLM_{user}$ does not simulate
realistic user behavior in the interaction with $LLM_{test}$? We explore these
concerns further in section 5. Once the task simulation has been run for each
completed persona, we are left with a set of generated data which includes
simulated user inputs and real $\bm{LLM_{test}}$ system outputs (we refer to
each simulated interaction as a sample).

###  2.2. Evaluation

The second part of our framework is responsible for evaluation through
automated annotation of the samples generated in task simulation. The
annotation process uses an LLM by providing it with annotation guidelines
which are manually crafted by domain experts and include harm definitions,
examples, and a defect definition. The defect definition specifies criteria
for determining whether a data sample is considered desirable or allowable in
the context of the LLM under test and any product or service it is embedded
in. Crafting this definition is a sociotechnical challenge which is deeply
entangled with the harm definitions created by domain experts and policy
decisions made by the organizations building the AI system under test.

The LLM can then annotate the given examples using the provided guidelines.
Automated annotation consists of multiple steps: the first step uses the
annotation guidelines to annotate each sample. These annotations are initially
created in text, where the LLM follows an annotation schema specified by few-
shot examples in the annotation guidelines. The next step parses the
annotation to extract expected metrics (e.g., defect score, reasoning, etc)
according to the provided guidelines. The final step involves aggregating the
extracted values and calculating a metric (e.g., defect rate.).

For each harm area, human-LLM annotation agreement experiments are conducted
during the development of measurement resources. After that, the measurement
resources and technical framework can be applied jointly to produce
measurements without human annotation. Ultimately, a defect rate is
calculated, which represents the proportion of samples which were annotated as
matching the defect definition.

For example, one way defect definitions may work is through severity
thresholds. Consider the case where we may wish to evaluate whether the LLM
under test produces extreme violent content. The domain experts may build a
severity scale (e.g., on an 1-10 scale where lower is less severe) for violent
content, and a defect definition could be a threshold within this severity
range or a particular severity scale (e.g., any sample with severity $\geq 7$
is a defect). Then, the defect rate can be determined by calculating the ratio
of samples that meet the defect definition relative to the total number of
samples. In this case, the defect rate can be computed as follows:

$DefectRate=\frac{|{x\in samples:x>threshold}|}{|samples|}$

##  3\. Interpreting Measurements

By combining this framework with measurement resources (templates, parameters,
harm definitions, and annotation guidelines), a repeatable measurement
pipeline can be created. Running this measurement pipeline on an AI system
yields a defect rate. It is important to interpret this defect carefully and
understand the utility of measurements derived this way. All defect rates
obtained through application of this technical framework are relative
measurements, which do not represent the absolute state of the world. In other
words, a 0% defect rate does not mean that there is zero chance of the
measured harm occurring in the real world. Instead, a 0% defect rate may be
interpreted to mean that the AI system under test did not appear to fail any
tests in the current measurement set.

Additionally, all resulting measurements are only as reliable and valid as the
measurement resources designed for the harm being tested. The process of
creating these measurement resources is a complex sociotechnical problem which
is fraught with pitfalls and opportunities for reliability and validity to be
impacted. If the measurement resources are created with a poorly constructed
harm definition, the resulting measurements can range from nonsensical to
directly harmful (if development decisions are misled by a poorly designed
measurement).

With this perspective, these measurements provide significant and targeted
utility. These measurements can serve as diagnostic tools. They enable
comparison of the efficacy of different mitigations as well as tracking of
progress in mitigating known defects over time. Lastly, when using identical
measurement sets to test two AI systems, the resulting measurements can be
used to compare the relative performance of each system on the challenges
represented in the measurement set.

##  4\. Case Study

Below we provide a deep dive on Groundedness. Then we provide an example of
how this framework can be leveraged to create measurements and compare
multiple models.

###  4.1. Deep Dive: Groundedness

Figure 4. Annotation Guidelines for Groundedness that is given to $LLM_{test}$
to answer the question made by $LLM_{user}$ using only provided context.

In this case study, we consider ungrounded generations from $LLM_{test}$ to be
harmful and refer to this measurement category as groundedness. We first had
to build measurement resources for this specific harm. As mentioned earlier,
measurement resources must include a set of templates and parameters. For the
groundedness case study, the templates and parameters were to yield a set of
of questions (prompts to $LLM_{test}$) and corresponding contextual files
(used by $LLM_{test}$ to answer the prompt questions). In the first stage of
the evaluation pipeline (i.e., data generation with task simulation), we
initiate conversations between $LLM_{test}$ and the simulated $LLM_{user}$.
$LLM_{user}$ follows the templates and parameters and asks each question from
the provided set. At the same time, we provide $LLM_{test}$ with access to the
context files and provide guidance to answer the questions based solely on the
context files. Figure 4 illustrates the prompt guidance for $LLM_{test}$ to
answer questions while relying exclusively on the context files as a source of
information.

Following the generation of conversations, we proceed to the evaluation stage
to assess generated samples. As part of our measurement resources, we must
provide annotation guidelines to an LLM (GPT-4) to evaluate whether a response
is grounded or not. In this case, we design a basic annotation guideline to
yield a 1 - 5 groundedness score. A score of 1 signifies that the response is
not grounded, while a score of 5 indicates that all information in the answer
is grounded. Figure 5 shows the annotation guidelines. The LLM annotator
(GPT-4) is then provided with the original question posed by $LLM_{user}$, the
response from $LLM_{test}$, and the context given to $LLM_{test}$ for
formulating its answer. Subsequently, the LLM annotator assigns a groundedness
score on a scale of 1 to 5 for each sample.

Figure 5. Grounding annotation guidelines used for evaluating LLM system
responses. Figure 6. Confusion matrix between human- and model-annotated
grounding scores based on the provided guidelines.

To evaluate the effectiveness of our annotation guidelines, we collected a
dataset of 266 examples including questions, responses, and the context used
to generate the responses. These examples were annotated by human evaluators
using the same scale from 1 to 5 for groundedness. In parallel, we employed
our proposed framework utilizing GPT-4 to annotate the same data, also on the
same scale from 1 to 5, using the crafted annotation guidelines.

Then, we assessed the agreement between the human and GPT-4 annotations using
two simple heuristic metrics. The first metric, exact agreement ratio,
measures the proportion of instances where the human and GPT-4 scores are
identical. The second metric serves more as a loose heuristic: relaxed
agreement ratio, which considers agreement in cases where the human and GPT-4
scores differ by no more than 1 point on the scale.

| Human-Model Agreement
---|---
Exact ($V_{Human}$ == $V_{Model}$) | 60%
Relaxed (——$V_{Human}$ \- $V_{Model}$—— ¡= 1) | 80.5%
Relaxed (——$V_{Human}$ \- $V_{Model}$—— ¡= 2) | 93.6%
Table 1. Agreement ratio between human and model annotations.

Our preliminary analysis revealed an exact agreement ratio of 60% and a
relaxed agreement ratio of 80.5% as shown in table 1. Figure 6 presents a
confusion matrix illustrating the relationship between the human and GPT-4
annotations. Further work on human-human agreement is required as well to
build an understanding of what an acceptable result is on each of these
metrics. Additionally, more robust agreement analysis will be performed in
future work. This sort of measurement provides a sense of the quality of the
annotation guidelines, which allows us to iterate on and improve the
guidelines. These preliminary results are also useful for building a rough
notion of how confident we can be in resulting measurements.

###  4.2. Experimental Design

We conducted a set of experiments to evaluate three LLMs with the proposed
evaluation framework. We refer to these three models as model 1, model 2, and
model 3.333We anonymized model names for now—more details will be provided in
future updates to this manuscript In all of the reported experiments, we
focused on conversation simulation tasks, where we engaged in a synthetic
conversation with the LLM under test ($LLM_{test}$) to measure its tendency to
violate RAI principles in the following aspects:

  1. (1)

Succeeding in Jailbreaks

  2. (2)

Generating Potentially Harmful Content, including but not limited to:444For
these highly sociotechnical harms, the measurement resources were constructed
by domain experts, leveraging techniques that are out of scope for this
manuscript.

     * •

Hateful or Unfair Content

     * •

Sexual Content

     * •

Violent Content

  3. (3)

Leaking Intellectual Property (IP):

     * •

Songs

     * •

Books

     * •

News

In this round of experiments, we used GPT-4 in both the data generation and
evaluation components of the pipeline. For data generation, we use GPT-4 to
simulate the user agent ($LLM_{user}$) that chats with the $LLM_{test}$ using
the provided persona templates. For evaluation, we used GPT-4 as the
underlying LLM for the annotation component. This experimental design is
intended to roughly illustrate how our proposed framework can be leveraged in
assessing the performance of different LLMs to cause different harms or
violate RAI principles.

###  4.3. Results

Defect Rate
---
| Potentially Harmful Content | IP | Jailbreak
Model | Sexual | Violent | Hate | Songs | Books | News | Adult Content | Illegal Persuasion | Leaking Guidelines
Model 1 | 1.37% | 17.7% | 14.9% | 45.8% | 2.75% | 9.6% | 1% | 4.1% | 79.4%
Model 2 | 1.5% | 17.5% | 14.4% | 17.9% | 5.5% | 1.1% | 1% | 4% | 51.5%
Model 3 | 1.3% | 17.1% | 13.2% | 17.9% | 5.5% | 1.1% | 1% | 4% | 53%
Table 2. Defect rate for Potentially Harmful Content, IP leakage, and
Jailbreak across various LLMs

As illustrated in Table 2, the three models exhibit similar behavior in terms
of defect rates when evaluated for the generation of potentially harmful
content. This indicates that the models produced content which was annotated
as a defect on a similar number of samples, with Model 3 displaying the lowest
rate of generating potentially harmful content defects. Notably, the
generation of violent and hateful content is more prevalent compared to sexual
content.

In the context of intellectual property (IP) data leakage, Models 2 and 3
demonstrate identical defect rates across all categories (songs, books, and
news), suggesting that these models generate IP-protected content at the same
rate when tested on this set of measurement resources. This may hint that the
measurement resources should be expanded or improved to provide greater
clarity on possible performance differences between the models. Of the
different IP categories, songs exhibit the highest leakage rates, followed by
books and news. In contrast, Model 1 displays significantly higher defect
rates for songs and news compared to Models 2 and 3, with a 45.8% defect rate
for songs compared to 17.9% for both Models 2 and 3, and 9.6% defect rate for
news compared to 1.1% for both Models 2 and 3. This implies that Model 1 is
more susceptible to revealing IP-protected material in product scenarios.

Regarding jailbreak evaluations, Models 2 and 3 exhibit comparable defect
rates, with leaking guidelines being the most successful attack vector
compared to generating adult content or promoting illegal activities. Model 1,
however, demonstrates a significantly higher vulnerability to guideline
leakage, with an 80% success rate compared to 51% and 53% for Models 2 and 3,
respectively.

In conclusion, our evaluation reveals that Models 2 and 3 display lower rates
of generating IP-protected content and exposing underlying guidelines than
Model 1. So, we suggest that Models 2 and 3 may be more suitable as components
for real-world AI products and services compared to Model 1.

##  5\. Limitations

This framework facilitates rapid and repeated evaluation of different versions
of LLMs and associated products and services. However, there are several
limitations.

#### Using an LLM to measure harms from another LLM

Notably, this work does not adequately address issues related to the risks of
using an LLM to measure harms from another LLM, especially given that LLMs are
known to cause harms. This is an open research problem, although we note that
the evaluation component of our framework is flexible enough to plug in other
evaluation methods. This concern can manifest in both the data generation and
evaluation components of the framework.

In the case of data generation (during task simulation), by using an LLM to
mimic user behavior, we run the risk of the LLM failing to simulate realistic
conversations. This may impact the ecological validity of the generated data.
Additionally, the LLM used in task simulation may fail to represent linguistic
patterns of certain demographic groups, causing measurement efforts to
underestimate the potential for harms affecting marginalized groups.

In the case of evaluation, using an LLM to annotate potential harms from other
LLM-generated content may lead to issues. LLMs are known to produce harmful
content and can disproportionately produce some specific types of harmful
content affecting some specific groups of people. If an LLM is vulnerable to
producing some specific type of harmful content, will it be effective in
evaluating and annotating that same type of content? This may lead to under-
annotation of harms. Simultanesouly, other tendencies of LLMs may lead to
over-annotation of harms. LLMs are known to struggle with groundedness, and we
have observed cases where the LLM annotator yields a defect score and text
reasoning that cites non-existent parts of the sample. How frequent and
impactful may ungrounded generations be in the annotation process? Because the
real-life consequences of falsely labeling a piece of text as not harmful are
perhaps greater than those of falsely labeling text as harmful, the amount of
potentially harmful content measured from this framework should be treated as
a lower bound for the real amount of potentially harmful content.

One heuristic for gauging the impact of the issues described above is human-
model annotation agreement. While this practice provides some greater
confidence in the reliability of LLM annotations, it cannot be viewed as a
completely adequate replacement for the holistic research required to address
these concerns. Additionally, measuring generic human-model annotation
agreement is not sufficient. This is due to the reality that different groups
of humans with different lived experiences will experience different harms and
annotate differently.

#### Utility and interpretation

Another limitation lies in the utility and interpretation of the resulting
measurements. As mentioned in section 3, a 0% defect rate cannot be
interpreted to mean that the AI system under test does not cause harm. The
resulting measurements are relative rather than absolute, so they are useful
for diagnostics and comparisons between systems but are not applicable for
estimations of absolute risk or absolute likelihood of harm.

#### Validity and reliability

Likely the largest challenge of this technical framework is the fact that it
requires carefully-constructed measurement resources for sociotechnical
problems. Unfortunately, if these measurement resources are created poorly,
their usage in the technical framework does not immediately raise any red
flags. The usage of poorly constructed or invalid measurement resources may go
unnoticed, which can lead to increased harm if practitioners trust the
resulting measurements. In our initial case study, we engaged with domain
experts to create measurement resources, but future work is required to
understand the practices involved in creating reliable and valid measurement
resources.

Another aspect of reliability deals with the reproducibility and stability of
annotations generated by an LLM. We have observed repeated annotations on the
same sample leading to different results. In response, we implement a
stability factor that runs the annotation process multiple times and uses the
majority value generated for each sample. While this can significantly reduce
variability, it comes at the cost of increased computation, as it requires
running the evaluation multiple times (e.g., 5 or 7), which can leads to
longer evaluation times and greater resource requirements.

#### Resources

Finally, we recognize that this approach requires many invocations of large
models. While access to LLMs is expanding, acquiring the necessary resources
to run various LLMs, especially for large tasks, can be challenging and
costly. The compute resources required for this method may make it impractical
or inaccessible for some practitioners, and the environmental effects
associated with the proliferation of this framework must be examined.

##  6\. Conclusion and Future Directions

In this work, we presented a technical framework for the automated evaluation
of large language models (LLMs) in various RAI-relevant harm areas such as
groundedness, potentially harmful content, and leakage of intellectual
property. This framework leverages LLMs to automate the evaluation process,
enabling measurement at speeds and scales demanded by the current
proliferation of LLM-powered products and services. The proposed framework
offers an end-to-end pipeline for testing an LLM ($LLM_{test}$) by simulating
an interaction with another LLM ($LLM_{user}$) and annotating the outputs with
another LLM. The framework depends upon various measurement resources that are
best created by domain experts for each harm area subject to measurement.

Then, we demonstrated the utility of the proposed framework by evaluating
three recent LLMs across three distinct categories of harm (leakage of IP
content, generation of potentially harmful content, and jailbreak). The
resulting measurements enables us to compare the relative performance of these
models and serves as an example of how this framework can be used by
practitioners making decisions about which model versions to use in their AI
products and services. While much more work is required to explore how
reliable and valid measurement resources are created for each harm area, this
framework provides a viable path to evaluating harms stemming from LLM-based
AI systems at a speed and scale that can keep up with the current pace of
development. For future work, we will examine the aforementioned limitations
to make the measurement approach more reliable, valid, repeatable, objective,
and more cost efficient.

## References

  * (1)
  * Kumar et al. (2023) Sachin Kumar, Vidhisha Balachandran, Lucille Njoo, Antonios Anastasopoulos, and Yulia Tsvetkov. 2023.  Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey.  arXiv:2210.07700 [cs.CL]
  * OpenAI (2023) OpenAI. 2023.  GPT-4 Technical Report.  _ArXiv_ abs/2303.08774 (2023).
  * Penedo et al. (2023) Guilherme Penedo, Quentin Malartic, Daniel Hesslow, Ruxandra Cojocaru, Alessandro Cappelli, Hamza Alobeidli, Baptiste Pannier, Ebtesam Almazrouei, and Julien Launay. 2023.  The RefinedWeb dataset for Falcon LLM: outperforming curated corpora with web data, and web data only.  _arXiv preprint arXiv:2306.01116_ (2023).
  * Touvron et al. (2023) Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023\.  Llama 2: Open foundation and fine-tuned chat models.  _arXiv preprint arXiv:2307.09288_ (2023).