.tifpng.pngconvert #1 .tif

# Safety Control of Service Robots with LLMs and Embodied Knowledge Graphs

Yong Qi1, Gabriel Kyebambo0, Siyuan Xie0, Wei Shen0, Shenghui Wang0, Bitao
Xie0, Bin He0, Zhipeng Wang0, Shuo Jiang0
School of Electronic Information and Artificial Intelligence, Shaanxi
University of Science and Technology, Xian, Shaanxi Province, 710026, China

###### Abstract

Safety limitations in service robotics across various industries have raised
significant concerns about the need for robust mechanisms ensuring that robots
adhere to safe practices, thereby preventing actions that might harm humans or
cause property damage. Despite advances, including the integration of
Knowledge Graphs (KGs) with Large Language Models (LLMs), challenges in
ensuring consistent safety in autonomous robot actions persist. In this paper,
we propose a novel integration of Large Language Models with Embodied Robotic
Control Prompts (ERCPs) and Embodied Knowledge Graphs (EKGs) to enhance the
safety framework for service robots. ERCPs are designed as predefined
instructions that ensure LLMs generate safe and precise responses. These
responses are subsequently validated by EKGs, which provide a comprehensive
knowledge base ensuring that the robot’s actions are continuously aligned with
safety protocols, thereby promoting safer operational practices in varied
contexts. Our experimental setup involved diverse real-world tasks, where
robots equipped with our framework demonstrated significantly higher
compliance with safety standards compared to traditional methods. This
integration fosters secure human-robot interactions and positions our
methodology at the forefront of AI-driven safety innovations in service
robotics.

###### Index Terms:

Large Language Models, Embodied Robotic Control Prompts, Embodied Knowledge
Graphs, Service Robots, Safety Control

##  I Introduction

Service robots [1, 2] have been deployed in various sectors, including
healthcare assistance [3], customer service enhancement [4], and logistical
operations.

Their integration into human-centric environments [5] introduces significant
safety challenges [6, 7], necessitating rigorous scrutiny of their operational
reliability and task execution safety.

Addressing these safety challenges [8, 9, 10] is critical to establishing
service robots as reliable AI-driven collaborators. Various approaches have
been developed, including the use of large language models [11, 12, 13] and
knowledge graphs [14, 15] to guide service robot behavior.

However, the isolated use of LLMs [16, 17] in service robots has notable
limitations. LLMs, adept at natural language processing, often fail to
contextualize and provide factual information in real-world environments,
leading to inappropriate responses in complex situations.

Knowledge Graphs [18], on the other hand, enable service robots to access and
utilize vast amounts of factual and structured information, enhancing their
decision-making process. However, KGs alone are not equipped to handle the
nuances of natural language or unstructured data, limiting their applicability
in diverse interaction scenarios.

Recognizing the complementary strengths of LLMs and KGs [19, 20, 21], recent
research has explored their integration by combining the natural language
capabilities of LLMs with the structured, factual knowledge of KGs. When
integrated, these technologies allow service robots to better understand human
language and access contextually relevant information.

Our research proposes a novel approach beyond the conventional integration of
LLMs and KGs. We introduce Embodied Robotic Control Prompts (ERCPs) and
Embodied Knowledge Graphs (EKGs), specifically tailored for service robots.
ERCPs are custom prompt templates designed to assist LLMs in generating
consistent task plans for service robot operations.

EKGs are central to our framework, validating task plans generated by the LLMs
and providing comprehensive factual information to ensure the safe operation
of service robots.

###  I-A Problem Statement

The central issue lies in the limitations of service robots’ safe conduct and
operational reliability.

The absence of real-time validation mechanisms, factual information, and
logical reasoning capabilities hinders these robots’ ability to operate
optimally and safely in complex, real-world scenarios.

To address these limitations, we propose a solution that is efficient in
handling complex tasks in unforeseen environments:

  * •

We propose the use of custom prompt templates for service robots: Embodied
Robotic Control Prompts (ERCPs).

  * •

We propose the use of Embodied Knowledge Graphs (EKGs) to guide the output of
LLMs for service robots while providing comprehensive factual information for
safe task completions.

  * •

We propose the integration of LLMs with ERCPs and EKGs to ensure the safe
operation of service robots.

Figure 1: An overview of the integration of Language Models (LLMs), Embodied
Robotic Control Prompts (ERCPs), and Embodied Knowledge Graphs (EKGs).

###  I-B ERCPs and EKGs

Prompt engineering [22, 23, 24] and custom templates [25] enhance LLMs by
ensuring precise, context-relevant interactions, significantly boosting
efficiency and accuracy in responses across varied applications, from
conversational AI to complex decision-making.

The Embodied Robotic Control Prompts (ERCPs) are specialized prompt templates
that guide robotic operations by breaking down broad user commands $c$ into
actionable steps, enabling more efficient communication between humans and
robots. ERCPs use a weighted combination of user commands $c$, composed text,
and APIs to enhance decision-making within Large Language Models (LLMs).

Embodied Knowledge Graphs (EKGs) are store information in triple formats,
helping service robots understand their environments, entities, relationships,
and obstacles. EKGs not only contain multimodal data but also rigorously
validate the responses of large language models (LLMs) to ensure safe task
execution.

The representation of an EKG can be denoted as

$\text{EKG}=\\{(h,r,t,a)\subseteq\mathcal{E}\times\mathcal{R}\times\mathcal{E}\times\mathcal{A}\\},$

where $\mathcal{E}$ is the set of entities, $\mathcal{R}$ is the set of
relations, and $\mathcal{A}$ is a set of attributes or additional information
specific to embodied contexts.

The integration of ERCPs and EKGs in service robotic systems forms a robust
framework. ERCPs facilitate human-robot interactions and the generation of
appropriate task sequences in LLMs, while EKGs offer a structured
representation of the robot’s environment and ensure stringent validation of
task sequences.

A feedback loop within our framework ensures consistently updated data between
the ERCPs and the EKGs. The APIs within the ERCPs are crucial for generating
safe task sequences; one of the fundamental functionalities of the EKGs is to
ensure these APIs are updated in real-time for subsequent tasks.

This feedback system updates the Embodied Knowledge Graph (EKG) and the action
primitive repositories (APIs), integral components of the ERCP. This update
process makes the ERCP more dynamic and responsive to changes in the real
world or corrections to initial outputs.

This adaptability ensures the system evolves based on actual interactions and
outcomes, enhancing the accuracy and relevance of the task sequences generated
for service robots and refining the knowledge stored within the EKG.

##  II Related Work

The integration of Large Language Models (LLMs), custom prompt templates
(ERCPs), and multi-modal knowledge graphs (EKGs) is essential for advancing
the safety and reliability of robotic task execution. Previous research has
explored various aspects of these components individually, demonstrating their
potential to enhance service robot performance. In this section, we review the
significance of LLMs, ERCPs, and EKGs, and discuss how their integration
addresses current challenges and improves best practices in service robot
safety.

###  II-A Large Language Models

Large Language Models (LLMs) have become instrumental across various fields,
including robotics. They are pivotal for achieving multi-modal semantic
understanding, significantly enhanced through advancements in fine-tuning and
pretraining methods [26, 27, 28].

LLMs leverage text descriptions to extract semantic information across
different modes, facilitating a deeper understanding of environmental contexts
[29]. This capability is crucial for executing tasks via API action
primitives, ensuring that robots can interact more intuitively with their
operational environments [30, 31].

Significant contributions to this domain include the work of J. Wu et al.
[32], who developed a method for integrating language-based planning with
perception capabilities in robotic systems. This approach enables robots to
rapidly adapt and infer generalized user preferences with minimal prior
interactions, thus enhancing user-centric robotic applications.

Ding et al. [33] applied LLMs to the complex task of multi-object
rearrangement, demonstrating the model’s capability to enhance robotic motion
planning and execution. Their research highlights the utility of LLMs in
managing intricate task sequences that require high-level cognitive functions.

Innovative applications such as CodeBotler by Hu et al. [34] and LLM-Grounder
by Yang et al. [35] illustrate the versatility of LLMs in programming service
robots and enhancing 3D visual grounding, respectively. These systems
underscore the significant potential of LLMs to transform how robots perceive
and interact with their environments.

Furthermore, ProgPrompt by Singh et al. [36] and the Code as Policies
initiative by Jiang et al. [37] showcase the adaptation of LLMs for generating
dynamic task plans and policy codes directly from natural language
instructions. This highlights a shift towards more agile and adaptable robotic
behaviors, supported by advanced language understanding capabilities.

Our research proposes the integration of LLMs with custom prompt templates
(ERCPs) and knowledge graphs (EKGs). This approach not only addresses the
execution of complex robotic tasks but also significantly enhances operational
safety. By incorporating ERCPs and EKGs, we aim to bridge the gap between
robotic actions and human expectations, ensuring that every operation adheres
to stringent safety standards and is informed by a comprehensive understanding
of the task environment.

###  II-B Prompt Engineering

The development of custom prompt templates for guiding Large Language Models
(LLMs) in executing downstream tasks represents a significant advancement in
computational linguistics. Cloze prompts, as developed by Jia et al. [38], and
prefix prompts by Chen et al. [39], exemplify structured methods that enhance
LLMs’ task-specific performance.

Ding et al. [40] provide a comprehensive overview of prompt design techniques,
categorizing them into manual and automatic methods. While manual methods
offer bespoke customization, automatic methods, particularly those generating
cloze prompts, are favored for their scalability and standardization. The
prefix prompt method is noted for its flexibility, allowing LLMs to
autonomously generate task-aligned outputs within set parameters.

The experimental study ”ChatGPT for Robotics” by Vemprala et al. [41]
showcases the integration of prompt engineering with robust function
libraries. This approach enhances the adaptability of ChatGPT across diverse
robotics applications—including different simulators and hardware setups—and
critically assesses the efficacy of various prompt engineering strategies in
practical robotic tasks, highlighting the model’s flexibility in real-world
scenarios.

Building on these foundational insights, our research introduces Embodied
Robotic Control Prompts (ERCPs), specifically designed for service robots.
These ERCPs incorporate environmental data and action primitives into a
coherent framework, enabling dynamic responses to the nuanced demands of
service tasks. This innovative approach leverages a JSON-formatted dataset,
systematically segmented into function calls and APIs, to provide a structured
yet adaptable template for task execution.

The action primitives are stored within a JSON-based repository, designated
for specific robotic tasks, ensuring that the LLMs are equipped with the
necessary context for task execution. Prefixed natural language descriptions
further aid the LLMs in interpreting the semantic layers of the tasks, thereby
enhancing the robots’ operational efficiency and contextual adaptability in
various settings.

This strategic integration of advanced prompt engineering techniques with LLMs
facilitates a new level of sophistication in robot interactions, ensuring that
robotic systems are not only responsive but also contextually aware and
safety-oriented in their operational environments.

Figure 2: An overview of the Embodied Robotic Control Prompts $(ERCPs)$
showing its structure, the combination of user queries and action primitives

###  II-C Knowledge Graphs

Knowledge Graphs (KGs) are increasingly central to advancing robotics,
significantly improving robot manipulation, autonomous task planning, and
human-robot interaction.

A pioneering framework for ”Semantic Representation of Robot Manipulation with
Knowledge Graph” was introduced by Miao et al. [42]. This framework employs a
multi-layered semantic representation model through a KG, enhancing robot
manipulation precision and task execution with high-precision predictions that
are pivotal in real-world applications.

Bai et al. [43] explored ”A Dynamic Knowledge Graph Approach to Distributed
Self-Driving Laboratories,” which integrates various laboratory components
into a cohesive digital twin via dynamic KGs. This method adapts seamlessly to
changing research objectives and facilitates the integration of new resources,
demonstrating the vital role of dynamic KGs in real-time control and
mitigating geographical constraints.

In the realm of human-robot collaboration, Ding et al. [44] proposed using KGs
to enhance complex tasks like disassembly. Their method equips robots with
enhanced knowledge and reasoning capabilities, fostering improved interaction
and collaboration in executing tasks.

Peng et al. [45] explored the challenges and opportunities in KGs in their
publication ”Knowledge Graphs: Opportunities and Challenges.” They highlighted
critical issues such as addressing long-tail knowledge, biases, and the
essential need for explainability and interpretability within KG systems.

Building on these seminal works, our research introduces Embodied Knowledge
Graphs (EKGs), designed specifically to support the autonomous execution of
tasks by service robots. EKGs play a crucial role in validating responses from
Large Language Models (LLMs) and provide structured, factual data that
enhances operational safety and accuracy. This integration addresses critical
safety concerns and potential inaccuracies, marking a significant step forward
in the application of KGs in robotics.

To refine and enhance the integration of Large Language Models (LLMs) and
Embodied Knowledge Graphs (EKGs), we incorporate recent research insights and
approaches identified in current studies.

###  II-D Integrating $LLMs$ and $(EKGs)$

The fusion of Large Language Models (LLMs) and Knowledge Graphs (KGs) has been
a focal point of recent research, aimed at harnessing the complementary
strengths of these technologies to address their individual limitations.

Recent advancements highlighted by Yang et al. [46] have demonstrated that
while LLMs like ChatGPT excel at predicting language patterns, their ability
to recall factual, knowledge-grounded content remains limited without the
support of KGs. This has led to the exploration of KGs as essential tools for
enhancing the factual accuracy of LLM outputs.

Furthermore, Alam et al. [47] have proposed innovative methods to improve KG
embeddings for graph completion tasks, which help overcome traditional
challenges of KGs, such as neglecting rich textual information. This approach
underscores the evolving synergy between textual and knowledge-based AI
systems.

Shirui Pan et al. [48] contribute to this dialogue by mapping out various
strategies for integrating LLMs with KGs. Their work lays the groundwork for
approaches like KG-enhanced LLM inference, which effectively merges textual
data with structured knowledge to enrich both the comprehension and generation
capabilities of AI models.

Despite these innovations, the integration of LLMs with KGs faces challenges
such as the need for frequent updates and the difficulty in generalizing to
unseen data [49]. Dynamic Knowledge Fusion methods like the Two-Tower
Architecture [50] attempt to address these issues by processing text and KG
inputs separately to enhance interaction, but often lack integrated feedback
loops between the two data streams.

More advanced models like KagNet [51] and MHGRN [52] introduce mechanisms for
augmenting text with KG inputs, providing a more nuanced interaction, albeit
typically in one direction. Enhancements such as those introduced by GreaseLM
[53], which facilitates deep interaction across all layers of the model, and
the bi-directional attention mechanisms of QA-GNN [54] and JointLK [55],
represent significant strides towards creating a more robust, interactive
system.

Sakib et al. [56] utilize Large Language Models (LLMs) to generate task trees
that are subsequently integrated into a unified graph structure to optimize
task execution. This integration strategically eliminates paths that are less
probable and resource-intensive by leveraging the predictive capabilities of
LLMs:

$G=\bigcup_{i=1}^{n}T_{i}-\\{\text{high-cost paths}\\}$

where $T=\\{T_{1},T_{2},\ldots,T_{n}\\}$ represents the task trees generated
by the LLM, and $G$ denotes the resulting optimized graph. A cost function $C$
evaluates the paths for efficiency and reliability:

$\text{Optimized Plan}=\arg\min_{p\in G}C(p)$

Contrastingly, our methodology extends beyond using LLMs for task tree
generation by integrating Embodied Robotic Control Prompts (ERCPs) and
Embodied Knowledge Graphs (EKGs), which serve as the primary knowledge base.
This integration emphasizes operational safety alongside efficiency:

$S=F(\text{LLM}(\text{Tasks}),\text{EKG}(K),\text{ERCP}(P))$

where $F$ functionally combines task plans Tasks derived from LLM outputs with
structured data from the knowledge graph $K$ and dynamic inputs from control
prompts $P$. This system not only orchestrates task execution but also
rigorously evaluates each action for safety and contextual appropriateness
through a real-time function $E$:

$\text{Validated Plan}=\max_{a\in S}E(a)$

This ensures that every action not only meets efficiency standards but also
adheres strictly to rigorous safety protocols, which are critical in high-
stakes environments.

While Sakib et al.’s methodology optimizes task execution by refining LLM-
generated task plans, our approach emphasizes robust validation of each
robotic action using a knowledge-based system. By embedding advanced control
mechanisms and factual validation within the operational framework of LLMs, we
ensure a dual focus on both safety and efficiency.

TABLE I: Comparative Analysis of Key Features: Knowledge Graphs as a Knowledge Base, Operation in Unseen Environments, Plan Visualization and Correction, Optimization of Plans, and Safety Evaluations — Our Approach vs. Related Works |  KG as KB? |  Unseen Env. |  Plan Vis. |  Plan Corr. |  Optimize Plan |  Safety Eval.
---|---|---|---|---|---|---
CodeBotler |  x |  – |  ✓ |  ✓ |  ✓ |  x
Code as Policies |  x |  ✓ |  ✓ |  ✓ |  ✓ |  x
ProgPrompt |  x |  ✓ |  ✓ |  ✓ |  x |  x
ToT |  x |  ✓ |  ✓ |  ✓ |  ✓ |  x
Ours |  ✓ |  ✓ |  ✓ |  ✓ |  ✓ |  ✓
Figure 3: The overall architecture of our framework, which takes user commands
through voice recognition systems. The transcribed audio is then added to the
action primitives and special text compositions to form the ERCP. The ERCP is
then fed into the LLM to generate the necessary task sequences which are then
validated by the EKGs before the task is executed.

##  III Methodology

This section outlines the operational framework of our proposed model, which
integrates Large Language Models (LLMs) with Embodied Robotic Control Prompts
(ERCPs) and Embodied Knowledge Graphs (EKGs) to ensure safe and efficient
robotic task execution. Our approach begins with the transcription of user
commands into textual representations, which are archived in a dedicated file.
These user commands, denoted as $c$, are then retrieved along with a JSON
repository containing Action Primitives (APIs). These elements are
strategically combined to create the ERCP, which serves as the foundation for
generating and validating task sequences.

The ERCP is subsequently inputted into an LLM, specifically OpenAI’s
ChatGPT-4. ChatGPT-4, leveraging a multi-layered, attention-centric
transformer architecture, processes the ERCP to generate coherent and relevant
task sequences. The attention mechanism within the transformer architecture
enables ChatGPT-4 to prioritize key elements of the ERCP, ensuring the
generated sequences are contextually appropriate.

Following the generation of task sequences, the LLM converts these sequences
into Cypher queries, which are then validated against the EKG. The EKG, built
on Neo4j’s architecture, includes comprehensive information such as spatial
data, physical attributes, sensor readings, temporal dynamics, task
parameters, and safety guidelines.

Validation involves executing the Cypher queries, which incorporate user
preferences, object details, and locational data. The EKG also verifies
additional attributes like object size, weight, and availability to guarantee
effective task execution.

Regardless of the validation outcome, our feedback system captures the log
outputs from the task sequences. These logs are used to update the EKG and the
APIs within the ERCP, ensuring continuous system improvement and adaptability.

###  III-A Problem Formulation

This subsection outlines the computational model for task planning in a
service robot, designated to perform cleaning within a defined space, referred
to as $R$. The robot’s operation is conceptualized as executing a series of
discrete actions, expressed as $A=\\{a_{1},a_{2},\ldots,a_{n}\\}$, where each
action $a_{i}$ is aimed at achieving part of the overall task. The operational
domain $S$ includes an array of objects $O=\\{o_{1},o_{2},\ldots,o_{m}\\}$,
each characterized by spatial coordinates $(x_{i},y_{i})$ and physical
attributes $P=\\{p_{1},p_{2},\ldots,p_{k}\\}$.

To navigate and interact within this environment, the robot employs an array
of sensors, denoted as $Sens=\\{s_{1},s_{2},\ldots,s_{l}\\}$, which includes
several Intel RealSense cameras. These cameras utilize advanced computer
vision techniques to accurately detect and process objects and details within
the robot’s environment, providing essential real-time data. Additionally, the
robot’s decision-making processes are informed by an understanding of temporal
dynamics and events, represented by $T=\\{t_{1},t_{2},\ldots,t_{q}\\}$,
further enhancing its capability to respond dynamically to changes in its
surroundings.

The primary goal is to derive an optimal sequence of actions $A^{*}$ that
efficiently completes the cleaning task while complying with predefined safety
and environmental constraints. This is formally defined as:

$A^{*}=\arg\min_{A}f(A,S,O,P,Sens,T)$

Here, $f$ represents a multi-dimensional objective function that integrates
spatial relationships, physical attributes, sensory inputs, temporal factors,
and safety regulations. The function aims to determine the sequence of actions
$A^{*}$ that minimizes the cost or maximizes the efficiency of cleaning the
specified area $R$.

The resolution of this objective function, $f(A,S,O,P,Sens,T)$, results in the
optimal action sequence $A^{*}$, enabling the service robot to execute its
tasks with maximum efficacy within the stipulated environmental settings. This
sequence, $A^{*}$, epitomizes a harmonized balance among the various
operational constraints and objectives, highlighting the robot’s advanced
capability to adapt and respond to dynamic environmental conditions.

###  III-B Embodied Robotic Control Prompt

User commands, articulated in natural language, establish the initial
linguistic context to be transformed into actionable directives for robots.

To facilitate this transformation, user commands are integrated with bespoke
textual compositions and predefined action primitives, forming the core of our
Embodied Robotic Control Prompt (ERCP) framework.

This framework leverages the sophisticated language comprehension capabilities
of Large Language Models (LLMs), such as ChatGPT-4, to convert the ERCPs into
precise task plans and query trajectories.

Consider a practical example where a user command, ”I am thirsty,” is given.
In its basic form, this command lacks the specificity needed for LLMs to
generate actionable task plans tailored to the user’s exact requirements.

Our ERCP framework addresses this by incorporating strategic prompting within
the template, prompting the LLM to seek additional information, such as ”What
would you like to drink?” and ”Do you prefer your drink cold or warm?” This
iterative clarification process refines the query, resulting in more targeted
and appropriate responses.

The sequential prompting technique central to our methodology systematically
guides the model through a structured series of steps to achieve the desired
outcome. The process starts with an initial state $S_{0}$ and the first prompt
$P_{1}$. The model responds to this prompt $P_{i}$ considering the previous
state $S_{i-1}$, as described by:

$R_{i}=L(P_{i},S_{i-1})$

Upon obtaining the response $R_{i}$, the state is updated to $S_{i}$, which
integrates the new information from $R_{i}$, illustrated by:

$S_{i}=f(S_{i-1},R_{i})$

Following the state update, the next prompt $P_{i+1}$ is generated based on
the updated state $S_{i}$, and can be expressed as:

$P_{i+1}=g(S_{i})$

This iterative cycle—spanning response generation, state updating, and prompt
reformulation—continues through a sequence of prompts
$\\{P_{1},P_{2},\ldots,P_{n}\\}$ and their corresponding responses
$\\{R_{1},R_{2},\ldots,R_{n}\\}$. This facilitates the model’s navigation
through complex tasks and reasoning pathways in a step-by-step fashion until
the desired outcome is achieved.

In scenarios where the LLMs are derived from open-source models, such as
GPT-3, GPT-2, ChatGLM, or LLAMA, adjustments to the model’s weights during the
training phase can be implemented to enhance performance in task plan
generation. Utilizing a weighted linear combination approach, the weights
$w_{u}$, $w_{t}$, and $w_{api}$ represent the relative importance of user
commands, composed text, and APIs respectively. This adjustable weighting
scheme enables precise customization of the model’s responses to meet specific
user needs and objectives, often resulting in superior task execution.

The components of the ERCP model are defined as follows:

  * •

User Commands: $\mathbf{u}=[u_{1},u_{2},\ldots,u_{m}]$, where $u_{i}$
represents the $i$th element of user commands.

  * •

Composed Text: $\mathbf{t}=[t_{1},t_{2},\ldots,t_{n}]$, where $t_{j}$
represents the $j$th element of composed text.

  * •

APIs from Repository: $\mathbf{api}=[api_{1},api_{2},\ldots,api_{k}]$, where
$api_{k}$ represents the $k$th API in the JSON repository.

The ERCP can be mathematically modeled as:

$\text{$ERCP$}=w_{u}\cdot\mathbf{u}+w_{t}\cdot\mathbf{t}+w_{api}\cdot\mathbf{api}$

Adjusting the weights $w_{u}$, $w_{t}$, and $w_{api}$ allows for emphasis on
the influence of each component in the decision-making process within the LLM.
For instance, a higher weight for user commands might prioritize human intent,
enhancing the alignment of outcomes with user expectations. Conversely,
increasing the weight for composed text could enhance the contextual
understanding of the model, improving its ability to generate coherent and
contextually appropriate responses. Similarly, adjusting the weight for APIs
can influence how effectively the model integrates and leverages external
functionalities.

This linear combination captures the integration of different elements to
ensure that the required output is provided for safe and effective task
completion, thereby optimizing the performance of LLMs in practical
applications.

###  III-C Embodied Knowledge Graph (EKG)

The EKG is structured as a graph database, utilizing nodes and relationships
to represent a comprehensive model of the operational environment, including
tasks (e.g., fetch_item, grab, pour), objects (e.g., cups, bottles), locations
(e.g., kitchen_table, fridge), obstacles (e.g., chair), and sensor data (e.g.,
cmd_vel, grip_force).

Each node in the graph is associated with attributes that detail its physical
characteristics and spatial relations, offering a granular view of the service
robot’s working domain.

Our approach emphasizes the EKG’s role in the task sequence validation
process. Upon generation of Cypher queries by the LLM, these queries are
cross-referenced with the EKG to ensure each action’s feasibility and safety.

The querying process employs Cypher, Neo4j’s graph query language, to perform
dynamic, complex queries. A query might check for the presence of obstacles
along a navigation path, validate the location of objects required for task
completion, and ensure all actions align with safety protocols and physical
constraints of the environment.

The structure of the Embodied Knowledge Graph can be denoted as $f(h,r,t)$,
where:

  * •

$h$ and $t$ represent entities.

  * •

$r$ represents the relationship between these entities.

Figure 4: The structure of the EKG of an indoor_task depicting the
relationships and entities.

We denote the relationship within the EKG as $f(h,r,t,a)$, where:

  * •

$h$ and $t$ represent the entities within the graph, such as the service robot
or the target object.

  * •

$r$ corresponds to the relationships that bind these entities, exemplified by
the spatial, interactive, or causal links between them.

  * •

$a$ signifies the attributes or additional information specific to embodied
contexts, enhancing the representation with details such as sensor readings or
operational states.

Consider a practical scenario in which a service robot is tasked with the
activity fetch_item. The EKG utilizes $h$ to represent the robot’s current
location, $t$ to signify the item to be retrieved, such as bread, and $r$ to
denote the potential presence of obstacles.

Upon the detection of an obstacle, which the EKG encodes as an additional
node, the graph is dynamically updated to reflect this new environmental
challenge. The robot, leveraging this updated information, employs an
alternate strategy to navigate around the hindrance.

In tasks requiring optimization, such as instructing a robot to clean a room,
the EKG serves as a structured foundation augmented with task-specific APIs
and their operational sequences. This includes:

  * •

A set of objects $O$ within the room, each characterized by spatial
attributes.

  * •

An array of potential actions $A$ available for execution by the robot.

  * •

Sensor data $Sens$ informing the robot of its internal and environmental
states.

  * •

Temporal constraints $T$ that dictate the timing and sequence of actions.

  * •

Safety and environmental constraints ensuring operations within predefined
parameters.

The overarching goal of the EKG is encapsulated by the optimization function
$f:\mathcal{P}(A)\times KG\rightarrow\mathbb{R}$, where $\mathcal{P}(A)$
represents the power set of all possible actions and $KG$ the knowledge graph
domain. The function $f$ maps each subset of actions $A$ and the knowledge
graph $KG$ to a real number indicative of the cost associated with that action
sequence. The optimal action sequence $A^{*}$ is determined by:

$A^{*}=\underset{A\subseteq\mathcal{P}(A)}{\arg\min}f(A,KG)$ (1)

under the conditions:

$C(A,KG):=\\{\phi\mid\forall a\in A,\text{ the preconditions of }a\text{ are
satisfied in }KG,\text{ and the post-conditions of }a\text{ lead to a valid
state in }KG\\}$ (2) $C(A,KG)\neq\varnothing$ (3)

A predicate function $V:A\times KG\rightarrow\\{True,False\\}$ validates the
alignment of actions generated by the LLM with the constraints and knowledge
encapsulated within the EKG:

$V(a,KG)=\begin{cases}True&\text{if }a\text{ is consistent with }KG\\\
False&\text{otherwise}\end{cases}$

Should human intervention necessitate overriding predefined constraints, a
function $H:KG\times A\rightarrow KG^{\prime}$ modifies the knowledge graph
state to reflect the updated trajectory:

$\displaystyle KG^{\prime}$ $\displaystyle=$ $\displaystyle H(KG,\\{a\mid
V(a,KG)=False$ (4) $\displaystyle\text{and human override is applied}\\})$

The resulting path is thus adapted as:

$\displaystyle p(h,n,t)^{\prime}$ $\displaystyle\subseteq$ $\displaystyle
KG^{\prime}$ (5)

indicating the navigational path $p$ under human intervention $n$ leads to the
target $t$ while circumventing the now-invalidated obstacles. This dynamic
interplay of graph updates and decision-making mechanisms ensures the robot’s
actions are both feasible and aligned with current environmental conditions
and operational objectives.

This involves:

  * •

Verifying the existence and location of target objects (e.g., verifying a cup
is located at the kitchen_table).

Listing 1: Verifying object location

⬇

MATCH (obj:Object {name: 'cup'})-[:LOCATED_AT]

->(loc:Location {name: 'kitchen_table'})

RETURN obj.name AS Object, loc.name AS Location

  * •

Assessing navigation paths for potential obstacles, utilizing the graph’s
spatial information to navigate the robot efficiently and safely.

Listing 2: Assessing navigation paths

⬇

MATCH path=(:Location {name: 'start_location'})

-[:CONNECTS_TO*]->(:Location {name: 'destination'})

WHERE NONE(rel IN relationships(path) WHERE type(rel) = 'HAS_OBSTACLE')

RETURN path AS SafePath

  * •

Confirming task prerequisites and conditions, such as object accessibility or
the need for specific manipulative actions (e.g., grab, pour).

Listing 3: Confirming task prerequisites

⬇

MATCH (obj:Object {name: 'cup'})

-[:LOCATED_AT]->(loc:Location),

(task:Task {name: 'grab'})

-[:REQUIRES_OBJECT]->(obj),

(task)-[:REQUIRES_CONDITION]

->(cond:Condition)

WHERE obj.accessible = true AND cond.name IN ['grip_force_within_range',
'no_obstacles']

RETURN task.name AS Task, obj.name AS Object, COLLECT(cond.name) AS
ConditionsMet

###  III-D Task Execution and Feedback Loop

Following the successful validation of a task sequence by the Embodied
Knowledge Graph (EKG), the execution phase is initiated. This phase involves
the robot performing a series of actions, curated and sequenced based on the
validation results provided by the EKG. The actions are selected from a
repository and organized to match the specific requirements validated by the
EKG, incorporating necessary parameters to guarantee safe execution.

Task execution is conceptualized as a Markov Decision Process (MDP), which
systematically defines the robot’s interactions with its environment through
states, actions, and rewards. This formalism helps guide the robot towards
achieving its goals effectively while maintaining adherence to safety and
efficiency standards.

Central to the continuous improvement of robotic operations is the feedback
loop, structured as a supervised learning mechanism. This loop captures and
analyzes the outcomes of the robot’s actions along with the rewards received
to refine the task planning and execution processes.

The components of the feedback loop include:

  * •

Feedback:

    * –

Input: Comprises $\text{Input}(State,Action,Reward)$, where State, Action, and
Reward are derived from the ongoing task execution.

    * –

Output: Defined as $\text{Output}(Feedback)$, this represents the insights
gained from analyzing the robot’s performance.

    * –

Update: Denoted as $\text{Update}(ERCP^{\prime},EKG^{\prime})$, this process
involves applying the learned modifications to the ERCP and updating the EKG
to reflect new environmental information or operational insights.

This feedback loop ensures that all processes within our model are
meticulously recorded, and key data such as updated environmental conditions
are captured. These data are crucial for continually refining the action
primitives and updating the EKG, thereby enhancing the robot’s performance and
adaptability in dynamic environments.

##  IV Experiments

To evaluate the effectiveness and robustness of our proposed framework, we
conducted a series of experiments using two different robots: the Realman
Intelligent Robot and the Yahboom Jetson TX2. These experiments aimed to test
the robots’ ability to execute complex tasks in dynamic environments using the
integrated system of Large Language Models (LLMs), Embodied Robotic Control
Prompts (ERCPs), and Embodied Knowledge Graphs (EKGs).

The experimental procedure began with translating user commands into textual
representations. These commands, along with action primitives, were stored as
JSON objects. The LLM (ChatGPT-4) processed these inputs to generate detailed
and contextually relevant task plans. The generated plans were then validated
against the EKG to ensure feasibility and safety before execution.

The primary objectives of these experiments were to:

  * •

Assess the accuracy and reliability of task execution using the proposed
framework.

  * •

Evaluate the ability of the system to adapt to dynamic changes in the
environment.

  * •

Validate the effectiveness of the feedback loop in refining task plans and
improving robot performance over time.

###  IV-A Evaluation Metrics

To comprehensively assess the efficacy of our framework in task execution, we
employed a set of evaluation metrics: Success Rate (SR), Executability (Exec),
Object Handling (OH), Safety Violations (SV), and Error Recovery (ER). SR
measures the fraction of executions achieving all goal conditions, while Exec
quantifies the executability of all planned actions within the environment.

OH assesses the system’s proficiency in interacting with and handling objects.
SV monitors breaches of safety protocols, and ER gauges the robot’s capacity
to recover from errors.

###  IV-B Yahboom-Robot Experiment

The experiments with the Yahboom ROSMASTER-X3 PLUS robot were designed to
assess its fetching and navigation capabilities. This setup incorporated high
frame rate cameras (Intel RealSense Camera, ORBEC FHD-1080P) to continuously
collect real-time data for precise obstacle detection and target
identification.

Detected objects and their relative positions were stored within the $EKG$,
facilitating real-time queries essential for assisting the robot in navigation
and task execution. The $EKG$ continuously evaluated potential collision
threats and optimized the robot’s motion commands to ensure safe and efficient
navigation. Key parameters monitored included the robot’s position ($x,y$),
orientation angle ($\theta$), linear speed ($\upsilon$), angular speed
($\omega$), and time step ($\delta t$). This comprehensive data integration
within the $EKG$ enabled precise and adaptive control, enhancing the robot’s
ability to interact with its environment effectively.

To test the system’s adaptability and functionality, user commands such as
”Pick up the bread” and ”Find the trash on the floor and put it in the trash
bin” were implemented. These commands were chosen to challenge the system’s
ability to handle tasks of varying complexity and environmental interaction.

Figure 5: The service robot performing a sequence of actions: locating the
object bread, navigating to the initial location, picking up the bread, and
moving to the final target location.

We detail the operational sequence employed by the Large Language Model (LLM)
for a typical bread fetching task. The sequence, validated by the Embodied
Knowledge Graphs (EKG), involves the following steps:

  1. 1.

Detecting the object and its location: find(obj: bread),

  2. 2.

Navigating to the object’s location: nav_goal{loc:(x, y)},

  3. 3.

Fetching the item while considering its orientation: pick_up{obj:bread},

  4. 4.

Returning to the initial or designated location: nav_goal(final_location).

This approach illustrates the methodical actions undertaken by the service
robot, highlighting the framework’s ability to generate and execute complex
task sequences autonomously.

In our experimental evaluation, the robot’s performance was rigorously tested
across a diverse set of tasks, including locating items, fetching and
delivering items, and cleaning.

To accurately assess the robustness and adaptability of our framework, we
introduced random distractors during each task, ensuring that our evaluations
reflect the real-world variability and challenges.

Each set of experiments was conducted a minimum of twenty times to validate
the robustness of our framework. This repeated testing strategy ensures that
our performance metrics, designed to measure the system’s efficiency and
effectiveness in real-world operations, are robust and reliable under various
conditions.

TABLE II: Performance Metrics for Additional Tasks on the Yahboom Robot Tasks |  SR |  Exec |  OH |  SV |  ER
---|---|---|---|---|---
Locating Items |  0.95 |  0.80 |  0.89 |  0.15 |  0.87
Fetching and Delivering |  0.93 |  0.78 |  0.79 |  0.18 |  0.95
Cleaning |  0.92 |  0.87 |  0.89 |  0.20 |  0.88

###  IV-C Realman Intelligent Robot Experiment

Experimenting with our model on the Realman Intelligent Robot, we performed
tasks such as sorting, pick-and-place, and pouring water from one cup to
another.

Figure 6: This demonstration highlights our framework’s ability to identify
and sort relevant objects and perform tasks like pouring liquid.

###  IV-D Configuring the Robot Arm

To ensure precise execution, the $EKG$ manages the robot’s active joint
configurations. It sets pose ranges for each joint and refines the actions
derived from the inverse kinematics solutions.

  * •

Validation of the transformation matrix $T\\_target\\_inv$ relative to the
current pose. This matrix represents the transformation required to transition
from the current position to the stored target pose.

  * •

From $T\\_target\\_inv$, the $EKG$ extracts the rotation matrix $R\\_error$
and the translation vector $P\\_error$. These components are critical for
accurately adjusting the robot’s pose to achieve the target configuration.

###  IV-E Gripper

The gripper’s functionality is crucial for the safe handling of objects,
including grabbing, fetching, and pouring.

To effectively manipulate various objects, we have developed a dataset
containing the physical attributes and required gripping forces for different
items, stored in the Embodied Knowledge Graph (EKG). This repository enables
precise and efficient interactions by leveraging detailed object information:

  * •

Object Pose Analysis (OPA): Uses a depth camera to determine an object’s 3D
orientation, generating a relative position vector $P_{relative}$ and
orientation quaternion $R_{relative}$. These parameters are stored in the EKG
and can be queried in real-time to inform task strategies.

  * •

Grasping Strategy Formulation (GSF): Derives an optimal grasping strategy from
the EKG, considering $P_{relative}$ and $R_{relative}$, tailored to the
object’s dimensions and the capabilities of the gripper. This strategy is
formulated based on real-time or pre-task queries to the EKG to ensure
adaptability and safety.

  * •

Gripping Parameters (GP): Records essential gripping data, such as the desired
force $F_{desired}$, in the EKG. These parameters are adjusted according to
object properties and gripper specifications and are accessed from the EKG
before or during task execution to optimize grip and ensure object security.

  * •

Task Execution Poses Retrieval (TEPR): Retrieves $P_{relative}$ and
$R_{relative}$ from the EKG during grasping tasks to ensure precise object
manipulation. This retrieval process involves querying the EKG in real-time to
adapt to any changes in the object’s orientation or position.

TABLE III: Performance Metrics for Various Tasks on the Yahboom Robot Tasks |  SR |  Exec |  OH |  SV |  ER
---|---|---|---|---|---
Sorting |  0.98 |  0.80 |  0.89 |  0.12 |  0.93
Pouring Water |  0.94 |  0.92 |  0.10 |  0.18 |  0.87
Pick-and-Place |  0.96 |  0.96 |  0.89 |  0.20 |  0.94

##  V Results

###  V-A Comparative Analysis Methodology

To rigorously evaluate the performance of our proposed framework (ERCP + LLMs
+ EKG), we conducted a series of comparative analyses against established
baselines. The baselines were categorized into three groups: LLM-only systems,
KG-only systems, and LLM + KG systems. The objective was to highlight the
effectiveness, safety, and adaptability of our approach in generating
executable and safe task plans in dynamic environments.

####  V-A1 LLM-Only Systems

For the LLM-only comparison, we selected three state-of-the-art models:

  * •

OpenAI GPT-3: A leading LLM used for generating task sequences based purely on
language understanding.

  * •

BERT (Bidirectional Encoder Representations from Transformers): Utilized for
generating task instructions and processing user commands.

  * •

ProgPrompt: Uses LLMs to generate situated task plans for robots, leveraging
LLMs’ language understanding capabilities to guide task execution.

####  V-A2 KG-Only Systems

For the KG-only comparison, we selected two systems known for their
comprehensive knowledge representation:

  * •

RoboBrain: An extensive knowledge engine integrating diverse sources of
information to guide robotic actions.

  * •

Robot Operating System (ROS) with Ontologies): Uses ontological frameworks to
guide robotic actions based on predefined knowledge bases.

####  V-A3 LLM + KG Systems

For the LLM + KG comparison, we selected two systems that integrate both LLMs
and KGs:

  * •

Code as Policies (CaP): Uses LLMs to generate executable code for robotic
policies, integrating with a KG for contextual knowledge.

  * •

Dynamic Knowledge Fusion (DKF): Utilizes a Two-Tower Architecture to process
text input and KG data for enhanced decision-making.

####  V-A4 Experimental Procedure

For each baseline system and our proposed framework, we conducted experiments
involving a series of tasks such as fetching, navigation, and object handling.
The tasks were performed in dynamic environments with real-time data
collection and continuous monitoring to simulate realistic conditions.

##### Data Collection and Task Execution

User commands were translated into textual representations and stored as JSON
objects. These were processed by the respective systems (GPT-3, BERT,
ProgPrompt, RoboBrain, ROS + Ontologies, CaP, DKF, and our proposed system) to
generate task plans. The plans were then executed by the robots, and
performance metrics were recorded.

##### Validation and Safety Checks

For our system, task plans generated by ChatGPT-4 were validated against the
Embodied Knowledge Graph (EKG) to ensure feasibility and safety before
execution. Safety protocols and environmental constraints were continuously
monitored to prevent any violations.

##### Performance Metrics

Each set of experiments was repeated a minimum of twenty times to ensure
statistical validity. The collected data was analyzed to calculate the
performance metrics:

  * •

Success Rate (SR): Calculated as the ratio of successfully completed tasks to
the total number of tasks.

  * •

Execution Time: Measured from the start of task execution to its completion.

  * •

Error Rate (ER): Determined by the number of execution errors relative to the
total tasks.

  * •

Safety Violations (SV): Counted by monitoring breaches of predefined safety
protocols.

  * •

Adaptability: Assessed based on the system’s ability to handle random
distractors and changes in the environment.

###  V-B Results Analysis

The results of the experiments were tabulated and compared across all baseline
systems and our proposed framework. Our system’s integration of ERCPs, LLMs,
and EKGs was found to significantly enhance task execution accuracy, reduce
errors, ensure safety compliance, and improve adaptability to dynamic
environments.

TABLE IV: Comparison of LLM-Only Systems vs. Ours Metric |  GPT-3 |  BERT |  ProgPrompt |  Ours
---|---|---|---|---
Success Rate (SR) |  71% |  62% |  71% |  90%
Execution Time |  25s |  35s |  32s |  35s
Error Rate (ER) |  15% |  18% |  12% |  3%
Safety Violations (SV) |  12 |  13 |  11 |  4
Adaptability |  Low |  Low |  Moderate |  High
TABLE V: Comparison of KG-Only Systems vs. Ours Metric |  RoboBrain |  ROS + Ontologies |  Ours
---|---|---|---
Success Rate (SR) |  75% |  76% |  90%
Execution Time |  25s |  25s |  35s
Error Rate (ER) |  15% |  13% |  3%
Safety Violations (SV) |  8 |  9 |  4
Adaptability |  Low |  Low |  High
TABLE VI: Comparison of LLM + KG Systems vs. Ours Metric |  CaP |  DKF |  Ours
---|---|---|---
Success Rate (SR) |  80% |  78% |  90%
Execution Time |  25s |  23s |  35s
Error Rate (ER) |  8% |  7% |  3%
Safety Violations (SV) |  10 |  14 |  4
Adaptability |  Moderate |  Moderate |  High

##  VI Limitations and Future Directions

Our framework, which integrates Large Language Models (LLMs) and Embodied
Knowledge Graphs (EKGs), holds promise for enhancing the safety of service
robots. Despite its potential, there are inherent limitations that need to be
addressed and opportunities for future developments:

###  VI-A Limitations

  * •

Dependency on Knowledge Graph Accuracy: The success of our model relies
heavily on the accuracy and completeness of the EKG. Inaccuracies or gaps in
the EKG data can lead to incorrect constraints and inappropriate guidance,
potentially compromising operational safety.

  * •

Generalization Challenges: Broadening the scope of the EKG to cover more
diverse situations is crucial for its effective application in varied
operational contexts. The current implementation may face difficulties when
applied to unfamiliar environments or tasks that were not anticipated during
the development phase.

  * •

Scalability Issues: Expanding the EKG to manage large and dynamic sets of
knowledge from multiple sources presents significant challenges. Ensuring the
accuracy and reliability of the knowledge graph as it scales is a critical
task that requires innovative solutions. This includes efficient data
integration, real-time updates, and handling heterogeneous data sources
without compromising performance.

  * •

Real-Time Performance: The integration of LLMs and EKGs must ensure that the
computational overhead does not impede real-time performance. Service robots
require prompt responses, and any latency in processing or decision-making
could impact their effectiveness and safety.

###  VI-B Future Directions

Looking forward, we aim to advance our framework by:

  * •

Enhancing Knowledge Graph Coverage: Future work should focus on expanding the
breadth and depth of the EKG. This involves integrating more comprehensive
datasets, covering a wider range of scenarios, and continuously updating the
knowledge graph with real-time data.

  * •

Improving Generalization Capabilities: Developing methods to enhance the
generalization capabilities of the EKG is essential. This includes employing
advanced machine learning techniques to enable the EKG to adapt to new,
unforeseen scenarios effectively.

  * •

Optimizing Scalability: Research should focus on optimizing the scalability of
the EKG. Techniques such as distributed computing, incremental updates, and
efficient data structures can be explored to manage large-scale knowledge
graphs effectively.

  * •

Real-Time Integration and Processing: Ensuring real-time integration and
processing of the LLM and EKG is paramount. Future work can explore the use of
faster algorithms, parallel processing, and edge computing to reduce latency
and improve response times.

  * •

Addressing Ethical and Privacy Issues: Future research should also focus on
developing frameworks to address ethical and privacy issues associated with
the use of extensive knowledge graphs. This includes ensuring data
anonymization, secure data handling practices, and compliance with privacy
regulations.

  * •

User-Centric Design Improvements: Incorporating user feedback into the design
of ERCPs and EKGs can help tailor the system to better meet user needs and
improve overall usability.

  * •

Interdisciplinary Collaboration: Collaborating with experts from fields such
as ethics, law, and social sciences can provide valuable insights into
addressing the broader implications of deploying advanced AI systems in public
and private spaces.

##  VII Conclusion

This research paper presents a novel framework for enhancing the safety
control of service robots, aiming to mitigate operational hazards. We proposed
the integration of Embodied Robotic Control Prompts (ERCPs), Large Language
Models (LLMs), and Embodied Knowledge Graphs (EKGs) to ensure that human-robot
interactions prioritize safety.

Our experimental results demonstrated a task execution rate of 95%,
underscoring the framework’s potential to revolutionize industries reliant on
service robots. This approach holds significant promise for applications in
healthcare, manufacturing, and beyond, enabling robots to comprehend natural
language and execute tasks with high accuracy. By effectively bridging
technology and human needs, our framework facilitates safer and more efficient
interactions between humans and robots, ultimately advancing the integration
of robotic systems into everyday life.

The successful implementation of this framework indicates a promising future
where service robots can operate autonomously while adhering to stringent
safety standards. This advancement paves the way for further innovations,
driving forward the capabilities of service robots in diverse sectors and
improving overall human-robot collaboration.

##  Appendix A A.1 API for $LLM$

In the course of our research, we employed our framework to execute the task
of acquiring the bread. Additionally, we devised a set of strategically
designed action primitives, systematically organized within our action
primitives (APIs).

To illustrate, we introduced the ’find_func’ for robotic perception and target
localization:

    "find_func": {
        "find": "find (arg1)"
    }

This function serves the purpose of enabling robots to identify and locate
objects within their environment, where ’arg1’ represents the target parameter
for replacement by large language models $(LLMs)$.

In conjunction with these foundational actions, we also established a suite of
basic motion route functions, including ’run,’ ’walk,’ and ’turn’:

    "Basic action functions of motion routes": {
        "run": "run_to (arg1)",
        "walk": "walk_to (arg1)",
        "turn": "turn_to (arg1)"
    }

Moreover, our research incorporated an extensive array of arm-related
functions such as ’open,’ ’close,’ ’drop,’ ’grab,’ ’plug_in,’ ’plug_out,’
’pull,’ ’push,’ ’lift,’ ’stretch,’ ’hold,’ ’put_on,’ ’put_in,’ and ’put_back’:

    "arm_func": {
        "open": "open (arg1)",
        "close": "close (arg1)",
        "drop": "drop (arg1)",
        "grab": "grab (arg1)",
        "plug_in": "plug_in (arg1)",
        "plug_out": "plug_out (arg1)",
        "pull": "pull (arg1)",
        "push": "push (arg1)",
        "lift": "lift (arg1)",
        "stretch": "stretch (arg1)",
        "hold": "hold (arg1)",
        "put_on": "put_on (arg1, arg2)",
        "put_in": "put_in (arg1, arg2)",
        "put_back": "put_back (arg1)"
    }

Additionally, we introduced specific actions like ’cut,’ ’pour_into,’ ’touch,’
’wash,’ ’wipe,’ ’sweep,’ and ’press’ for specialized tasks:

    "specific_actions": {
        "cut": "cut (arg1)",
        "pour_into": "pour_into (arg1, arg2)",
        "touch": "touch (arg1)",
        "wash": "wash (arg1)",
        "wipe": "wipe (arg1)",
        "sweep": "sweep (arg1)",
        "press": "press (arg1)"
    }

For instance, when our robotic system collected environmental data and stored
it in $JSON$ format within the ’env’ structure, it focused on locating a
specific target, namely ’bread,’ positioned on a table. The robot’s three-
dimensional coordinate data was pre-collected from the environment, enabling
the $EKG$ to seamlessly replace the parameter. Here, ’XYZ’ represents the
three-dimensional coordinate representation based on ’odom,’ and ’orientation’
denotes the spatial direction when the robot reaches the target position:

    goal.target_pose.pose.position.x = 1.74874
    goal.target_pose.pose.position.y = 0.0673331
    goal.target_pose.pose.position.z = 0.0

    goal.target_pose.pose.orientation.x = 0.0
    goal.target_pose.pose.orientation.y = 0.0
    goal.target_pose.pose.orientation.z = 0.97906
    goal.target_pose.pose.orientation.w = -0.21842

We meticulously maintained a record of the environmental information, ensuring
that key details such as the table’s position and orientation were readily
accessible:

    bread: on the table
    # Store known environment information
    table: {
        position: {
            x: -3.84107,
            y: -1.92424,
            z: 0.0
        },
        orientation: {
            x: 0.0,
            y: 0.0,
            z: 0.63668,
            w: 0.7711
        }
    }

##  Appendix B Embodied Knowledge Graph (EKG)

In this appendix, we provide a detailed overview of the creation of the
Embodied Knowledge Graph (EKG) for enhancing the reasoning ability of a
service robot in the context of safe task executions.

The Embodied Knowledge Graph (EKG) is structured as follows:

  * •

We initialize the EKG with the creation of the EKG node, representing the
entire knowledge graph.

  * •

Two tasks, denoted as ”Task 1” and ”Task 2,” are defined within the EKG.

  * •

We establish relationships between the EKG and these tasks, indicating that
the EKG encompasses these tasks.

⬇

CREATE (ekg:KnowledgeGraph {name: "Embodied Knowledge Graph (EKG)"})

CREATE (task1:Task {name: "Task 1"})

CREATE (task2:Task {name: "Task 2"})

CREATE (ekg)-[:HAS_TASK]->(task1)

CREATE (ekg)-[:HAS_TASK]->(task2)

###  B-A Pose Representation

  * •

Three poses are defined within the EKG: ”Target Pose 1,” ”Target Pose 2,” and
”Current Pose.”

  * •

Task 1 is associated with ”Target Pose 1,” and Task 2 is associated with
”Target Pose 2.”

  * •

The ”Current Pose” is stored within the EKG and is retrievable for both Task 1
and Task 2.

###  B-B Gripper Pose

The following Cypher code snippet demonstrates the creation and relationships
within the Gripper EKG related to ”Gripper Object Pose Analysis”:

⬇

CREATE (gripperPoseAnalysis:Functionality {name: "Gripper Object Pose
Analysis"})

CREATE (gripperRelativePosition:Pose {name: "Gripper Relative Position Vector
P_relative"})

CREATE (gripperRelativeOrientation:Pose {name: "Gripper Relative Orientation
Quaternion R_relative"})

CREATE (gripperEKG)-[:HAS_FUNCTIONALITY]

->(gripperPoseAnalysis)

CREATE (gripperPoseAnalysis)-[:GENERATES]

->(gripperRelativePosition)

CREATE (gripperPoseAnalysis)-[:GENERATES]

->(gripperRelativeOrientation)

###  B-C Gripper Grasping Strategy Formulation

The following describes the establishment of the ”Gripper Grasping Strategy
Formulation” functionality within the Gripper EKG:

  * •

The functionality of ”Gripper Grasping Strategy Formulation” is established
within the Gripper EKG.

⬇

CREATE (gripperGraspingStrategy:Functionality {name: "Gripper Grasping
Strategy Formulation"})

CREATE (gripperEKG)-[:HAS_FUNCTIONALITY]

->(gripperGraspingStrategy)

###  B-D Gripper Gripping Parameters

The following points describe the introduction of ”Gripper Gripping
Parameters” as a functionality within the Gripper EKG:

  * •

”Gripper Gripping Parameters” is introduced as a functionality within the
Gripper EKG.

  * •

This functionality includes parameters such as ”Gripper Desired Gripping Force
F_desired.”

⬇

CREATE (gripperGrippingParameters:Functionality {name: "Gripper Gripping
Parameters"})

CREATE (gripperDesiredForce:Parameter {name: "Gripper Desired Gripping Force
F_desired"})

CREATE (gripperEKG)-[:HAS_FUNCTIONALITY]

->(gripperGrippingParameters)

CREATE (gripperGrippingParameters)-[:STORES]

->(gripperDesiredForce)

###  B-E Gripper Force Control Information

The following points describe the definition of ”Gripper Force Control
Information” as a functionality within the Gripper EKG:

  * •

”Gripper Force Control Information” is defined as a functionality within the
Gripper EKG.

  * •

This functionality stores information about gripper control algorithms (e.g.,
PID) and gripping force control parameters.

⬇

CREATE (gripperForceControl:Functionality {name: "Gripper Force Control
Information"})

CREATE (gripperControlAlgorithms:Parameter {name: "Gripper Control Algorithms
(e.g., PID)

## Acknowledgements

We extend our heartfelt gratitude to Shaanxi University of Science and
Technology, particularly the School of Electronic Information and Artificial
Intelligence, for their support and resources throughout this research.

We are indebted to Liu Haozhe (KAUST) for his invaluable guidance, mentorship,
and unwavering support. Liu Haozhe’s expertise and insights have played a
pivotal role in shaping this research.

This work was made possible by the collaborative efforts and contributions of
our research team and the generous assistance of our colleagues. We appreciate
the research community’s valuable feedback and support, which have greatly
enriched our study.

We also express our gratitude to our families and loved ones for their
unwavering encouragement and understanding during the research process.

## References

  * [1] Juan Angel Gonzalez-Aguirre, Ricardo Osorio-Oliveros, Karen L Rodríguez-Hernández, Javier Lizárraga-Iturralde, Ruben Morales Menendez, Ricardo A Ramírez-Mendoza, Mauricio Adolfo Ramírez-Moreno, and Jorge de Jesus Lozoya-Santos, _Service robots: Trends and technology_ , Applied Sciences, vol. 11, no. 22, pp. 10702, 2021, MDPI.
  * [2] Stefanie Paluch, Jochen Wirtz, and Werner H Kunz, _Service robots and the future of services_ , Marketing Weiterdenken: Zukunftspfade für eine marktorientierte Unternehmensführung, pp. 423–435, 2020, Springer.
  * [3] Jane Holland, Liz Kingston, Conor McCarthy, Eddie Armstrong, Peter O’Dwyer, Fionn Merz, and Mark McConnell, _Service robots in the healthcare sector_ , Robotics, vol. 10, no. 1, pp. 47, 2021, MDPI.
  * [4] Jochen Wirtz, Werner Kunz, and Stefanie Paluch, _The service revolution, intelligent automation and service robots_ , European Business Review, vol. 29, no. 5, pp. 909, 2021.
  * [5] Hailian Qiu, Minglong Li, Boyang Shu, and Billy Bai, _Enhancing hospitality experience with service robots: The mediating role of rapport building_ , Journal of Hospitality Marketing & Management, vol. 29, no. 3, pp. 247–268, 2020, Taylor & Francis.
  * [6] Marco Ceccarelli, _Problems and issues for service robots in new applications_ , International Journal of Social Robotics, vol. 3, pp. 299–312, 2011, Springer.
  * [7] Tobias Körtner, _Ethical challenges in the use of social service robots for elderly people_ , Zeitschrift für Gerontologie und Geriatrie, vol. 49, no. 4, pp. 303–307, 2016, Springer.
  * [8] Anthony J Soroka, Renxi Qiu, Alexandre Noyvirt, and Ze Ji, _Challenges for service robots operating in non-industrial environments_ , in _IEEE 10th International Conference on Industrial Informatics_ , pp. 1152–1157, 2012, IEEE.
  * [9] Carme Torras, _Service robots for citizens of the future_ , European Review, vol. 24, no. 1, pp. 17–30, 2016, Cambridge University Press.
  * [10] Hironori Matsuzaki and Gesa Lindemann, _The autonomy-safety-paradox of service robotics in Europe and Japan: a comparative analysis_ , AI & society, vol. 31, pp. 501–517, 2016, Springer.
  * [11] Zichao Hu, Francesca Lucchetti, Claire Schlesinger, Yash Saxena, Anders Freeman, Sadanand Modak, Arjun Guha, and Joydeep Biswas, _Deploying and Evaluating LLMs to Program Service Mobile Robots_ , IEEE Robotics and Automation Letters, 2024, IEEE.
  * [12] W. Huang, C. Wang, R. Zhang, Y. Li, J. Wu, and F.F. Li, _VoxPoser: composable 3D value maps for robotic manipulation with language models_ , arXiv preprint arXiv:2307.05973, 2023.
  * [13] Ceng Zhang and others, _Large language models for human-robot interaction: A review_ , Biomimetic Intelligence and Robotics, 2023, pp. 100131.
  * [14] Shengchen Zhang and others, _Patterns for representing knowledge graphs to communicate situational knowledge of service robots_ , in Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, 2021.
  * [15] Gi Hyun Lim, Il Hong Suh, and Hyowon Suh, _Ontology-based unified robot knowledge for service robots in indoor environments_ , IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans, vol. 41, no. 3, pp. 492–509, 2010.
  * [16] Marc Hanheide and others, _Robot task planning and explanation in open and uncertain worlds_ , Artificial Intelligence, vol. 247, pp. 119–150, 2017.
  * [17] Xiyang Wu and others, _On the Safety Concerns of Deploying LLMs/VLMs in Robotics: Highlighting the Risks and Vulnerabilities_ , arXiv preprint arXiv:2402.10340, 2024.
  * [18] Ashutosh Saxena and others, _Robobrain: Large-scale knowledge engine for robots_ , arXiv preprint arXiv:1412.0691, 2014.
  * [19] Freddy Lecue, _On the role of knowledge graphs in explainable AI_ , Semantic Web, vol. 11, no. 1, pp. 41–51, 2020.
  * [20] Carson Stark and others, _Dobby: A Conversational Service Robot Driven by GPT-4_ , arXiv preprint arXiv:2310.06303, 2023.
  * [21] Peifeng Liu and others, _Joint Knowledge Graph and Large Language Model for Fault Diagnosis and Its Application in Aviation Assembly_ , IEEE Transactions on Industrial Informatics, 2024.
  * [22] Ggaliwango Marvin and others, _Prompt Engineering in Large Language Models_ , in International Conference on Data Intelligence and Cognitive Informatics, Springer Nature Singapore, Singapore, 2023.
  * [23] Jules White and others, _A prompt pattern catalog to enhance prompt engineering with chatgpt_ , arXiv preprint arXiv:2302.11382, 2023.
  * [24] Ellen Jiang and others, _Promptmaker: Prompt-based prototyping with large language models_ , in CHI Conference on Human Factors in Computing Systems Extended Abstracts, 2022.
  * [25] Mimo Shirasaka and others, _Self-Recovery Prompting: Promptable General Purpose Service Robot System with Foundation Models and Self-Recovery_ , arXiv preprint arXiv:2309.14425, 2023.
  * [26] Pengfei Liu and others, _Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing_ , ACM Computing Surveys, vol. 55, no. 9, pp. 1–35, 2023.
  * [27] L. Hu, Z. Liu, Z. Zhao, L. Hou, L. Nie, and J. Li, _A survey of knowledge enhanced pre-trained language models_ , IEEE Transactions on Knowledge and Data Engineering, pp. 1–19, 2023, doi:10.1109/TKDE.2023.3310002.
  * [28] K. Zhou, J. Yang, C.C. Loy, and Z. Liu, _Conditional prompt learning for vision-language models_ , in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 16795–16804, IEEE, New Orleans, LA, 2022.
  * [29] T. Kojima, S.S. Gu, M. Reid, Y. Matsuo, and Y. Iwasawa, _Large language models are zero-shot reasoners_ , in Advances in Neural Information Processing Systems (NeurIPS), pp. 22199–22213, New Orleans, Louisiana, 2022.
  * [30] W. Huang, F. Xia, T. Xiao, H. Chan, J. Liang, P. Florence, A. Zeng, J. Tompson, I. Mordatch, Y. Chebotar, and others, _Inner monologue: embodied reasoning through planning with language models_ , arXiv preprint arXiv:2207.05608, 2022.
  * [31] J. Liang, W. Huang, F. Xia, P. Xu, K. Hausman, B. Ichter, P. Florence, and A. Zeng, _Code as policies: language model programs for embodied control_ , in IEEE International Conference on Robotics and Automation (ICRA), pp. 9493–9500, IEEE, London, United Kingdom, 2023.
  * [32] J. Wu, R. Antonova, A. Kan, M. Lepert, A. Zeng, S. Song, J. Bohg, S. Rusinkiewicz, and T. Funkhouser, _Tidybot: personalized robot assistance with large language models_ , arXiv preprint arXiv:2305.05658, 2023.
  * [33] Yan Ding and others, _Task and motion planning with large language models for object rearrangement_ , arXiv preprint arXiv:2303.06247, 2023.
  * [34] Zichao Hu and others, _Deploying and Evaluating LLMs to Program Service Mobile Robots_ , arXiv preprint arXiv:2311.11183, 2023.
  * [35] Jianing Yang and others, _LLM-grounder: Open-vocabulary 3d visual grounding with large language model as an agent_ , arXiv preprint arXiv:2309.12311, 2023.
  * [36] I. Singh and others, _ProgPrompt: Generating Situated Robot Task Plans using Large Language Models_ , in 2023 IEEE International Conference on Robotics and Automation (ICRA), pp. 11523–11530, IEEE, London, United Kingdom, 2023, doi:10.1109/ICRA48891.2023.10161317.
  * [37] J. Liang and others, _Code as Policies: Language Model Programs for Embodied Control_ , in 2023 IEEE International Conference on Robotics and Automation (ICRA), pp. 9493–9500, IEEE, London, United Kingdom, 2023, doi:10.1109/ICRA48891.2023.10160591.
  * [38] M. Jia, L. Tang, B.C. Chen, C. Cardie, S. Belongie, B. Hariharan, and S.N. Lim, _Visual prompt tuning_ , in Computer Vision – ECCV 2022, pp. 709–727, Springer Nature Switzerland, Cham, Switzerland, 2022.
  * [39] X. Chen, L. Li, S. Deng, C. Tan, C. Xu, F. Huang, L. Si, H. Chen, and N. Zhang, _LightNER: a lightweight tuning paradigm for low-resource NER via pluggable prompting_ , arXiv preprint arXiv:2109.00720, 2021.
  * [40] N. Ding, S. Hu, W. Zhao, Y. Chen, Z. Liu, H.T. Zheng, and M. Sun, _Openprompt: an open-source framework for prompt-learning_ , arXiv preprint arXiv:2111.01998, 2021.
  * [41] Sai Vemprala and others, _Chatgpt for robotics: Design principles and model abilities_ , Microsoft Auton. Syst. Robot. Res, vol. 2, pp. 20, 2023.
  * [42] Runqing Miao and others, _Semantic Representation of Robot Manipulation with Knowledge Graph_ , Entropy, vol. 25, no. 4, pp. 657, 2023.
  * [43] Jiaru Bai and others, _A dynamic knowledge graph approach to distributed self-driving laboratories_ , Nature Communications, vol. 15, no. 1, pp. 462, 2024.
  * [44] Yiwen Ding and others, _Robotic task oriented knowledge graph for human-robot collaboration in disassembly_ , Procedia CIRP, vol. 83, pp. 105–110, 2019.
  * [45] Ciyuan Peng and others, _Knowledge graphs: Opportunities and challenges_ , Artificial Intelligence Review, pp. 1–32, 2023.
  * [46] Linyao Yang and others, _ChatGPT is not Enough: Enhancing Large Language Models with Knowledge Graphs for Fact-aware Language Modeling_ , arXiv preprint arXiv:2306.11489, 2023.
  * [47] Mirza Mohtashim Alam, Md Rashad Al Hasan Rony, Mojtaba Nayyeri, Karishma Mohiuddin, Mst. Mahfuja Akter, Sahar Vahdati, and Jens Lehmann, _Language Model Guided Knowledge Graph Embeddings_ , IEEE Access, vol. 10, pp. 1–1, 2022, doi:10.1109/ACCESS.2022.3191666.
  * [48] Shirui Pan and others, _Unifying large language models and knowledge graphs: A roadmap_ , IEEE Transactions on Knowledge and Data Engineering, 2024.
  * [49] T. McCoy, E. Pavlick, and T. Linzen, _Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference_ , in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 3428–3448, Florence, Italy, 2019, https://aclanthology.org/P19-1334.
  * [50] X. Wang, P. Kapanipathi, R. Musa, M. Yu, K. Talamadupula, I. Abdelaziz, M. Chang, A. Fokoue, B. Makni, N. Mattei, and M. Witbrock, _Improving natural language inference using external knowledge in the science questions domain_ , in The Thirty-Third AAAI Conference on Artificial Intelligence, pp. 7208–7215, AAAI Press, Honolulu, Hawaii, USA, 2019.
  * [51] B.Y. Lin, X. Chen, J. Chen, and X. Ren, _KagNet: Knowledge-aware graph networks for commonsense reasoning_ , in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 2829–2839, Hong Kong, China, 2019.
  * [52] Y. Feng, X. Chen, B.Y. Lin, P. Wang, J. Yan, and X. Ren, _Scalable multi-hop relational reasoning for knowledge-aware question answering_ , in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020.
  * [53] X. Zhang, A. Bosselut, M. Yasunaga, H. Ren, P. Liang, C.D. Manning, and J. Leskovec, _Greaselm: Graph reasoning enhanced language models_ , in International conference on learning representations, 2022.
  * [54] M. Yasunaga, H. Ren, A. Bosselut, P. Liang, and J. Leskovec, _QA-GNN: Reasoning with language models and knowledge graphs for question answering_ , in Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 535–546, 2021.
  * [55] Yueqing Sun and others, _JointLK: Joint reasoning with language models and knowledge graphs for commonsense question answering_ , arXiv preprint arXiv:2112.02732, 2021.
  * [56] Md Sadman Sakib and Yu Sun, _Consolidating Trees of Robotic Plans Generated Using Large Language Models to Improve Reliability_ , arXiv preprint arXiv:2401.07868, 2024.


#  $L^{p}$-improving properties of certain singular measures on the Heisenberg
group

Pablo Rocha  Departamento de Matemática, Universidad Nacional del Sur, Av.
Alem 1253 - Bahía Blanca 8000, Buenos Aires, Argentina.
<EMAIL_ADDRESS>
###### Abstract.

Let $\mu_{A}$ be the singular measure on the Heisenberg group $\mathbb{H}^{n}$
supported on the graph of the quadratic function $\varphi(y)=y^{t}Ay$, where
$A$ is a $2n\times 2n$ real symmetric matrix. If $\det(2A\pm J)\neq 0$, we
prove that the operator of convolution by $\mu_{A}$ on the right is bounded
from $L^{\frac{2n+2}{2n+1}}(\mathbb{H}^{n})$ to $L^{2n+2}(\mathbb{H}^{n})$. We
also study the type set of the measures
$d\nu_{\gamma}(y,s)=\eta(y)|y|^{-\gamma}d\mu_{A}(y,s)$, for $0\leq\gamma<2n$,
where $\eta$ is a cut-off function around the origin on $\mathbb{R}^{2n}$.
Moreover, for $\gamma=0$ we characterize the type set of $\nu_{0}$.

Key words and phrases: Heisenberg group; singular Borel measure;
$L^{p}$-improving property.

2.020 Math. Subject Classification: 43A80, 42A38.

##  1\. Introduction

Let $I_{n}$ be the $n\times n$ identity matrix and $J$ be the $2n\times 2n$
skew-symmetric matrix given by

(1) $J=\left(\begin{array}[]{cc}0&I_{n}\\\ -I_{n}&0\\\ \end{array}\right).$

The Heisenberg group is $\mathbb{H}^{n}=\mathbb{R}^{2n}\times\mathbb{R}$
endowed with the group law (non-commutative)

$\left(x,t\right)\cdot\left(y,s\right)=\left(x+y,\,t+s+\left\langle
x,y\right\rangle\right),$

where $\langle x,y\rangle$ is the standard symplectic form on
$\mathbb{R}^{2n}$, i.e. $\langle x,y\rangle=x^{t}Jy$, with neutral element
$(0,0)$, and with inverse $\left(x,t\right)^{-1}=\left(-x,-t\right)$. The
topology in $\mathbb{H}^{n}$ is the induced by $\mathbb{R}^{2n+1}$, so the
borelian sets of $\mathbb{H}^{n}$ are identified with those of
$\mathbb{R}^{2n+1}$. The Haar measure in $\mathbb{H}^{n}$ is the Lebesgue
measure of $\mathbb{R}^{2n+1}$, thus $L^{p}(\mathbb{H}^{n})\equiv
L^{p}(\mathbb{R}^{2n+1})$. Given a borelian function
$f:\mathbb{H}^{n}\to\mathbb{C}$ and a Borel measure $\mu$ on $\mathbb{H}^{n}$,
define the convolution by $\mu$ on the right by

(2) $(f\ast\mu)(x,t)=\int_{\mathbb{H}^{n}}f((x,t)\cdot(y,s)^{-1})\,d\mu(y,s),$

provided the integral exists.

A Borel measure $\mu$ on the Heisenberg group $\mathbb{H}^{n}$ is said to be
$L^{p}$-improving if the operator $T_{\mu}:f\mapsto f\ast\mu$ is bounded from
$L^{p}(\mathbb{H}^{n})$ into $L^{q}(\mathbb{H}^{n})$ for some $1\leq
p<q<\infty$.

A remarkable fact is that singular measures can be $L^{p}$-improving. If in
(2) we replace the Heisenberg group $\mathbb{H}^{n}$ by $\mathbb{R}^{n}$ with
the ordinary convolution in $\mathbb{R}^{n}$ and considering there
$\mu=\eta\sigma_{M}$, where $\sigma_{M}$ is the surface measure on a given
manifold $M$ (in $\mathbb{R}^{n}$) and $\eta$ is a smooth cut-off function,
then the $L^{p}$-improving properties of a measure of this type are closely
related to the existence of a certain amount of curvature of the manifold $M$
(see [5], [6], [7]). A similar result holds on general Lie groups (see Theorem
1.1, pp. 362 in [9]).

A more delicate problem consists in determining the exact range of pairs
$(p,q)$ for which $L^{p}\ast\mu\subseteq L^{q}$ embeds continuously. Given a
manifold $M$ (in $\mathbb{H}^{n}$), define the type set $E_{\eta\sigma_{M}}$
by

$E_{\eta\sigma_{M}}=\left\\{\left(\frac{1}{p},\frac{1}{q}\right)\in[0,1]\times[0,1]:\|T_{\eta\sigma_{M}}\|_{p,q}<\infty\right\\}.$

A very interesting survey of results concerning the type sets for convolution
operators with singular measures in $\mathbb{R}^{n}$ can be found in [8].

In the $\mathbb{H}^{n}$ setting, S. Secco in [10] and [11] obtained
$L^{p}$-improving properties of measures supported on curves in
$\mathbb{H}^{1}$, under certain assumptions. In [9], F. Ricci and E. Stein
showed that the type set of the measure given by (3), for the case
$\varphi\equiv 0$, $\gamma=0$ and $n=1$, is the triangle with vertices
$(0,0),$ $(1,1),$ and $\left(\frac{3}{4},\frac{1}{4}\right)$. In [3] and [4],
the author jointly with T. Godoy generalized the work of Ricci and Stein for
the case $\varphi(w)=w^{t}Aw=\sum_{j=1}^{n}\alpha_{j}|w_{j}|^{2}$, where $A$
is a $2n\times 2n$ real diagonal matrix such that $a_{ii}=a_{(i+1)(i+1)}$ for
$i=2j-1$ with $j=1,2,...,n$, $\alpha_{j}=a_{(2j-1)(2j-1)}$,
$w_{j}\in\mathbb{R}^{2}$, $0\leq\gamma<2n$, and $n\in\mathbb{N}$. There we
also gave some examples of surfaces with degenerate curvature at the origin.

Let $\varphi:\mathbb{R}^{2n}\to\mathbb{R}$ be the function defined by
$\varphi(y)=y^{t}Ay$, where $A$ is a $2n\times 2n$ real symmetric matrix. It
is well known that if $A$ is an arbitrary matrix, then there exists a
symmetric matrix $\widetilde{A}$ such that $y^{t}Ay=y^{t}\widetilde{A}y$, for
all $y$. We consider two borelian measures on $\mathbb{H}^{n}$ supported on
the graph of $\varphi$ , $\mu_{A}$ and $\nu_{\gamma}$, $0\leq\gamma<2n$, given
by

$\mu_{A}(E)=\int_{\mathbb{R}^{2n}}\,\chi_{E}(y,\varphi(y))\,dy,$

and

(3)
$\nu_{\gamma}(E)=\int_{\mathbb{R}^{2n}}\,\chi_{E}(y,\varphi(y))\,\eta(y)\,|y|^{-\gamma}\,dy,$

where $\eta:\mathbb{R}^{2n}\to[0,1]$ is a smooth cut-off function such that
$\eta(y)=1$ if $|y|\leq 1$, $\eta(y)=0$ if $|y|\geq 2$, and $E$ is a borelian
set of $\mathbb{H}^{n}$. Let $T_{\mu_{A}}f=f\ast\mu_{A}$ and
$T_{\nu_{\gamma}}f=f\ast\nu_{\gamma}$ be the operators of convolution by
$\mu_{A}$ and $\nu_{\gamma}$ on the right respectively.

We are interested in studying the $L^{p}$-improving properties of the operator
$T_{\mu_{A}}$ and in the characterization of the type set $E_{\nu_{\gamma}}$.
We point out that our measure $\mu_{A}$ is not the surface measure on the
graph $\mathop{\rm gr}(\varphi)$ of $\varphi$, however the measures
$\eta\mu_{A}$ and $\eta\sigma_{\mathop{\rm gr}(\varphi)}$ are equivalent, see
Proposition 2 below, so $E_{\eta\mu_{A}}=E_{\eta\sigma_{\mathop{\rm
gr}(\varphi)}}$.

The following restrictions for the type sets $E_{\nu_{\gamma}}$,
$0\leq\gamma<2n$, were proved in [3] and [4] for the case
$\varphi(w_{1},...,w_{n})=\sum_{j=1}^{n}\alpha_{j}|w_{j}|^{2}$ with
$w_{j}\in\mathbb{R}^{2}$. It is easy to see that such an argument works as
well for our function $\varphi(y)=y^{t}Ay$.

Thus if $\left(\frac{1}{p},\frac{1}{q}\right)\in E_{\nu_{\gamma}}$,
$0\leq\gamma<2n$, then

(4) $p\leq
q,\,\,\,\,\,\,\,\,\,\,\,\,\frac{1}{q}\geq\frac{2n+1}{p}-2n,\,\,\,\,\,\,\,\,\,\,\,\,\frac{1}{q}\geq\frac{1}{(2n+1)p}.$

Another necessary condition for the pair $(\frac{1}{p},\frac{1}{q})$ to be in
$E_{\nu_{\gamma}}$ is the following:

(5) $\frac{1}{q}\geq\frac{1}{p}-\frac{2n-\gamma}{2n+2}.$

This last condition is relevant only for the case $0<\gamma<2n$.

Let $D$ be the point of intersection, in the $(\frac{1}{p},\frac{1}{q})$
plane, of the lines $\frac{1}{q}=\frac{2n+1}{p}-2n$,
$\frac{1}{q}=\frac{1}{p}-\frac{2n-\gamma}{2n+2}$, and let $D^{\prime}$ be its
symmetric image with respect to the symmetry axis $\frac{1}{q}=1-\frac{1}{p}$.
So

$D=\left(\frac{4n^{2}+2n+\gamma}{2n(2n+2)},\frac{2n+(2n+1)\gamma}{2n(2n+2)}\right)=\left(\frac{1}{p_{D}},\frac{1}{q_{D}}\right)\text{
and\ }D^{\prime}=\left(1-\frac{1}{q_{D}},1-\frac{1}{p_{D}}\right).$

Since $0\leq\gamma<2n$ it is clear that $\|T_{\nu_{\gamma}}f\|_{p}\leq
c\|f\|_{p}$ for all Borel function $f\in L^{p}(\mathbb{H}^{n})$ and all $1\leq
p\leq\infty$, so $(\frac{1}{p},\frac{1}{p})\in E_{\mu_{\gamma}}$.

Thus for $0<\gamma<2n$ the set $E_{\nu_{\gamma}}$ is contained in the closed
trapezoid with vertices $(0,0)$, $(1,1)$, $D$ and $D^{\prime}$, and the set
$E_{\nu_{0}}$ is contained in the closed triangle with vertices $(0,0)$,
$(1,1)$, and $(\frac{2n+1}{2n+2},\frac{1}{2n+2})$.

In the Section 3, our main result appears. There we prove that the operator
$T_{\mu_{A}}$ is bounded from $L^{\frac{2n+2}{2n+1}}(\mathbb{H}^{n})$ to
$L^{2n+2}(\mathbb{H}^{n})$, see Theorem 3 below. This result allows us to
characterize the type set $E_{\nu_{0}}$ as well as the interior of
$E_{\nu_{\gamma}}$ for $0<\gamma<2n$. More precisely, we show that
$E_{\nu_{0}}$ is the closed triangle with vertices $(0,0)$, $(1,1)$, and
$(\frac{2n+1}{2n+2},\frac{1}{2n+2})$ and the interior of $E_{\nu_{\gamma}}$
coincides with the interior of the closed trapezoid with vertices $(0,0)$,
$(1,1)$, $D$ and $D^{\prime}$, see Theorem 4 and Theorem 6 below.

Throughout this paper, $c$ will denote a positive real constant not
necessarily the same at each occurrence. The symbol $A\lesssim B$ stands for
the inequality $A\leq cB$ for some constant $c$. We use the following
convention for the Fourier transform in $\mathbb{R}^{n}$
$\widehat{f}(\xi)=\int f(x)e^{-i\xi\cdot x}dx$. The Fourier transform
$\widehat{u}$ of a distribution $u$ on $\mathbb{R}^{n}$ is the distribution
defined by $(\widehat{u},\phi)=(u,\widehat{\phi}\,)$ for all rapidly
decreasing functions $\phi$ on $\mathbb{R}^{n}$.

##  2\. Preliminaries

In the sequel $J$ will denote the $2n\times 2n$ skew-symmetric matrix defined
in (1). It is easy to check that

a) $J^{2}=-I$.

b) $J^{t}=-J$.

c) $x^{t}Jx=0$, for all $x\in\mathbb{R}^{2n}$.

d) $x^{t}Jy=-y^{t}Jx$, for all $x,y\in\mathbb{R}^{2n}$.

######  Lemma 1.

Let $A$ be a $2n\times 2n$ real diagonal matrix. Then

$\det(A\pm
J)=(a_{11}\,a_{(n+1)(n+1)}+1)\cdot(a_{22}\,a_{(n+2)(n+2)}+1)\cdot\cdot\cdot(a_{nn}\,a_{(2n)(2n)}+1),$

where the $a_{ii}$’s are the diagonal entries of $A$.

###### Proof.

Since $\det(A+J)=\det((A+J)^{t})=\det(A-J)$, it is sufficient to prove the
statement of the lemma for $\det(A+J)$. Applying induction on $n$ the lemma
follows. ∎

######  Proposition 2.

Let $A$ be a $2n\times 2n$ real symmetric matrix. Then the graph of the
function $\varphi(y)=y^{t}Ay$ generates all the group $\mathbb{H}^{n}$.
Moreover, the measure $\nu_{0}=\eta\mu_{A}$ is equivalent to the measure
$\eta\sigma$, where $\eta$ is a cut-off function and $\sigma$ is the surface
measure on the graph of $\varphi$.

###### Proof.

The first statement will follow if we prove that $(x,0)$ and $(0,t)$ belong to
the set $G_{\mathop{\rm gr}(\varphi)}$ generated by the graph $\mathop{\rm
gr}(\varphi)$ of $\varphi$, since $(x,t)=(x,0)\cdot(0,t)$. It is clear that
$(x,\varphi(x))\in G_{\mathop{\rm gr}(\varphi)}$, so
$(-t^{1/2}x,\varphi(t^{1/2}x))=(-t^{1/2}x,\varphi(-t^{1/2}x))\in
G_{\mathop{\rm gr}(\varphi)}$ for all $x\in\mathbb{R}^{2n}$ and all $t>0$.
From that it follows that $(0,t\varphi(x))\in G_{\mathop{\rm gr}(\varphi)}$
for all $t>0$ and all $x$. If $A$ is a non-null matrix, then
$(0,-t)=(0,t)^{-1}\in G_{\mathop{\rm gr}(\varphi)}$ and
$(x,0)=(x,\varphi(x))\cdot(0,-\varphi(x))\in G_{\mathop{\rm gr}(\varphi)}$. If
$A$ is the null matrix, it is sufficient to prove that $(0,t)\in
G_{\mathop{\rm gr}(\varphi)}$ for all $t$. Indeed, for $x$ and $y$ such that
$\langle x,y\rangle\neq 0$ we have $(0,t)=(x,0)\cdot(\frac{ty}{\langle
x,y\rangle},0)\cdot(-x-\frac{ty}{\langle x,y\rangle},0)\in G_{\mathop{\rm
gr}(\varphi)}$. So $G_{\mathop{\rm gr}(\varphi)}=\mathbb{H}^{n}$.

For the second part of the proposition, we have that the surface measure on
the graph of $\varphi$ is given by

$\sigma(E)=\int_{\phi^{-1}(E)}\sqrt{\det[(\partial_{x_{i}}\phi,\partial_{x_{j}}\phi)_{x}]}\,dx,$

where $\phi(x)=(x,\varphi(x))$ and $E$ is a borelian set of
$\mathbb{R}^{2n+1}$ (see pp. 43-45 in [1]). A computation gives

$\det[(\partial_{x_{i}}\phi,\partial_{x_{j}}\phi)_{x}]=1+\sum_{j=1}^{2n}(\partial_{x_{j}}\varphi(x))^{2},\,\,for\,\,all\,\,x.$

So

$\int_{\mathbb{R}^{2n}}\chi_{E}(\phi(x))\eta(x)dx\leq\int_{\phi^{-1}(E)}\sqrt{\det[(\partial_{x_{i}}\phi,\partial_{x_{j}}\phi)_{x}]}\eta(x)\,dx\lesssim\int_{\mathbb{R}^{2n}}\chi_{E}(\phi(x))\eta(x)dx.$

Then $\nu_{0}$ is equivalent to $\eta\sigma$. ∎

The $\lambda$-twisted convolution is defined by

$(f\times_{\lambda}g)(x)=\int_{\mathbb{R}^{2n}}f(x-y)g(y)e^{-i\lambda
x^{t}Jy}dy.$

Given a $2n\times 2n$ real symmetric matrix $A$, we put

$e_{A}(x)=e^{ix^{t}Ax}.$

It is easy to check, using the properties ${\bf b)}$ and ${\bf c)}$ of the
matrix $J$, that

$(f\times_{\lambda}e_{\lambda A})(x)=e_{\lambda A}(x)(e_{\lambda
A}(\cdot)f(\cdot))\,\,\widehat{}\,\left(\lambda(2A+J)x\right),$

where $\widehat{f}(\xi)=\int_{\mathbb{R}^{2n}}f(x)e^{-ix\cdot\xi}dx$ is the
Fourier transform of $f$. Thus for each $f\in L^{1}(\mathbb{R}^{2n})\cap
L^{2}(\mathbb{R}^{2n})$ we have

(6) $\left\|f\times_{\lambda}e_{\lambda
A}\right\|_{L^{2}(\mathbb{R}^{2n})}=(2\pi)^{n}|\lambda|^{-n}|\det(2A\pm
J)|^{-1/2}\left\|f\right\|_{L^{2}(\mathbb{R}^{2n})},$

if $\det(2A\pm J)\neq 0$.

##  3\. Main result

To prove the $L^{\frac{2n+2}{2n+1}}(\mathbb{H}^{n})-L^{2n+2}(\mathbb{H}^{n})$
boundedness of the operator $T_{\mu_{A}}$ we embed our operator in an analytic
family $\\{T_{z}\\}$ of operators on the strip $-n\leq\Re(z)\leq 1$, and then
we apply the complex interpolation theorem.

######  Theorem 3.

If $\det(2A\pm J)\neq 0$, then the operator $T_{\mu_{A}}$ is bounded from
$L^{\frac{2n+2}{2n+1}}(\mathbb{H}^{n})$ to $L^{2n+2}(\mathbb{H}^{n})$.

###### Proof.

To prove the statement of the theorem we consider the family $\\{|s|^{z-1}\\}$
of functions initially defined when $\Re(z)>0$ and
$s\in\mathbb{R}\setminus\\{0\\}$. This family of functions can be extended, in
the $z$ variable, to an analytic family of distributions on
$\mathbb{C}\setminus\\{-2k:k\in\mathbb{N}\cup\\{0\\}\\}$. By abuse of
notation, we denote this extension by $|s|^{z-1}$. The family
$\\{|s|^{z-1}\\}$ have simple poles in $z=-2k$ for
$k\in\mathbb{N}\cup\\{0\\}$. Since the meromorphic continuation of the
function $\Gamma\left(\frac{z}{2}\right)$ (we keep the notation for his
continuation) has simple poles at the same points (i.e. $z=-2k$), the family
$\\{I_{z}\\}$ of distributions defined by

(7)
$I_{z}(s)=\frac{2^{-\frac{z}{2}}}{\Gamma\left(\frac{z}{2}\right)}|s|^{z-1}$

results in an entire family of distributions (see pp. 55-56 in [2]).

From this construction, and by taking the ratios of the corresponding residues
at $z=0$, we have $I_{0}=\delta$, where $\delta$ is the Dirac distribution at
the origin on $\mathbb{R}$ (see equation (3), pp. 57 in [2]), also
$\widehat{I_{z}}=cI_{1-z}$ for some real constant $c$ independent of $z$ (see
equation ($12^{\prime}$), pp. 173 in [2]).

For $z\in\mathbb{C}$, we also define $U_{z}$ as the distribution on
$\mathbb{H}^{n}$ given by the tensor product

$U_{z}=\delta_{\mathbb{R}^{2n}}\otimes I_{z},$

where $\delta_{\mathbb{R}^{2n}}$ is the Dirac distribution at the origin on
$\mathbb{R}^{2n}$ and $I_{z}$ is given by (7).

Let $\\{T_{z}\\}$ be the analytic family of operators on the strip
$-n\leq\Re(z)\leq 1$, given by

$T_{z}f=f\ast\mu_{A}\ast U_{z}.$

It is clear that $T_{0}=T_{\mu_{A}}$. For $\Re(z)=1$ we have

$\left\|T_{z}f\right\|_{\infty}=\left\|f\ast\mu_{A}\ast
U_{z}\right\|_{\infty}\leq\left\|f\right\|_{1}\left\|\mu_{A}\ast
U_{z}\right\|_{\infty}.$

Since $\mu_{A}\ast
U_{1+ib}(x,t)=I_{1+ib}(t-\varphi(x))=\frac{2^{-(1+ib)/2}}{\Gamma((1+ib)/2)}|t-\varphi(x)|^{ib}$
it follows that

$\left\|T_{1+ib}\right\|_{1,\infty}\leq\left|\frac{2^{-(1+ib)/2}}{\Gamma((1+ib)/2)}\right|,\,\,\forall\,b\in\mathbb{R}.$

For $\Re(z)=-n$, we will prove that the operator $T_{z}$ is bounded on
$L^{2}(\mathbb{H}^{n})$. This is equivalent to showing that

$\int_{\mathbb{R}^{2n}}\,\left|(T_{z}f)^{\lambda}(x)\right|^{2}\,dx\leq
c\int_{\mathbb{R}^{2n}}\,\left|f^{\lambda}(x)\right|^{2}\,dx,$

where $h^{\lambda}(x):=\int_{\mathbb{R}}h(x,t)e^{-i\lambda t}\,dt.$ A
computation gives

$(T_{-n+ib}f)^{\lambda}(x)=\widehat{I_{-n+ib}}(\lambda)\int_{\mathbb{R}^{2n}}\,f^{\lambda}(x-y)e_{\lambda
A}(y)e^{-i\lambda x^{t}Jy}dy$
$=\widehat{I_{-n+ib}}(\lambda)\,\left(f^{\lambda}\times_{\lambda}e_{\lambda
A}\right)(x).$

From the identity in (6) and since $\widehat{I_{z}}=cI_{1-z}$, we get

$\left\|(T_{-n+ib}f)^{\lambda}\right\|_{L^{2}(\mathbb{R}^{2n})}=\left|\frac{c2^{-(1+n-ib)/2}}{\Gamma\left(\frac{1+n-ib}{2}\right)}\right|(2\pi)^{n}|\det(2A\pm
J)|^{-1/2}\|f^{\lambda}\|_{L^{2}(\mathbb{R}^{2n})},$

for each $b\in\mathbb{R}$. So $T_{-n+ib}$ is bounded on
$L^{2}(\mathbb{H}^{n})$ if $\det(2A\pm J)\neq 0$. Finally, it is easy to see,
with the aid of the Stirling formula (see e.g. [13]), that the family
$\left\\{T_{z}\right\\}$ satisfies, on the strip $-n\leq\Re(z)\leq 1$, the
hypothesis of the complex interpolation theorem (see [12], pp. 205) and so
$T_{0}=T_{\mu_{A}}$ is bounded from $L^{\frac{2n+2}{2n+1}}(\mathbb{H}^{n})$
into $L^{2n+2}(\mathbb{H}^{n})$. ∎

######  Theorem 4.

Let $\nu_{0}$ be the measure defined by $(\ref{nu})$ with $\gamma=0$. If
$\det(2A\pm J)\neq 0$, then the type set $E_{\nu_{0}}$ is the closed triangle
with vertices $(0,0)$, $(1,1)$, and
$\left(\frac{2n+1}{2n+2},\frac{1}{2n+2}\right)$.

###### Proof.

Since the following inequality

$T_{\nu_{0}}f\leq T_{\mu_{A}}f$

holds for each borelian function $f\geq 0$, the theorem follows from the
restrictions that appear in (4), Theorem 3 and the Riesz convexity Theorem. ∎

######  Corollary 5.

If $\det(2A\pm J)\neq 0$, then the operator $T_{\mu_{A}}$ is bounded from
$L^{p}(\mathbb{H}^{n})$ into $L^{p}(\mathbb{H}^{n})$ if and only if
$p=\frac{2n+2}{2n+1}$ and $q=2n+2$.

###### Proof.

The ”if” part of the corollary is the theorem 3. To see the reciprocal we
introduce the action of the dilation group $\mathbb{R}^{>0}$ on
$\mathbb{H}^{n}$, i.e.: $\delta\cdot(x,t)=(\delta x,\delta^{2}t)$, $\delta>0$.
For a function $f$ defined on $\mathbb{H}^{n}$ we put
$f_{\delta}(x,t)=f(\delta\cdot(x,t))$. It is easy to check that

$(T_{\mu_{A}}f)_{\delta}=\delta^{2n}\,T_{\mu_{A}}(f_{\delta}).$

If $\|T_{\mu_{A}}f\|_{q}\leq c_{p,q}\|f\|_{p}$, then

$\delta^{-\frac{2n+2}{q}}\|T_{\mu_{A}}f\|_{q}=\|(T_{\mu_{A}}f)_{\delta}\|_{q}=\delta^{2n}\|T_{\mu_{A}}(f_{\delta})\|_{q}\leq\delta^{2n}c\|f_{\delta}\|_{p}=\delta^{2n-\frac{2n+2}{p}}c\|f\|_{p},$

for all $\delta>0$. So $\frac{1}{q}=\frac{1}{p}-\frac{2n}{2n+2}$. Since
$T_{\nu_{0}}f\leq T_{\mu_{A}}f$ for $f\geq 0$. From Theorem 4 it follows that
$p=\frac{2n+2}{2n+1}$ and $q=2n+2$. ∎

We recall that

$D=\left(\frac{4n^{2}+2n+\gamma}{2n(2n+2)},\frac{2n+(2n+1)\gamma}{2n(2n+2)}\right)=\left(\frac{1}{p_{D}},\frac{1}{q_{D}}\right)\text{
and\ }D^{\prime}=\left(1-\frac{1}{q_{D}},1-\frac{1}{p_{D}}\right).$

######  Theorem 6.

Let $\nu_{\gamma}$ be the measure defined by $(\ref{nu})$ with $0<\gamma<2n$.
If $\det(2A\pm J)\neq 0$, then the type set $E_{\nu_{\gamma}}$ is contained in
the closed trapezoid with vertices $(0,0)$, $(1,1)$, $D$ and $D^{\prime}$ with
the only possible exception of the closed segment joining the two points $D$
and $D^{\prime}$.

###### Proof.

For each $k\in\mathbb{N}\cup\\{0\\}$ we define the sets
$A_{k}\subset\mathbb{R}^{2n}$ by

$A_{k}=\left\\{y\in\mathbb{R}^{2n}:2^{-k}<\left|y\right|\leq
2^{-k+1}\right\\}$

Let $\nu_{\gamma,k}$ be the fractional Borel measure given by

$\nu_{\gamma,k}(E)=\int_{A_{k}}\chi_{E}\left(y,\varphi(y)\right)\eta\left(y\right)|y|^{-\gamma}dy$

and let $T_{\nu_{\gamma,k}}$ be its corresponding convolution operator, i.e:
$T_{\nu_{\gamma,k}}f=f\ast\nu_{\gamma,k}$. Now, it is clear that
$\nu_{\gamma}=\sum_{k}\nu_{\gamma,k}$ and
$\left\|T_{\nu_{\gamma}}\right\|_{p,q}\leq\sum_{k}\left\|T_{\nu_{\gamma,k}}\right\|_{p,q}$.
For $f\geq 0$ we have that

$\int_{\mathbb{H}^{n}}f(y,s)d\nu_{\gamma,k}(y,s)\leq
2^{k\gamma}\int_{\mathbb{R}^{2n}}f\left(y,\varphi(y)\right)\eta\left(y\right)dy.$

Thus $\left\|T_{\nu_{\gamma,k}}\right\|_{p,q}\leq
c2^{k\gamma}\left\|T_{\nu_{0}}\right\|_{p,q}$, from Theorem 4 it follows that

$\left\|T_{\nu_{\gamma,k}}\right\|_{\frac{2n+2}{2n+1},2n+2}\leq c2^{k\gamma}.$

It is easy to check that
$\left\|T_{\nu_{\gamma,k}}\right\|_{1,1}\leq\left|\nu_{\gamma,k}(\mathbb{R}^{2n+1})\right|\sim\int_{A_{k}}\left|y\right|^{-\gamma}dy=c2^{-k(2n-\gamma)}.$
For $0<\theta<1$, we define

$\left(\frac{1}{p_{\theta}},\frac{1}{q_{\theta}}\right)=\left(\frac{2n+1}{2n+2},\frac{1}{2n+2}\right)(1-\theta)+(1,1)\theta.$

By the Riesz convexity Theorem we have

$\left\|T_{\nu_{\gamma,k}}\right\|_{p_{\theta},q_{\theta}}\leq
c2^{k\gamma(1-\theta)-k(2n-\gamma)\theta}.$

Choosing $\theta$ such that $k\gamma(1-\theta)-k(2n-\gamma)\theta=0$ yields
$\displaystyle{\sup_{k\in\mathbb{N}}}\left\|T_{\nu_{\gamma,k}}\right\|_{p_{\theta},q_{\theta}}\leq
c<\infty$. A simple computation gives $\theta=\frac{2n-\gamma}{2n}$, then
$\left(\frac{1}{p_{\theta}},\frac{1}{q_{\theta}}\right)=\left(\frac{1}{p_{D}},\frac{1}{q_{D}}\right)$,
so $\left\|T_{\nu_{\gamma,k}}\right\|_{p_{D},q_{D}}\leq c$, where $c$ is
independent of $k$. Interpolating once again, but now between the points
$\left(\frac{1}{p_{D}},\frac{1}{q_{D}}\right)$ and $(1,1)$ we obtain, for each
$0<\tau<1$ fixed

$\left\|T_{\nu_{\gamma,k}}\right\|_{p_{\tau},q_{\tau}}\leq
c2^{-k(2n-\gamma)\tau}.$

Since
$\left\|T_{\nu_{\gamma}}\right\|_{p,q}\leq\sum_{k}\left\|T_{\nu_{\gamma,k}}\right\|_{p,q}$
and $0<\gamma<2n$, it follows that

$\left\|T_{\nu_{\gamma}}\right\|_{p_{\tau},q_{\tau}}\leq
c\sum_{k\in\mathbb{N}}2^{-k(2n-\gamma)\tau}<\infty.$

By duality we also have

$\left\|T_{\nu_{\gamma}}\right\|_{\frac{q_{\tau}}{q_{\tau}-1},\frac{p_{\tau}}{p_{\tau}-1}}\leq
c_{\tau}<\infty.$

Finally, the theorem follows from the Riesz convexity Theorem, and the
restrictions that appear in (4) and (5). ∎

We conclude this note with the following remarks.

######  Remark 7.

Let $\nu_{0}$ be the measure of compact support defined by $(\ref{nu})$, but
now with $\det(2A\pm J)=0$. In this case, by Theorem 1.1 in [9] and
Proposition 2, we can assure that the type set $E_{\nu_{0}}$ has a non-empty
interior.

######  Remark 8.

Lemma 1 provides us of examples of diagonal matrices $A$ such that $\det(2A\pm
J)=0$. By the above remark we know that the interior of the type set of the
measure $\nu_{0}=\eta\mu_{A}$ is non-empty. If $n\geq 2$ and $A$ also
satisfies that $\varphi(y)=y^{t}Ay=\sum_{j=1}^{n}\alpha_{j}|y_{j}|^{2}$
$(\alpha_{j}\in\mathbb{R}$ and $y_{j}\in\mathbb{R}^{2})$, then the type set of
$\nu_{0}$ is the closed triangle with vertices $(0,0)$, $(1,1)$ and
$(\frac{2n+1}{2n+2},\frac{1}{2n+2})$. This result is independent of the value
of $\det(2A\pm J)$ $($see Theorem 1, pp. 102 in [3]$)$.

These final comments illustrate the limits of the techniques used in this note
as well as of those developed in the works [3] and [4].

Acknowledgement. I express my thanks to Prof. Fulvio Ricci for his numerous
useful suggestions. My thanks also go to the referee for the useful
suggestions and corrections which made the manuscript more readable.

## References

  * [1] Carmo M. do, Riemannian Geometry. Birkhäuser. Boston - Basel - Berlin, (1992). Zbl 0752.53001
  * [2] I.M.Gelfand, G. E. Shilov, Generalized Functions, Properties and Operations. Vol. 1, Academic Press Inc., (1964). Zbl 0115.33101, MR 0166596
  * [3] T. Godoy and P. Rocha, $L^{p}-L^{q}$ estimates for some convolution operators with singular measures on the Heisenberg group. Colloq. Math., 132 (1), (2013), 101-111. Zbl 1277.43013, DOI: 10.4064/cm132-1-8
  * [4] T. Godoy and P. Rocha, Convolution Operators with Singular Measures of Fractional Type on the Heisenberg Group. Studia Math., 245, (3), (2019), 213-228. Zbl 1412.43009, DOI: 10.4064/sm8781-12-2017
  * [5] W. Littman, $L^{p}-L^{q}$ estimates for singular integral operators arising from hyperbolic equations. Proc. Symp. Pure Appl. Math., AMS 23, (1973), 479-481. Zbl 0263.44006, MR 0358443
  * [6] D. Oberlin, Convolution estimates for some measures on curves. Proc. Amer. Math. Soc. 99, (1), (1987), 56-60. Zbl 0613.43002, MR 866429, DOI: 10.1090/S0002-9939-1987-0866429-6
  * [7] Y. Pan, A remark on convolution with measures supported on curves. Canad. Math. Bull.Vol. 36 (2), (1993), 245-250. Zbl 0820.43002, MR 1222541, DOI: 10.4153/CMB-1993-035-2
  * [8] F. Ricci, Limitatezza $L^{p}-L^{q}$ per operatori di convoluzione definiti da misure singolari in $\mathbb{R}^{n}.$ Bollettino U.M.I. (7) 11-A, (1997), 237-252. Zbl 0946.42006
  * [9] F. Ricci and E. Stein. Harmonic analysis on nilpotent groups and singular integrals. III, Fractional integration along manifolds. Journal of Functional Analysis 86, (1989), 360-389. Zbl 0684.22006, MR 1021141
  * [10] S. Secco, $L^{p}$-improving properties of measures supported on curves on the Heisenberg group. Studia Mathematica 132 (2), (1999), 179-201. Zbl 0960.43009, MR 1669682
  * [11] S. Secco, $L^{p}$-improving properties of measures supported on curves on the Heisenberg group II. Bollettino dell’Unione Matematica Italiana, Serie 8, Vol. 5-B, n.2, (2002), 527-543. Zbl 1113.42012
  * [12] E. M. Stein and G. Weiss, Introduction to Fourier Analysis on Euclidean Spaces. Princeton University Press, (1971). Zbl 0232.42007
  * [13] E.M Stein and R. Shakarchi, Princeton Lectures in Analysis - Complex Analysis. Princeton University Press. Princeton, New Jersey, (2003). Zbl 1020.30001