# AutoGRAMS: Autonomous Graphical Agent Modeling Software

Ben Krause, Lucia Chen, Emmanuel Kahembwe
Autograms AI, USA
<EMAIL_ADDRESS>
###### Abstract

We introduce the AutoGRAMS framework for programming multi-step interactions
with language models. AutoGRAMS represents AI agents as a graph, where each
node can execute either a language modeling instruction or traditional code.
Likewise, transitions in the graph can be governed by either language modeling
decisions or traditional branch logic. AutoGRAMS supports using variables as
memory and allows nodes to call other AutoGRAMS graphs as functions. We show
how AutoGRAMS can be used to design highly sophisticated agents, including
self-referential agents that can modify their own graph. AutoGRAMS’s graph-
centric approach aids interpretability, controllability, and safety during the
design, development, and deployment of AI agents. We provide our framework as
open source at https://github.com/autograms/autograms.

###### Contents

  1. 1 Introduction
  2. 2 AutoGRAMS Framework
    1. 2.1 General overview
    2. 2.2 Node types
    3. 2.3 Transitions
    4. 2.4 Variables
      1. 2.4.1 Variable Assignment and references
      2. 2.4.2 Variable scopes
      3. 2.4.3 Example of Variable Usage
    5. 2.5 Function calls and scopes
      1. 2.5.1 Function Nodes and Calls
      2. 2.5.2 Return Transitions
      3. 2.5.3 Scopes in Function Calls
      4. 2.5.4 Example of Function Usage
      5. 2.5.5 Visualization of Function Calls
    6. 2.6 Methods of Implementation and Visualization
    7. 2.7 AutoGRAMS Configurations
  3. 3 AutoGRAMS graph compiler
  4. 4 AutoGRAMS interpreter
    1. 4.1 Main steps for obtaining conversational replies
    2. 4.2 Managing Memory
    3. 4.3 Interpreting Python statements
  5. 5 Self-modifying and meta autograms
    1. 5.1 Introduction
    2. 5.2 Meta autograms
    3. 5.3 Self-modifying autograms
    4. 5.4 Future Directions
  6. 6 Related Work
  7. 7 Discussion
  8. A Prompt formation
  9. B Language models in AutoGRAMS
  10. C Interjection Nodes
  11. D Simulating a user in conversational autograms
  12. E Wild card transition visualization
  13. F Additional details of the AutoGRAMS graph compiler
  14. G The reply algorithm
  15. H A simple self-referential autogram

##  1 Introduction

Artificial intelligence (AI) agents driven by large language models (LLMs) can
tackle complex, multi-faceted tasks. LLMs pretrained on large datasets
generate human-like text and display knowledge of various domains Radford et
al. (2019); Brown et al. (2020). This enables them to perform a range of tasks
and learn from training examples given in their prompt.

Training these language models to follow instructions enhances their utility,
allowing them to generate outputs that align with given instructions Ouyang et
al. (2022). This instruction-following behavior is crucial for the development
of AI agents that provide useful and contextually relevant responses across a
variety of applications (Yao et al., 2022; Paranjape et al., 2023; Khattab et
al., 2022). It also enables these agents to drive autonomous operations, such
as making decisions, accessing external information, and engaging in
conversations. Despite their potential, current approaches often struggle with
maintaining coherent behavior over extended interactions, adapting to
unexpected user inputs, and providing designers with intuitive and flexible
control over the agent’s decision-making process.

To overcome these limitations, we introduce the AutoGRAMS (Autonomous
Graphical Agent Modeling Software) framework to empower designers to
orchestrate intricate interactions with LLMs. AutoGRAMS enables the creation
of sophisticated AI agents and chatbots by representing their behavior as a
series of interconnected nodes, each with distinct actions and transition
rules. This graphical representation provides a programming language for agent
development, where the execution path dynamically adapts based on language
model predictions or predefined conditions.

With AutoGRAMS, designers gain the ability to:

  * •

Design branched multi-step interactions with a language model: Define a series
of interconnected nodes that represent the steps involved in language model
generation. This graph outlines a pre-defined procedure, and its nodes may
include multiple branches and decision points. Interaction is defined by graph
traversal, by iteratively stepping through the nodes of the graph.

  * •

Design complex conversational flows: Define a graph of conversational reply
steps with prompts that govern how the agent gives conversational replies and
makes decisions about which conversational branches to take.

  * •

Control interaction steps with with node-specific prompts: Craft prompts
tailored to each node, guiding the language model’s responses.

  * •

Leverage language model predictions in transitions: Define how language model
predictions are used to navigate branch points and guide the agent’s behavior.

  * •

Incorporate conditional logic: Define specific conditions under which
different actions or transitions occur.

  * •

Use variables to control memory: Define how variables in memory are set and
used by a language model

  * •

Integrate code: Define Python statements to manipulate variables or interact
with external APIs at any node of the interaction

  * •

Call agent modules as functions: Define callable agent systems that can be
called at points in the interaction, allowing an interaction to return to a
previous point, and giving additional control over prompt scopes.

  * •

Visualize agent behavior: Gain a clear understanding of the agent’s decision-
making process and current state through an intuitive graphical interface.

  * •

Design agent interactions in spreadsheets: Enable the design of complex agents
using spreadsheet software such as Microsoft Excel, Google Sheets, etc.

The designer does not necessarily need to be a person–the high level of
flexibility of AutoGRAMS makes it possible for an AutoGRAMS agent to design
other AutoGRAMS agents. AutoGRAMS also contains functionality to allow agents
to modify their own graph directly, opening up possibilities for agents that
learn by modifying their own behavior.

By bridging the gap between intuitive design and complex agent behavior,
AutoGRAMS opens up new possibilities for developing AI agents that can
effectively navigate diverse scenarios or tasks, adapt to user input, and
maintain coherent, goal-oriented conversations. Specifically, by enabling
designers to define the behavior of individual nodes and transitions flexibly
and easily within a graphical framework, AutoGRAMS facilitates precise control
over multi-step interactions with language models. This versatile tool serves
as both a high-level programming language and a comprehensive framework for
developing language model-based agents and chatbots.

The layout of this paper is as follows: We cover the AutoGRAMS framework in
Section 2. We demonstrate the mapping from python-like code and AutoGRAMS,
along with the AutoGRAMS compiler–in Section 3. We introduce how AutoGRAMS
agents are executed in Section 4. And finally, we discuss self-modifying
agents and agents that can design agents in Section 5.

##  2 AutoGRAMS Framework

We introduce AutoGRAMS and go over the process of defining a set of nodes and
associated fields to create agents in Subsection 2.1. We then delve into the
key concepts and mechanics of the AutoGRAMS frameworks, focusing on; nodes and
node types in Subsection 2.2, transitions in Subsection 2.3, variables and
memory in Subsection 2.4, functions in Subsection 2.5, methods of designing
autograms in Subsection 2.6, and finally, autogram configurations and settings
in Subsection 2.7. Some other details of the framework are included in the
appendix, including how prompts are formed (Appendix A), and details of how
language models are used (Appendix B), a description of interjection nodes
that help model unexpected user replies (Appendix C), and simulating user
replies in conversational autograms (Appendix D),

###  2.1 General overview

AutoGRAMS allows for a unified representation between graphical chatbots that
use states and transitions, general code, and approaches that leverage LLMs
internal reasoning abilities. An AutoGRAMS program, which we refer to as an
autogram (autonomous program) 111An “autogram” can also refer to a sentence
that describes itself by providing an inventory of its own characters.
Likewise, autograms in this framework are also self-referential–they contain a
reference to their own object that can be used to self-modify., is defined by
a collection of nodes and the transition functions between them. Within the
nodes, there are associated fields that define instructions to be executed and
transition behaviors/functions to other nodes.

Figure 1: Graphical representation of a chatbot created using an autogram with
only chat-type nodes. Nodes represents a point where the language model will
give a response, and each edge represents a different type of user response.
This particular chatbot models an AI tutor that repeatedly quizzes the user
about a subject. The chatbot starts with the ask_question node where it asks
the user a question, and then decides which node to visit next depending on
whether the user’s answer is right or wrong. If the answer is wrong, the
interaction goes to the answer_wrong node where the chatbot lets the user know
the answer is wrong and asks them to try again. If the user answers wrong
twice, the chatbot gives the answer. Eventually the chatbot reaches a node
where it repeatedly answers questions the user has about the original
question, and when the user is done asking questions, the chatbot revisits the
original ask_question node to ask another question. Each node executes a
specific planned step in the conversation. Each node has a separate
instruction for how the model should reply. Each node with multiple outgoing
edges contains a multiple choice question that the language model uses to
determine which node to visit next. So for instance, in this example, the
ask_question node has a transition question of “is the user’s answer correct?
A. Yes B. No”. If after viewing the user’s reply, the language model predicts
the answer is Yes, then the autogram will go to the answer_right node to get
the next instruction. Otherwise it will go to the answer_wrong node.

An autogram is represented by a set of nodes and their associated fields. The
nodes are used to form a data structure representing a graph, and execution of
the autogram consists of taking a trajectory of one or more steps along this
graph. In a conversational setting, one of the main goals is to model
different conversational trajectories, where each node represents a chatbot’s
reply, and each edge represents a user’s reply. For instance, consider the
conversational graph in Figure 1–An AI agent designed to quiz the user on a
subject may start by asking a question, and depending on whether the answer is
correct, will continue by either going over the answer with the user, or
asking them to try again. If a user repeatedly answers incorrectly, the agent
may provide an explanation and then proceed to a new question after addressing
any further inquiries. This type of agent can be modeled as a set of nodes,
each with a unique set of instructions and transition behaviors.

There are many possible fields than can be defined for each node, but some of
the most important fields are:

  * •

instruction: This field determines the node’s behavior and is interpreted
differently depending on the node type. The instruction could be a language
model prompt, executable code, or a call to another AutoGRAMS function.

  * •

action: This field determines the node type and how the instruction is
processed. In simpler scenarios, such as Figure 1, all nodes can be designated
as chat type, indicating that each node generates a response.

  * •

name: This provides a unique identifier for the node, allowing other nodes to
reference it during transitions.

  * •

transition_question: This is a question posed to the language model to help
determine the next node to transition to. Common examples include “What did
the user say?” or “Does the user want X?”

  * •

transition_choices: This is a multiple-choice list of potential answers to the
transition_question. For example, it could be [“The user said X“, “The user
said Y”, “The user said Z”], [“yes”, “no”], or similar. The language model
uses the transition_question and conversation history to predict the most
fitting transition_choices.

  * •

transitions: This is a list of node names that the agent can transition to,
depending on the predicted answer to the transition_question. For instance, if
transitions is [“node1”, “node2”], transition_question is “Does the user want
X?”, and transition_choices is [“yes”, “no”], then the autogram transitions to
“node1” if the model predicts “yes” and to “node2” if it predicts “no”. The
lists are order-dependent.

Each node in chatbots like the one described in Figure 1 performs the
following steps:

  1. 1.

Generate a Contextually Relevant Reply: This is achieved by passing node-
specific instructions to an instruction-following language model, ensuring the
reply aligns with the node’s intended purpose.

  2. 2.

Determine the Next Node: In conversational autograms, this decision may depend
on the user’s new reply. To accomplish this, the designer may:

     * •

Define a list of permissible next nodes within the graph structure.

     * •

Define a multiple-choice question about the user’s reply, with each answer
choice corresponding to one of the possible next nodes. This question is then
presented to a language model, and the predicted answer determines the node
transition.

Nodes come in different types as defined by their actions (see Section 2.2),
and in conversational settings (e.g. chatbots), chat-type nodes pause to
return a reply to the user, pausing the trajectory through the graph until the
user replies. When the user replies, the autogram can decide which of the
possible transitions to select based on the user’s reply. So for instance, a
node that provides a reply that asks a yes or no question may transition to a
different node depending on whether the user answers affirmatively or
negatively. This allows the designer of the autogram, which could be a person
or itself an autogram, to pre-program proactive responses that lead
conversations in well thought out ways towards objectives such as obtaining
information from a user, troubleshooting a problem step by step, or completing
a task.

Other types of nodes can perform other functions; e.g. use a language model to
generate thoughts, call external APIs, or even execute arbitrary Python code,
allowing for AutoGRAMS to be used for more general applications. Memory in an
AutoGRAMS is set by variables (Section 2.4) defined at the execution of nodes,
which can be referenced in later instructions or code executed by nodes at a
future point. The AutoGRAMS framework is Turing complete with respect to its
node definitions; the designer can implement nodes that result in loops and
conditionals using graph logic, and individual nodes can be configured to
execute basic code statements and variable assignments. We show that most
functional Python programs can be represented using an autogram, and we
implement an AutoGRAMS compiler that can convert vanilla Python code (Section
3). It also enables Python code and AutoGRAMS specific nodes to be easily
integrated together. AutoGRAMS subgraphs can also be called as functions,
allowing for AutoGRAMS modules to be reused (Section 2.5). An autogram can
also reference its own object, allowing an autogram to be fully self
referential and self-modifying (Section 5.3).

###  2.2 Node types

An autogram is defined by a set of nodes, where each node performs a single
step of execution. An execution step could contain anything from calling a
language model with a prompt to generate text, to executing arbitrary code.
Each node has an instruction that determines what will be executed at that
step. Each node also has an ’action’ which defines what type of node it is,
and how that instruction will be interpreted.

The main actions in AutoGRAMS, which also correspond to types of nodes, are as
follows

  * •

Chat actions:

    * –

chat: In these nodes, the instruction serves as a turn-specfic conversational
prompt for the language model, incorporating the current memory state of the
autogram. The autogram returns both the language model’s response and the
updated memory state, allowing the system to pause and wait for further user
input.

    * –

chat_exact: This action bypasses the language model’s response generation.
Instead, it directly outputs the instruction as a pre-programmed
conversational reply, providing more precise control over specific responses.

  * •

Thought actions: These nodes utilize the instruction as a text generation
prompt for the language model. Unlike chat nodes, there is no pause for user
input after execution; instead, the autogram immediately transitions to the
next node. This type of node is useful for internal reasoning steps.

  * •

Python actions: In this case, the instruction is treated as Python code and
executed directly by the Python interpreter, enabling custom logic and
functionality within the autogram.

  * •

Function actions: These actions trigger the execution of a separate subgraph
within the AutoGRAMS graph. The instruction contains a reference to a callable
AutoGRAMS node (potentially with arguments) to initiate this subgraph,
returning control to the calling node upon completion. (For more details,
refer to Section 2.5 on AutoGRAMS functions.)

  * •

Prompt actions: This action type focuses on modifying the initial prompt
presented to the language model. Language models receive both a turn-specific
prompt (the instructions for chat and thought nodes) and an initial,
overarching prompt. This action provides the capability to reset or alter this
initial prompt, influencing the model’s behavior across the entire
conversation.

  * •

Transition actions: These actions are primarily structural and do not utilize
the instruction field. They serve as placeholders within the AutoGRAMS graph,
allowing for additional branching points and greater control over the flow of
the conversation without requiring specific instructions.

We describe how AutoGRAMS instructions are executed depending on node type in
Figure 2.

Figure 2: Depiction of how a node’s instruction is executed in AutoGRAMS.
Behavior can vary greatly depending on the type of node. ‘$’ variables are
covered in Section 2.4

###  2.3 Transitions

After executing the instructions within a node, the autogram must select the
next node to execute. Each node contains fields that govern this selection
process.

Each node has a name field and a transitions field, which stores a list of
strings. In the simplest case, each string in transitions directly references
the name of another node in the graph. A node with only one possible
transition would list a single name, while nodes with branching possibilities
would list multiple names.

In other cases, a string in transitions may not directly refer to a node name.
For example:

  * •

Wildcard Transitions: A transition string with a “.” suffix (e.g.,
“mynode.*”) indicates a conditional transition. This assumes the existence of
nodes named “mynode.a,” “mynode.b,” etc., each with a defined
boolean_condition field containing a Python statement that may reference
variables in memory (see Section 2.4 for more on Variables). If-elseif-else
logic is then used to select the appropriate node based on the evaluation of
these boolean conditions. (see Algorithm 1 for a pseudo-code example, and
Appendix E for a visualization)

  * •

Return Transitions: A transition string of “return” (or “return variable_name”
to return a variable) signifies a return from a function call within an
AutoGRAMS function, transitioning back to the calling node.

  * •

Variable Transitions (advanced use case): A transition string can also
reference a variable, using its value in memory to determine the next node.
This requires careful validation to ensure the variable’s value corresponds to
a valid node name. A variable transition could become a return or wildcard
transition, since a variable could specify any string–which could correspond
to any transition type.

Function-type nodes override this standard transition behavior. They
transition directly to the node specified in their instruction, returning to
the original node upon function completion. (See Section 2.5 for details on
AutoGRAMS functions.)

Algorithm 1
Pseudo-code showing how wildcard transitions implement if/else logic. Assume
selected transition is “mynode.*”, and there is a dictionary called nodes
which contains nodes with names “mynode.a”,“mynode.b”,“mynode.c”

1:

2:if nodes[“mynode.a”].boolean_condition then

3:

4: $node\leftarrow$ nodes[“mynode.a”]

5:else if nodes[“mynode.b”].boolean_condition then

6:

7: $node\leftarrow$ nodes[“mynode.b”]

8:else

9:

10: $node\leftarrow$ nodes[“mynode.c”]

Figure 3 presents a flowchart of how nodes are selected from the previous
node.

Figure 3: Depiction of how the next node is selected from a previous one in
AutoGRAMS. If the previous instruction executed a function call, the typical
transition behavior is overridden and the autogram transitions to the node
called in the previous instruction. If the number of defined transitions for
that node is more than one, then the nodes transition question and transition
choices are used to construct a multiple choice question, which is passed to a
language model. The language model’s prediction determines which transition to
select. If the selected transition is a variable, this is set during
transition post-processing. If the selected transition corresponds to the name
of an existing node (standard transition), than that node is selected. If the
transition is a “return” transition, the previous function calling node is
selected as the next node. Lastly, if the transition is a wildcard transition
(ending in “.*”), if/else-if/else logic is used to determine which of 2 or
more nodes with the same prefix as the transition is selected.

###  2.4 Variables

AutoGRAMS allows the use of variables to store and manipulate memory, enabling
dynamic behavior within an autogram. Variables in AutoGRAMS are managed as
Python variables and can be created, referenced, and modified throughout the
execution of an autogram.

####  2.4.1 Variable Assignment and references

Variables are assigned using the ‘=‘ sign within the instructions of nodes.
The node’s instruction (with the assignment parsed out) will be executed in
accordance with the node’s action, as previously described in Section 2.2.
This means that the type of node will also influence what value is assigned to
the variable.

  * •

Python Nodes: Execute Python code directly. For example, a node with the
instruction,

x = [0, 1, 2]

initializes a variable x to a list.

  * •

Chat and Thought Nodes: Use the language model to generate text. For example,
an instruction like,

x = summarize the recent conversation history

will store the generated text in the variable x. In this specific example, the
text stored in the variable x will be a language model summary of the recent
conversation history. i.e. at run time, this instruction will be executed and
its results stored as a Python string in the variable x.

  * •

Function Nodes: When a function node is applied, it can pass variables as
arguments to the called AutoGRAMS function. The calling node’s instruction
specifies the variables being passed. Within the called function, these
variables are assigned and can be used or modified. For instance, a function
call,

summary = summarize(text)

will have the variable text accessible within the subgraph of the summarize()
node. When that subgraph encounters a return transition, it will return a
value that will be stored in the variable summary. See Section 2.5 for more
details on how this works.

Although it is technically possible to assign variables in other node types,
it is not recommended and currently has no utility.

Variables can also be referenced in two ways:

  1. 1.

Direct Use: Variables are used directly in Python code within the instructions
of Python or Function-type nodes, or in the boolean_condition attribute of a
node (used for wildcard-transitions). For instance:

            sorted(list1)

could be used as an instruction for a Python node to sort a list called list1
that was assigned as a variable at a previous node.

  2. 2.

$ Variable Embedding ($ syntax): Variables are embedded as strings within
instructions or other node fields using $ syntax. This is useful in chat and
thought-type nodes. For example:

            Here is a summary of what was previously discussed: $summary.
        Reply to the user and remind them what was previously discussed.

$ variable embedding can also be applied in the transition_question attribute
of a node to allow the multiple transition question to change dynamically.
They can also be used as transitions directly, although this requires care.
For example, setting

    transitions = ["node1", "node2", "$variable_node"]

, for a particular node would allow the third transition to be dynamically
dependant on the value of variable_node, however this will not execute
successfully if variable_node is not a valid transition.

####  2.4.2 Variable scopes

The memory in AutoGRAMS is managed through a memory object, which stores
variables and the conversation history. This object is structured as a stack,
where each function call adds a new layer, and each return removes the top
layer. If no functions are used, a variable can be accessed anywhere within an
autogram. However, local functions are unable to view their calling scope,
including any variables in that calling scope. See Section 2.5 for more
details.

####  2.4.3 Example of Variable Usage

Consider a set of nodes that demonstrate variable assignment and referencing:

    name: "write_topics"
    instruction: "topics = List the topics covered in today’s lesson."
    action: "thought"
    transitions: ["append_topics"]

    name: "append_topics"
    instruction: "all_topics.append(topics)"
    action: "python_function"
    transitions: ["tell_topics"]

    name: "tell_topics"
    instruction: "Inform the user about the topics covered today, which were: $topics"
    action: "chat"

In this example:

  * •

The write_topics node initializes calls a language model to write down a list
of topics and store it in a variable called topics, which will be a string.

  * •

The append_topics node applies its instruction as Python code, appending the
variable topics to a list called all_topics that is presumed to have been
defined previously.

  * •

The tell_topics node uses $ variable embedding to dynamically insert the
topics variable within the instruction string to get a response.

By utilizing variables, AutoGRAMS facilitates memory management and dynamic
interactions, enabling the creation of sophisticated and adaptable AI agents.

Figure 4: Example of a set of nodes that use variables. The exact behavior of
variable assignment can depend on the type of node, but for thought and chat
style nodes, this is the text generated by the language model, and for python-
type nodes this is the result of the code execution using the Python
interpreter. Variable assignment allows for memories to be stored that can be
referenced in future nodes. Variables can be referenced in 2 ways–directly or
using $-syntax. Direct use means using a variable as is, and can only be used
in fields that will be passed to directly the Python interpreter. This
includes the instruction of python-type nodes (see the $python\\_function$
node above), and the $boolean\\_condition$ attribute of a node that is often
used in combination with wild card transitions. $-syntax means embed the
variable as a string on the fly; for instance in the example above, the model
will reply with the exact text set in the topics variable inserted into the
instruction (a $chat\\_exact$ node replies with the exact instruction without
calling the language model). $-variable references are most straightforwardly
used in $instruction$, $transition\\_question$ to facilitate on-the-fly
instructions to a language model or on-the-fly transition questions that
incorporate a memory. With care, $-variable references can also be used in the
instruction of python-type nodes to create dynamically changing code, or as
elements of the transition list to allow for dynamically changing transitions
in the graph.

###  2.5 Function calls and scopes

AutoGRAMS introduces the ability to call graphical modules as functions,
allowing subgraphs to be called similarly to functions in programming. This
enables modularity, reusability, and control over variable scopes.

####  2.5.1 Function Nodes and Calls

Function nodes allow for the execution of separate subgraphs within the main
AutoGRAMS graph. These nodes can call other nodes within the graph or external
programs. The process involves:

  * •

Defining the Function Node: A node designated as a function node will have its
instruction specify the function to be called, such as,

summary = summarize(document).

  * •

Passing Arguments: Variables can be passed as arguments to the called
function. Within the function, these variables are assigned and can be used or
modified.

  * •

Defining a callable node: A node can be made callable by its name. Node name
with parentheses are callable. So a node named summarize cannot be called a s
function, a node named summarize() can be called as as function with no
arguments. A node named summarize(text1,text2) can be called with 2 arguments.
The arguments specified by the node name determine what the variables will be
named in the scope of the called node.

  * •

Defining the Subgraph of the callable node: The callable node, which acts as
the root node of the subgraph, can have transitions to other nodes that
perform other operations. Transitions use the string “return” or “return
varname” can be added to specify that the subgraph should end.

  * •

Executing the Subgraph of the callable node: The function node transitions to
the callable node specified in the instruction. The autogram executes the
subgraph of the callable node until it reaches a return transition, which may
optionally return specific information back to the calling node.

####  2.5.2 Return Transitions

Return transitions mark the end of a function’s execution and return control
to the calling node. They can also return specific values to be used by the
calling node. For example, return topics_summary will pass the value of
topics_summary back to the calling node.

####  2.5.3 Scopes in Function Calls

The scope of variables in function calls is crucial for maintaining the
integrity of the memory and ensuring correct execution. AutoGRAMS supports
different scope types for function calls:

  * •

Local Scope: The function can only access variables passed as arguments.
Variables within the function do not affect the calling scope. This is
specified using the local_function action of the calling node.

  * •

Global Scope: The function can access and modify all variables and
conversation turns in the calling scope. This is specified using the
global_function action of the calling node.

  * •

Mixed Scope: The function can read all variables and conversation turns from
the calling scope, but the variable and conversation turns set during
execution are erased after returning. This is specified using the function
action of the calling node.

####  2.5.4 Example of Function Usage

Consider a set of nodes that demonstrate the use of function calls and scopes:

    name: "call_summarize_docs"
    instruction: "summary = summarize_and_combine(document1,document2)"
    action: "local_function"
    transitions=["process_summary"]

    name: "summarize_and_combine(text1,text2)"
    instruction: "summary1=write a summary of the following text: $text1"
    action: "thought"
    transitions: ["summarize_second"]

    name: "summarize_second"
    instruction: "summary2=write a summary of the following text: $text2"
    action: "thought"
    transitions: ["combine_summaries"]

    name: "combine_summaries"
    instruction: "combined_summary=Write a summary combining
                summary 1 -- $summary1
                summary 2 -- $summary2"
    action: "thought"
    transitions: ["return combined_summary"]

In this example:

  * •

The call_summarize_topics node calls the summarize_and_combine() function
using its instruction. The document1 and document2 variables are passed as
arguments.

  * •

summarize_and_combine() function starts at the node named
summarize_and_combine(text1,text2). The variable document1 in the calling
namespace will be mapped to text1 in the called namespace, and the variable
document2 in the calling name space will be mapped to text2 in the called
namespace.

  * •

After the function call, the autogram will execute the summarize_and_combine()
node, followed by the summarize_second and combine_summaries nodes.

  * •

After executing the combine_summaries node, it encounters a return transition
specifying return combined_summary. This will cause the autogram to return to
the calling node, which was call_summarize_docs. The variable summary defined
by the instruction of call_summarize_docs will be set equal to the returned
variable combined_summary.

  * •

after the return is complete, the autogram will transition to to a node called
process_summary (not defined above) specified by the transitions of the
call_summarize_docs node.

####  2.5.5 Visualization of Function Calls

Figures 5(a) and 5(b) illustrate how function calls are managed within
AutoGRAMS, including the transitions and scope handling.

(a) An autogram where multiple branches use a function call to the same
conversational module to ask a question. Dashed lines represent function
calls. When the return node is encountered, the autogram will transition to
either call_ask_math_question or call_ask_history_question, depending on which
node called it. After returning, the autogram will go to either
write_math_question or write_history_question and repeat the process. For
AutoGRAMS functions, the variables passed are specified in the calling node’s
instruction and are available within the function’s scope using the name of
the called node. The node’s name must include parentheses and the appropriate
arguments to be callable. The global_function action defined in the calling
node specifies that conversation turns that occur during the function call
will still be visible after returning.

(b) AutoGRAMS functions can also be utilized for non-conversational modules to
enable module reuse or scope control. For example, setting the node action to
function for write_math_question() allows the node to access past conversation
history without permanently altering the stored conversation turns. The use of
this scope with functions containing thought-type nodes prevents unnecessary
additions to the permanent conversation history, allowing for more precise
prompt control and reducing potential noise. The resulting question can be
stored in a variable for later use. Additionally, since AutoGRAMS functions
can be called externally from Python, they are suitable for non-conversational
applications where the only external inputs are Python variables that the
function can accept.

Figure 5: Depiction of function calls in AutoGRAMS. (a) depicts a function
with chat-style nodes that allows the conversation to return to the calling
node when a function returns. (b) depicts a function with thought-style nodes,
which is useful for reusing modules and controlling scopes. AutoGRAMS
functions can be called from within an autogram or from Python using the
autogram.apply_fn() method.

###  2.6 Methods of Implementation and Visualization

At the time of writing, there are three ways to implement an autogram:

  1. 1.

Spreadsheet-Based Design:

     * •

Each row in the spreadsheet corresponds to a different node in the AutoGRAM.

     * •

Each column corresponds to a different node field, determined from the heading
of the column.

     * •

This method allows autograms to be defined and managed using a familiar
spreadsheet interface.

  2. 2.

Pure Python Implementation:

     * •

Nodes of an autogram can be defined in Python by initializing an autogram
object and using the autogram.add_node() method to create new nodes with
fields corresponding to the arguments provided.

     * •

This is the most direct way to design an autogram, as all other methods of
design are mapped to this.

     * •

The main limitation is that it can be inconvenient for autograms that need to
execute Python code within the autogram. Python code can be passed in as a
string instruction, but this makes the code more difficult to read.

     * •

Standard conditionals and loops can theoretically be implemented using
wildcard transitions and loops in the graph with exiting branches, but these
can be inconvenient to define graphically.

  3. 3.

AutoGRAMS Compiled from Python:

     * •

This method works best for autograms deeply integrated with Python code.

     * •

It is technically a new language that closely relates to Python but has some
differences that allow Python code to be integrated with AutoGRAMS nodes using
Python syntax.

     * •

AutoGRAMS is general enough to execute most functional Python programs using a
combination of Python and transition-type nodes along with the right graph
structure and wildcard transitions for branching.

     * •

AutoGRAMS compiled from Python applies this conversion automatically, while
also allowing autogram nodes with non-standard Python behaviors to be defined
using a special function called exec_node, which can also be written in the
code to define where in the code those nodes should be applied.

Any autogram can be fully visualized using a graphical interface. An
interactive HTML file can be automatically generated for any autogram,
displaying all the fields associated with each node. By clicking on a node,
the code for that node is shown in a panel to the right. For more complex
autograms, it is possible to specify subcategories of nodes to visualize,
making the graph easier to understand, provided the category attribute is used
in the nodes defined in the autogram. An interactive graph of an autogram is
shown in Figure 6. Future versions of AutoGRAMS will enable users to directly
design and edit autograms from a similar interface.

Figure 6: AutoGRAMS can also be used to generate interactive graphs that allow
all the fields associated with each node to be viewed by clicking on the
appropriate node. It can be useful to periodically re-compile and view the
graph while implementing an autogram. It is also possible to define node
categories to allow a subset of the graph to be visualized. It is
theoretically possible for any autogram to be implemented completely in a
graphical interface such as the one above, and in the future we plan to
implement graph-based AutoGRAMS editors that allow node fields to be changed
and nodes to be added or removed.

###  2.7 AutoGRAMS Configurations

AutoGRAMS configurations are essential for controlling various settings of an
autogram. Key aspects of these configurations include:

  * •

Model Selection: Defining which models will be used within the autogram.

  * •

Prompt Templates: Establishing templates to guide the language model
responses.

  * •

Python Imports and APIs: Specifying the Python modules and APIs accessible
from the AutoGRAMS code.

Configurations are typically managed through a JSON file, with the exception
of the Python modules field. This field requires actual Python modules
containing code, which are passed to the autogram, allowing the use of any
Python-defined function within Python function nodes.

For further details:

  * •

Appendix A: Provides an overview of how prompts for language models are set.

  * •

Appendix B: Details the language models used in AutoGRAMS and their
configuration settings.

These settings ensure that the autogram operates correctly, utilizing the
appropriate models, prompts, and Python functionalities.

##  3 AutoGRAMS graph compiler

AutoGRAMS gives the ability to embed general code within graphical AI agents
and chatbots. One way to design autograms is to define nodes one-by-one, with
transitions to other nodes specified in node definitions. This is especially
useful for chatbots, since conversations often have certain states that can be
modeled as a graph or tree. AutoGRAMS also allows nodes in this graph to
execute code, rather than give a reply. However, this requires the code that
executes at a node to be passed as a string in a node’s instruction. This
graphical representation also means that loops and conditionals in the
AutoGRAMS graph need to be defined using the appropriate transitions. We were
motivated to allow deeper integration between general code statements and the
graphical representations used by autograms. To do this, we implemented the
AutoGRAMS compiler, which can map a combination of Python code and statements
that define AutoGRAMS nodes, to an autogram that can be executed by the
AutoGRAMS interpreter (Section 4). It does this by mapping the code in a file
to a set of nodes with the appropriate attributes to perform the computation
specified in the code file.

AutoGRAMS is general enough to execute functional programs that can be
represented common features such as loops, conditionals, functions and
variables. This is possible by combining python-type nodes with the desired
flowchart needed to execute a program. For instance, consider a program that
computes the nth element Fibonacci sequence recursively (While this is
inefficient, we use it for illustrative purposes). Python code for this is
given in Figure 7(a).

(a) Python code for computing nth Fibonacci sequence number recursively.
Functional Python code can be compiled into an AutoGRAMS graph automatically,
with a few limitations on statements that call multiple functions or try to
assign multiple variables.

(b) Automatically compiled AutoGRAMS graph for the code in (a). Code elements
such as loops, conditionals, and function calls (including recursive function
calls) can be handled by the autograms compiler, traverses the Python abstract
syntax tree to compile an autogram from Python code. When the above autogram
executes, the graph will follow the same trajectory as the program in (a) and
yield an equivalent result. The AutoGRAMS compiler also allows other node
types that use standard AutoGRAMS transitions to be interleaved with normal
Python code, allowing Python coding elements to easily be conveniently be
combined with any autogram.

Figure 7: Mapping from (a) Python code to (b) AutoGRAMS graph computed
automatically by the AutoGRAMS compiler. With a few exceptions that are not
yet implemented, functional Python code can be mapped to an autogram directly
by compiling the AutoGRAMS graph using the loops, conditions, functions, and
variable assignments in the code. The AutoGRAMS compiler also allows for
Python code, which will be mapped to python-type nodes, to be interleaved with
AutoGRAMS thought and chat-type nodes. It also allows for AutoGRAMS functions
and variables assignments to be written with similar syntax to Python

The function $fibonacci(n)$ accepts a number, returns 0 of the number is 1,
returns 1 if the number is 2, and returns the sum of $fibonacci(n-1)$ and
$fibonacci(n-2)$ if the number is greater than 2. this function can be
represented completely in AutoGRAMS with a combination of python-type nodes,
function-calling nodes, transition nodes, return transitions, and wild card
transitions. The first node of the graph must be a callable node named
“fibonacci(n)”. The parentheses in the name means that it can be called
function calling nodes, and using n means that the first argument passed can
be referenced with the variable name “n”. A node with an action
$local\\_function$ and instruction of $x=fibonacci(n)$ can be used to call
this function as set the result to a variable called “x”.

The first conditional of the program is implemented with a wildcard
transition, where the root node of the conditional applies a transition to
$fibonacci\\_conditional.*$, which means that the next node will either be
$fibonacci\\_conditional.a$, $fibonacci\\_conditional.b$, etc. In this
specific case, since there are 4 possible branches, it is necessary to define
nodes named $fibonacci\\_conditional.c$ and $fibonacci\\_conditional.d$. Each
of these nodes has a $boolean\\_condition$ attribute that can be used to
specify the condition at which this branch can be executed. Boolean conditions
are passed to the Python interpreter in AutoGRAMS so they can include any
Python code or references to any variables visible at the current scope.
$fibonacci\\_conditional.d$ does not need a boolean conditional because it is
the last node in alphabetical order. If the $n==1$ or $n==2$ condition are
reached, the appropriate value can be returned by using a python-type that
simply includes the desired number, and then applying a return transition
immediately after, which returns the result of the previous node if no return
variable is specified. If the else condition is met reaching the node
$fibonacci\\_conditional.d$, the autogram can use two $local\\_function$ nodes
that call the Fibonacci graph again with $n-1$ and $n-2$, sum the result, and
return the result. The AutoGRAMS graph, which was compiled automatically from
the Python code using the AutoGRAMS compiler, is given in Figure 7(b).

The AutoGRAMS compiler is used to convert code with python-like syntax into a
set of AutoGRAMS nodes that can be executed as an autogram. It is also
possible to code any autogram in pure Python (or a spreadsheet) by defining
nodes one by one, in which case the AutoGRAMS compiler is not used. However,
for autograms that contain Python statements or programmatic features like
loops and conditionals, the compiled version has several advantages:

  * •

Python statements can be included directly as code instead of as strings in
the instruction of python_function nodes

  * •

Python style variable assignments can be be used instead of assignment
variable in an instruction

  * •

AutoGRAMS function calls and functions can be defined using Python-like
syntax. Note that functions are by default treated as local AutoGRAMS
functions, but this behavior can be changed using the function decorator
‘@global_function” or “@function” above the function definition.

  * •

the order of node chains to be inferred automatically from the order they
appear in code, even if names and transitions aren’t defined.

  * •

Python loops and if/else conditionals to compile into a graph automatically

Compared with standard Python programs, AutoGRAMS compiled from Python allows
for exec_node() statements to define AutoGRAMS nodes at any point in the code.
These nodes can connect with other nodes defined by exec_node() by specifying
those nodes in the transitions argument, in which case these transitions will
be governed by multiple choice questions asked to the language model. Chat
type nodes defined in exec_node() temporarily return a serializable object
defining the entire state of the autogram (see Section 4.2), allowing a
conversation to continue from a specific place in the code after receiving a
new user reply. The graphical representations used by AutoGRAMS also give it
certain advantages for self-modifying code, as discussed in Section 5.

The AutoGRAMS compiler recursively traverses the Python abstract syntax tree
representation of the code to add AutoGRAMS nodes to the graph. A overview of
how the Autograms compiler converts code into AutoGRAMS nodes is given in
Appendix F.

##  4 AutoGRAMS interpreter

Autograms are executed via an iterative process that selects nodes and
executes them. Within the AutoGRAMS framework, an autogram can be applied
using the reply method, which is for conversational modules, and the apply_fn
method, which allows an AutoGRAMS function to be called directly from Python,
and it meant for non-conversational modules. Both work in a very similar
fashion, other than the way they terminate their main loop. The reply method
terminates and returns a result when it encounters a chat node, and is
designed to restart from the node it left off at when it receives another user
reply. The apply_fn method terminates when it encounters a return
statement–meaning the AutoGRAMS function is done executing and is returning a
result. An illustration of the outer loop of the AutoGRAMS reply method is
given in Figure 8.

###  4.1 Main steps for obtaining conversational replies

The outer loop of autogram.reply() has 6 main steps:

  1. 1.

get the variable output of previous node (skip this step if no previous node).
This step calls a node specific method called get_variable_output() which
returns the variable output of the node. The specific behavior of this will
depend on the node type, but this is often the text generated by the chatbot.

  2. 2.

assign variables assigned in previous node to memory (skip this step if no
previous node). If any variable outputs were in the previous node’s
instruction, the memory object (Section 4.2) assigns the nodes variable output
to a variable with the name defined in that instruction. This goes in the top
level of the memory objects stack.

  3. 3.

apply transition function from previous node (skip this step if no previous
node). This calls a node specific method called apply_transition() to get an
unprocessed new node id. The behavior of this method will can be different for
different types of nodes, but is generally similar for non-function calling
nodes. If there is only 1 possible transition in the transitions list of the
node, then the result will usually this transition.If there are multiple
transitions, the transition will depend on what the classifier predicts.There
also may be interjection transitions if the node is a chat type. If the node
is calling a function, the new node id will be the root node being called.

  4. 4.

post-process the new node id (skip this step if no previous node). If the new
node id output by apply_transition() corresponds to another node in the graph,
and the next step will be to simply select that node. However, if the new node
id corresponds to a return statement or a wildcard transition, additional post
processing will be needed. A return statement will require using the memory
object to find the previous function calling node, and transition back to this
node. A wild card transition requires finding the nodes with the matching
prefix and applying if/else logic to determine which node to select.

  5. 5.

get the new node and apply its instruction. This calls a node specific method
called $apply\\_instruction()$ to execute the instruction of the node. This
can very greatly for different node types. Apply instruction will always strip
any variable assignments on the left of the node’s instruction string. It will
then render the string for any $ variables in the instruction. The
$apply\\_instruction()$ for chat and thought type nodes will then call the
language model with the instruction and conversation history used in the
prompt. In python-type nodes, the instruction will be executed as code by the
Python eval function, with all in scope variables and Python modules set by
the autogram config included as local variables.

  6. 6.

if the new node is a chat node, return a result with the models reply and the
memory object storing the state of the program, continue the loop otherwise

Figure 8: Illustration of the core outer loop of the AutoGRAMS reply method.
An autogram takes in a user reply and a memory object that models the full
state of the autogram. For at least one iteration, the autogram selects the
most recently executed node as saved in the memory object, sets any variables
that were assigned at that node, and applies that node’s transition function
to select the next node. These first steps are skipped if no nodes have been
executed yet and/or if the memory object is empty, in which case a designated
start node is selected. The instruction of whichever node is executed, where
the instruction may call a language model, be used to execute code or call an
external Python API, or perform a number of other actions depending on the
action defined for the node. If the node is a chat-type node, it will return a
reply to the user. The autogram will continue to propagate along a trajectory
through the graph until it encounters a chat-type node.

###  4.2 Managing Memory

Figure 9: Illustration of the memory object stack during 3 different points-1.
the calling, 2. the execution, and 3. the returning of a function of type
global_function. Each layer of the memory object’s stack stores a dictionary
of variables assigned and the language model (LM) turns executed at that
layer. When a function is called, a new layer is added to the stack. When
variables or LM turns are retrieved during the execution of nodes, the memory
object retrieves all previous turns and variables until it encounters a stack
layer corresponding to a function of type local_function, which cuts off
access to lower layers of the stack. When the layer corresponding to a
global_function returns, all variables and LM turns are added to the calling
layer, which in the above example is a local_function layer.

AutoGRAMS uses a “memory object” to keep track of the state of the program,
including all the variables set by AutoGRAMS nodes and conversation turns. The
memory object stores a stack of all of the layers of functions in an autogram.
Each function call adds a layer to the stack, and each function return removes
a layer from the stack. The type of function determines which variables and
language model conversation turns are considered to be in scope at a certain
layer of the stack. Within a global function or regular function, variables
and conversation turns at the calling layer are visible, whereas in a local
function these are not visible. Methods for obtaining variables and language
model conversation turns within the memory object class add all variables or
turns until they encounter a layer corresponding to a local function, which
cuts off access to everything below the local function in the stack. When
returning from a function, a layer in the stack is deleted. If the function
was a global function, all variables and conversation turns are added to the
calling layer. Otherwise these fields are erased, and only variables modified
or returned by the function will be remembered. The stack of the memory object
is illustrated in Figure 9.

In addition to storing the stack, the memory object is also used to log model
turns and nodes visited. The entire state of an autogram can be recovered from
the memory object, allowing autograms to continue from where they left off
after executing chat-type nodes that wait for a user reply. The memory object
and user reply can be passed to the autogram to obtain the model’s reply.

###  4.3 Interpreting Python statements

Python statements (as well as the ‘boolean_condition‘ field for wild card
transitions, and arguments to AutoGRAMS functions) are interpreted by the
Statement Interpreter. At initialization, the Statement Interpreter uses the
Autogram Config to load all Python imports and modules that will be allowed
within the scope of the program. It also overrides any Python builtins that
are not explicitly listed in the Autogram Config to prevent them from being
called. These imports are effectively treated as global variables that can be
accessed anywhere from within the autogram. When the Statement Interpreter
called to execute code, it uses the Python eval command–it includes all
variables in scope in the memory object, as well as Python modules and imports
passed from the Autogram Config in the scope of the code execution during the
call to eval.

##  5 Self-modifying and meta autograms

###  5.1 Introduction

AutoGRAMS allows a designer to specify a series of instructions and
transitions that form a program that control this process. However, the space
of possible useful processes like this that can be formed is extremely large.
For instance, there are likely many processes in AutoGRAMS that would be
useful for solving certain problems, or handling certain conversational
scenarios. It could be difficult for the designer to specify every possible
process that the agent may need to handle. It would therefore be very useful
to have a “meta-process” that can design large sets of these processes in
advance, or be able to define new processes on the fly when encountering new
scenarios.

AutoGRAMS is designed in such a way that the meta-process can be of the same
form (an AutoGRAMS graph) as the processes that the meta-process designs. This
makes it possible to design meta-processes that can self-modify, giving
greater flexibility as compared with a system where the meta-process is of a
different form from the process. Within AutoGRAMS, we refer to these meta-
processes as meta-autograms.

Figure 10: An autogram can be used to design another autogram, which we refer
to as a meta-autogram. For instance from a prompt. This diagram gives an
overview of a meta-autogram we developed. A series of thought-type nodes can
be used to first draw a high level representation of the new autogram’s graph.
After drafting this, the meta-autogram can extract the nodes in the new graph
as a list, and iterate through thought nodes that write the fields for each
node in the new autogram. The meta-autogram finishes by returning the
arguments needed to create a new autogram based on the instructions fed in at
the start of the meta-autogram.

###  5.2 Meta autograms

One of the goals of AutoGRAMS was to design an agent framework general enough
that an agent could be used to design another agent of the same form.
AutoGRAMS provides a simple working example of an AutoGRAMS function that can
be used to design a chatbot autogram. The function, which we refer to as a
meta-autogram, takes in a prompt as an argument. It applies the following
steps

  * •

apply a thought node that asks the language model to design an outline of the
possible ways the conversation can go from this prompt

  * •

apply a thought node that asks the language model to design a graph using the
dot (graphviz) 222https://graphviz.org/ language, where each node should be
labeled by how the agent should reply, and each edge should be labeled by how
the user should reply.

  * •

call an external Python module to parse the nodes from the graphviz graph, go
back to step 2 if problems with the graph are detected

  * •

For each node parsed from the graphviz graph, call thought type nodes to
generate the attributes associated with that node. The prompt for the thought
node conditions on the graph labels for that node from step 2 to guide it. All
the attributes are saved as variables and used to form a diction ary of
arguments for each nodes

  * •

return the arguments needed to initialize a new autogram

This process is visualized in Figure 10.

###  5.3 Self-modifying autograms

One advantage of the graphical representation of AutoGRAMS is that a node at
one point in the graph can perform operations that modify nodes at other
points in the graph. For instance, a series of nodes could be configured to
perform the following operations

  1. 1.

select a node in the graph

  2. 2.

call a language model to decide what the new attributes of that node should
be, and save the arguments to define this node in variables

  3. 3.

if any new nodes need to be defined (because transitions of original node
changed), loop through and define new nodes using the process in step 2.
Continue to define child nodes until all nodes are connected to the graph (the
system will eventually need to connect all nodes back to existing nodes so
that it doesn’t get stuck in an infinite loop).

  4. 4.

loop through to initialize/modify nodes using the arguments saved in variables
during steps 2 and 3

This is one of many possible ways an autogram could self-modify by using nodes
to modify other nodes or adding new nodes.

We provided the ability for autograms to directly access their own data
structure during their execution when running in “self-referential” mode. An
autogram is implemented as an instance of the Autogram class in Python, and
when self-referential mode is enabled in the autogram config, nodes are able
to reference a variable called self which refers to the autogram’s own object
that interprets the autogram. This potentially allows any of the autograms
nodes, methods, or other attributes to be accessed during the execution of the
autogram. One potential usecase for this is dynamic modification of the
autogram. For instance, it is possible for an autogram to add new node to
itself by calling the self.add_node() method with the appropriate arguments
from a python-type node. We implemented a simple proof of concept of this
where an autogram simply adds and executes a new node at each turn, instead of
having the nodes defined before hand. This works by having an AutoGRAMS
function to generate all the attributes of a new node and add it to the
autogram, and then when the function returns, apply a dynamically defined
transition with a variable name that specifies the name of the new node. The
new node then transitions back to the node that calls the function to design a
node, allowing the chatbot to continue indefinitely. The full set of nodes
needed to perform this algorithm along with explanations are provided in
Appendix H.

###  5.4 Future Directions

Our future work will focus on developing software libraries of self-modifying
autograms capable of defining useful autogram processes and learning from
their interactions with the environment. In this paper, we demonstrated an
autogram that can design an AutoGRAMS chatbot based on a prompt describing the
chatbot’s required functionality. While the scope of the present paper was to
develop useful representations for designing such algorithms, future work will
aim to implement autograms that perform these functions.

Future applications could include autograms that process conversations
conducted by a human agent and generate new autograms to replicate the human
agent’s logic. Additionally, an autogram could be programmed to analyze its
own interactions, identify where it went wrong due to unexpected user input,
and modify its nodes accordingly.

Advanced autograms could also retrieve information on complex topics and
adjust their reasoning structures to better integrate this material into their
conversational or reasoning routines. This has significant implications for
using large language models or generative AI in robotics. By fine-tuning
language models to better model autograms, we hope to enhance the ability of
autograms to design other autograms and self-modify.

##  6 Related Work

The development of AutoGRAMS is situated within a broader landscape of
research encompassing natural language processing (NLP), AI agents, and
programming languages for AI. This section reviews key advancements and their
contributions to the field, highlighting how AutoGRAMS builds on and extends
these innovations.

Significant progress in NLP and neural language modeling (Bengio et al., 2000)
has been driven by transformer-based architectures (Vaswani et al., 2017),
such as BERT (Bidirectional Encoder Representations from Transformers) Devlin
et al. (2019), GPT-3 (Generative Pre-trained Transformer 3) Brown et al.
(2020), and T5 (Text-To-Text Transfer Transformer) Raffel et al. (2020). These
models have shown exceptional capabilities in understanding and generating
human-like text, providing a robust foundation for developing advanced
conversational agents. Notable conversational agents include Google’s Meena
Adiwardana et al. (2020), Facebook’s BlenderBot Roller et al. (2020), and
Microsoft’s DialoGPT Zhang et al. (2019), which leverage large-scale pre-
training and fine-tuning on diverse datasets to perform a variety of
conversational tasks. AutoGRAMS relies heavily on the ability of pretrained
Transformer models to generate realistic text that can be used in
conversational replies and apply reasoning steps.

The intersection of programming languages and AI has been extensively explored
to facilitate the integration and development of AI models. Python, along with
frameworks like TensorFlow (Abadi et al., 2016) and PyTorch (Paszke et al.,
2019), has become integral to AI research, providing comprehensive libraries
for model development, training, and deployment. Domain-specific languages
(DSLs) like Keras for neural network modeling (Gulli & Pal, 2017) and Rasa for
building conversational AI (Bocklisch et al., ) are designed to simplify AI
programming and make it accessible to a broader audience. Code language models
(Chen et al., 2021; Nijkamp et al., 2022; Roziere et al., 2023) have enabled
AI to automatically generate code that can solve complex tasks, allowing for a
deeper integration between programming languages and AI. In the AI agent
space, Flows Josifoski et al. (2023) is conceptual framework that facilitates
structured reasoning and collaboration among AI systems through modular,
message-based interactions. DSPyKhattab et al. (2023) is a programming model
that abstracts language model pipelines as text transformation graphs.Jojic et
al. (2023) demonstrate the Iterations by Regimenting Self-Attention technique,
which manipulates the self-attention mechanism in language models to trigger
and control iterative behaviors for executing algorithmic tasks. Building on
the integration of natural language with programming, the AIOS Compiler (Xu et
al., 2024) is a system that leverages LLMs to interpret and execute
instructions in natural language. AutoGRAMS integrates programming languages
and language models by combining graphically represented agent nodes with
graphically represented program nodes. computer programs essentially model
flowcharts (Goldstine & von Neumann, 1947), where each node in the flow chart
executes and instruction and each transition is facilitate by a branch. In
AutoGRAMS, the idea of using a flow chart for programming is combined with the
idea of using a flowchart to model multi-step language model interactions. The
definition of explicit nodes, which can represent conversational states, and
transitions between nodes make AutoGRAMS especially well suited for
conversational AI applications Transition operations can optionally be
governed by multiple choice questions for a language model, and instructions
can optionally be governed by prompts to a language model, allowing language
models to be deeply embedded in the execution of a program. The visual
programming component of an autogram also gives added interpretability, as any
autogram can easily be visualized as a graph.

AutoGRAMS is motivated by previous work showing that language models can
reason by breaking difficult problems into smaller steps. Chain of thought
prompting (Wei et al., 2022) showed that prompting LLMs to generate reasoning
steps can significantly improve their problem-solving performance of large
language models on complex reasoning tasks. Kojima et al. (2022) further show
how a simple prompting strategy, ”Let’s think step by step,” can significantly
enhance zero-shot reasoning across various complex tasks. Tree of Thoughts
(Yao et al., 2024) and Graph of Thoughts framework (Besta et al., 2024) models
allow for more complex dependencies of thought units. Demonstrate-Search-
Predict (Khattab et al., 2022) incorporates sophisticated interaction
pipelines between language models and retrieval models. ReAct (Yao et al.,
2022) integrates reasoning and action generation in language models. AutoGRAMS
gives the flexibility for the designer, which could itself be an autogram, to
design specific reasoning steps using thought-type nodes to help the language
model solve complex programs. A graph of thought type nodes in combination
with multiple choice transitions allows an autogram to represent highly
complex thought processes that can be used to solve problems.

Like many previous works, AutoGRAMS enables the ability to incorporate
external knowledge tools into an LLM, as well allowing for the LLM to make
decisions. Retrieval-Augmented Generation (RAG) models Lewis et al. (2020),
allow models to access non-parametric memory systems. Toolformer (Schick et
al., 2024) is a model that autonomously learns to interact with external APIs
. The Automatic Reasoning and Tool-use (ART) framework Paranjape et al. (2023)
automates the generation of intermediate reasoning steps and integrates
external tool use. AutoGRAMS gives a large amount of flexibility in how
external tools are used. Python function nodes can specify the use of an
external tool if they call a Python API that applies that tool. The arguments
to the Python API can be variables that were set by a language model or
another Python API at a previous turn. The node-specific transitions and
transition multiple choice questions of AutoGRAMS give additional control over
decisions about which tools are selected for use.

AutoGRAMS relates to previous works that use LLMs to engineer prompts that
control the execution of tasks.Automatic Prompt Engineer (Zhou et al., 2022)
automates the generation and optimization of instructions for LLMs and
systematically refines prompts through an LLM-driven search and evaluation
process. Active-Prompt (Diao et al., 2023) uses uncertainty metrics within an
active learning framework to optimize chain-of-thought reasoning prompts for
LLMs. Promptbreeder (Fernando et al., 2023) uses self-referential self-
improvement to evolve and adapt prompts for specific tasks. Likewise-the
prompts in AutoGRAMS are also able to result in actions that lead to the
modification of an autogram. AutoGRAMS allows any part of an autogram to be
modified by the autogram itself, not just the prompts.

Previous approaches have considered meta-learning (Hochreiter et al., 2001;
Finn et al., 2017), learning to learn, or meta-programming. Meta-genetic
programming (Schmidhuber, 1987) relates to our motivation of developing
program modifying programs. Unsupervised meta-learning (Metz et al., 2018)
relates to the long term vision of of AutoGRAMS to develop agents that can
learn from interactions with their environment in unsupervised ways. Shinn et
al. (2024) considers agents that self-reflect on their own mistakes to learn.
Self-Refine Madaan et al. (2023) leverages iterative feedback by enabling an
LLM to act as both the generator and refiner. MetaGPT Hong et al. (2023) is
meta-programming framework that enables LLM-based multi-agent collaborations
by embedding standardized operating procedures into prompts. While AutoGRAMS
itself is a programming language rather than a specific algorithm, its
features such the flexibility to fully self-modify, the graphical structure,
Turing completeness, and easy integration of language models for executing
instructions and determining transitions, make AutoGRAMS for well-suited meta-
programming applications. We demonstrate an autogram can be used to design an
autogram chatbot from a prompt. AutoGRAMS gives the potential for the
development of highly complex meta-agents and self-modifying agents by
enabling autograms to design new AutoGRAMS graphs or modify their own graphs.

AutoGRAMS also relates to other approaches for controlling text generation.
Previous work has used discriminators (Dathathri et al., 2019; Krause et al.,
2020; Yang & Klein, 2021; Qin et al., 2022), special input tokens (Keskar et
al., 2019), prompts (Reynolds & McDonell, 2021), instruction following (Ouyang
et al., 2022; Longpre et al., 2023) or finetuning using a reward signal
(Ziegler et al., 2019; Rafailov et al., 2024) to better constrain the outputs
of language models for improved controllability and safety. AutoGRAMS allows a
complex network of prompts to be defined to control multi-turn or multi-step
interactions.

##  7 Discussion

This work presented the AutoGRAMS, a novel framework and high level
programming language for controlling multi-step interactions with an LLM.
AutoGRAMS gives the designer the ability to create branched multi-step
interactions with an LLM in a way that is analogous to how programming
languages allow for the design of branched multi-step interactions with a
processor. The designer can define these steps of the interaction in advance
by writing prompts for a language model at each step. AutoGRAMS also allows
language models to exert direct control over transitions in the interaction by
enabling the designer to write multiple choice questions that determine which
branch to take. This structure enables the designer to use natural language to
effectively govern the flow of the of steps that an AutoGRAMS takes. The
graphical structure of an autogram allows it to be implemented and visualized
in an intuitive way.

AutoGRAMS is especially useful for designing complex conversational flows.
AutoGRAMS allows for agents be designed to behave in more predictable ways
over longer conversations, giving added control-ability and safety to
conversational agents. It accomplishes this by allowing the designer to define
a prompt for how to reply to the user for every possible turn in the
conversation in advance. The designer has to anticipate the users response,
for which The designer can then also design a series of multiple choice
questions for a language model that govern which turns will actually be
implemented based on that response. Additionally, if the conversational agent
needs more complex intermediate reasoning steps or memory, these can be
implemented in AutoGRAMS as well. The full state of the autogram can be saved
in a serializable object, making it easy to reload the complete state and
memory of a complex conversational agent from a database.

By allowing conditionals and code statements, and variables, AutoGRAMS allows
the functionality of a Turing complete programming language to be integrated
into the graphs that define the steps of the language model interaction. We
demonstrate how Python programs can be mapped to an AutoGRAMS graph of python-
type nodes and embedded with other types of nodes. This enables the designer
to define data structures to store and access memories, which could be text
generated by a language model or variables created in code. These variables
can be embedded into prompts, transition questions, referenced in code, or
even used to create dynamically executing code or dynamic transitions in the
graph. This high level of flexibility makes it possible for AutoGRAMS to be
used for very advanced applications of AI agents.

We also demonstrate mechanisms for AutoGRAMS to design other AutoGRAMS. We
also demonstrate their ability for self-adaptation and modification. AutoGRAMS
allows an agent to be fully self-referential with respect to the nodes that
define the agent. We anticipate self-modifying AI agents to be useful for
generalizing agents to more situations and allowing agents to learn from
experiences and information, and AutoGRAMS provides a useful framework to
build such agents.

#### Acknowledgments

The authors thank Subu Subramaniam for helpful discussions on how autograms
could be applied and for helping to test the AutoGRAMS repository.

## References

  * Abadi et al. (2016) Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, et al.  $\\{$TensorFlow$\\}$: a system for $\\{$Large-Scale$\\}$ machine learning.  In _12th USENIX symposium on operating systems design and implementation (OSDI 16)_ , pp. 265–283, 2016.
  * Adiwardana et al. (2020) Daniel Adiwardana, Minh-Thang Luong, David R So, Jamie Hall, Noah Fiedel, Romal Thoppilan, Zi Yang, Apoorv Kulshreshtha, Gaurav Nemade, Yifeng Lu, et al.  Towards a human-like open-domain chatbot.  _arXiv preprint arXiv:2001.09977_ , 2020.
  * Bengio et al. (2000) Yoshua Bengio, Réjean Ducharme, and Pascal Vincent.  A neural probabilistic language model.  _Advances in neural information processing systems_ , 13, 2000.
  * Besta et al. (2024) Maciej Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger, Michal Podstawski, Lukas Gianinazzi, Joanna Gajda, Tomasz Lehmann, Hubert Niewiadomski, Piotr Nyczyk, et al.  Graph of thoughts: Solving elaborate problems with large language models.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 38, pp. 17682–17690, 2024.
  * (5) Tom Bocklisch, Joey Faulkner, Nick Pawlowski, and Alan Nichol.  Rasa: Open-source language understanding and dialogue.
  * Brown et al. (2020) Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al.  Language models are few-shot learners.  _Advances in neural information processing systems_ , 33:1877–1901, 2020.
  * Chen et al. (2021) Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al.  Evaluating large language models trained on code.  _arXiv preprint arXiv:2107.03374_ , 2021.
  * Dathathri et al. (2019) Sumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung, Eric Frank, Piero Molino, Jason Yosinski, and Rosanne Liu.  Plug and play language models: A simple approach to controlled text generation.  _arXiv preprint arXiv:1912.02164_ , 2019.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In Jill Burstein, Christy Doran, and Thamar Solorio (eds.), _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pp. 4171–4186, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics.  doi: 10.18653/v1/N19-1423.  URL https://aclanthology.org/N19-1423.
  * Diao et al. (2023) Shizhe Diao, Pengcheng Wang, Yong Lin, and Tong Zhang.  Active prompting with chain-of-thought for large language models.  _arXiv preprint arXiv:2302.12246v4_ , 2 2023.  URL http://arxiv.org/pdf/2302.12246v4.
  * Fernando et al. (2023) Chrisantha Fernando, Dylan Banarse, Henryk Michalewski, Simon Osindero, and Tim Rocktäschel.  Promptbreeder: Self-referential self-improvement via prompt evolution.  _arXiv preprint arXiv:2309.16797v1_ , 9 2023.  URL http://arxiv.org/pdf/2309.16797v1.
  * Finn et al. (2017) Chelsea Finn, Pieter Abbeel, and Sergey Levine.  Model-agnostic meta-learning for fast adaptation of deep networks.  In _International conference on machine learning_ , pp. 1126–1135. PMLR, 2017.
  * Goldstine & von Neumann (1947) Herman H. Goldstine and John von Neumann.  Planning and coding of problems for an electronic computing instrument.  Technical Report Part II, Volume 1, Institute for Advanced Study, Princeton, NJ, 1947.
  * Gulli & Pal (2017) Antonio Gulli and Sujit Pal.  _Deep learning with Keras_.  Packt Publishing Ltd, 2017.
  * Hochreiter et al. (2001) Sepp Hochreiter, A Steven Younger, and Peter R Conwell.  Learning to learn using gradient descent.  In _Artificial Neural Networks—ICANN 2001: International Conference Vienna, Austria, August 21–25, 2001 Proceedings 11_ , pp. 87–94. Springer, 2001.
  * Hong et al. (2023) Sirui Hong, Mingchen Zhuge, Jonathan Chen, Xiawu Zheng, Yuheng Cheng, Ceyao Zhang, Jinlin Wang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, Chenyu Ran, Lingfeng Xiao, Chenglin Wu, and Jürgen Schmidhuber.  Metagpt: Meta programming for a multi-agent collaborative framework.  _arXiv preprint arXiv:2308.00352v5_ , 8 2023.  URL http://arxiv.org/pdf/2308.00352v5.
  * Jojic et al. (2023) Ana Jojic, Zhen Wang, and Nebojsa Jojic.  Gpt is becoming a turing machine: Here are some ways to program it.  _arXiv preprint arXiv:2303.14310v1_ , 3 2023.  URL http://arxiv.org/pdf/2303.14310v1.
  * Josifoski et al. (2023) Martin Josifoski, Lars Klein, Maxime Peyrard, Nicolas Baldwin, Yifei Li, Saibo Geng, Julian Paul Schnitzler, Yuxing Yao, Jiheng Wei, Debjit Paul, and Robert West.  Flows: Building blocks of reasoning and collaborating ai.  _arXiv preprint arXiv:2308.01285v3_ , 8 2023.  URL http://arxiv.org/pdf/2308.01285v3.
  * Keskar et al. (2019) Nitish Shirish Keskar, Bryan McCann, Lav R Varshney, Caiming Xiong, and Richard Socher.  Ctrl: A conditional transformer language model for controllable generation.  _arXiv preprint arXiv:1909.05858_ , 2019.
  * Khattab et al. (2022) Omar Khattab, Keshav Santhanam, Xiang Lisa Li, David Hall, Percy Liang, Christopher Potts, and Matei Zaharia.  Demonstrate-search-predict: Composing retrieval and language models for knowledge-intensive nlp.  _arXiv preprint arXiv:2212.14024v2_ , 12 2022.  URL http://arxiv.org/pdf/2212.14024v2.
  * Khattab et al. (2023) Omar Khattab, Arnav Singhvi, Paridhi Maheshwari, Zhiyuan Zhang, Keshav Santhanam, Sri Vardhamanan, Saiful Haq, Ashutosh Sharma, Thomas T. Joshi, Hanna Moazam, Heather Miller, Matei Zaharia, and Christopher Potts.  Dspy: Compiling declarative language model calls into self-improving pipelines.  _arXiv preprint arXiv:2310.03714v1_ , 10 2023.  URL http://arxiv.org/pdf/2310.03714v1.
  * Kojima et al. (2022) Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa.  Large language models are zero-shot reasoners.  _arXiv preprint arXiv:2205.11916v4_ , 5 2022.  URL http://arxiv.org/pdf/2205.11916v4.
  * Krause et al. (2020) Ben Krause, Akhilesh Deepak Gotmare, Bryan McCann, Nitish Shirish Keskar, Shafiq Joty, Richard Socher, and Nazneen Fatema Rajani.  Gedi: Generative discriminator guided sequence generation.  _arXiv preprint arXiv:2009.06367_ , 2020.
  * Lewis et al. (2020) Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela.  Retrieval-augmented generation for knowledge-intensive nlp tasks.  _arXiv preprint arXiv:2005.11401v4_ , 5 2020.  URL http://arxiv.org/pdf/2005.11401v4.
  * Longpre et al. (2023) Shayne Longpre, Le Hou, Tu Vu, Albert Webson, Hyung Won Chung, Yi Tay, Denny Zhou, Quoc V Le, Barret Zoph, Jason Wei, et al.  The flan collection: Designing data and methods for effective instruction tuning.  In _International Conference on Machine Learning_ , pp. 22631–22648. PMLR, 2023.
  * Madaan et al. (2023) Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark.  Self-refine: Iterative refinement with self-feedback.  _arXiv preprint arXiv:2303.17651v2_ , 3 2023.  URL http://arxiv.org/pdf/2303.17651v2.
  * Metz et al. (2018) Luke Metz, Niru Maheswaranathan, Brian Cheung, and Jascha Sohl-Dickstein.  Meta-learning update rules for unsupervised representation learning.  _arXiv preprint arXiv:1804.00222_ , 2018.
  * Nijkamp et al. (2022) Erik Nijkamp, Bo Pang, Hiroaki Hayashi, Lifu Tu, Huan Wang, Yingbo Zhou, Silvio Savarese, and Caiming Xiong.  Codegen: An open large language model for code with multi-turn program synthesis.  _arXiv preprint arXiv:2203.13474_ , 2022.
  * Ouyang et al. (2022) Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al.  Training language models to follow instructions with human feedback.  _Advances in neural information processing systems_ , 35:27730–27744, 2022.
  * Paranjape et al. (2023) Bhargavi Paranjape, Scott Lundberg, Sameer Singh, Hannaneh Hajishirzi, Luke Zettlemoyer, and Marco Tulio Ribeiro.  Art: Automatic multi-step reasoning and tool-use for large language models.  _arXiv preprint arXiv:2303.09014v1_ , 3 2023.  URL http://arxiv.org/pdf/2303.09014v1.
  * Paszke et al. (2019) Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al.  Pytorch: An imperative style, high-performance deep learning library.  _Advances in neural information processing systems_ , 32, 2019.
  * Qin et al. (2022) Lianhui Qin, Sean Welleck, Daniel Khashabi, and Yejin Choi.  Cold decoding: Energy-based constrained text generation with langevin dynamics.  _Advances in Neural Information Processing Systems_ , 35:9538–9551, 2022.
  * Radford et al. (2019) Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al.  Language models are unsupervised multitask learners.  _OpenAI blog_ , 1(8):9, 2019.
  * Rafailov et al. (2024) Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn.  Direct preference optimization: Your language model is secretly a reward model.  _Advances in Neural Information Processing Systems_ , 36, 2024.
  * Raffel et al. (2020) Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu.  Exploring the limits of transfer learning with a unified text-to-text transformer.  _Journal of machine learning research_ , 21(140):1–67, 2020.
  * Reynolds & McDonell (2021) Laria Reynolds and Kyle McDonell.  Prompt programming for large language models: Beyond the few-shot paradigm.  In _Extended abstracts of the 2021 CHI conference on human factors in computing systems_ , pp. 1–7, 2021.
  * Roller et al. (2020) Stephen Roller, Emily Dinan, Naman Goyal, Da Ju, Mary Williamson, Yinhan Liu, Jing Xu, Myle Ott, Kurt Shuster, Eric M Smith, et al.  Recipes for building an open-domain chatbot.  _arXiv preprint arXiv:2004.13637_ , 2020.
  * Roziere et al. (2023) Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Tal Remez, Jérémy Rapin, et al.  Code llama: Open foundation models for code.  _arXiv preprint arXiv:2308.12950_ , 2023.
  * Schick et al. (2024) Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom.  Toolformer: Language models can teach themselves to use tools.  _Advances in Neural Information Processing Systems_ , 36, 2024.
  * Schmidhuber (1987) Jürgen Schmidhuber.  _Evolutionary Principles in Self-Referential Learning, or on Learning How to Learn: The Meta-Meta-… Hook_.  PhD thesis, Technische Universität München, 1987.  PhD thesis.
  * Shinn et al. (2024) Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao.  Reflexion: Language agents with verbal reinforcement learning.  _Advances in Neural Information Processing Systems_ , 36, 2024.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin.  Attention is all you need.  _Advances in neural information processing systems_ , 30, 2017.
  * Wei et al. (2022) Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou.  Chain-of-thought prompting elicits reasoning in large language models.  _arXiv preprint arXiv:2201.11903v6_ , 1 2022.  URL http://arxiv.org/pdf/2201.11903v6.
  * Xu et al. (2024) Shuyuan Xu, Zelong Li, Kai Mei, and Yongfeng Zhang.  Aios compiler: Llm as interpreter for natural language programming and flow programming of ai agents.  _arXiv preprint arXiv:2405.06907v2_ , 5 2024.  URL http://arxiv.org/pdf/2405.06907v2.
  * Yang & Klein (2021) Kevin Yang and Dan Klein.  Fudge: Controlled text generation with future discriminators.  _arXiv preprint arXiv:2104.05218_ , 2021.
  * Yao et al. (2022) Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao.  React: Synergizing reasoning and acting in language models.  _arXiv preprint arXiv:2210.03629v3_ , 10 2022.  URL http://arxiv.org/pdf/2210.03629v3.
  * Yao et al. (2024) Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan.  Tree of thoughts: Deliberate problem solving with large language models.  _Advances in Neural Information Processing Systems_ , 36, 2024.
  * Zhang et al. (2019) Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, and Bill Dolan.  Dialogpt: Large-scale generative pre-training for conversational response generation.  _arXiv preprint arXiv:1911.00536_ , 2019.
  * Zhou et al. (2022) Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba.  Large language models are human-level prompt engineers.  _arXiv preprint arXiv:2211.01910v2_ , 11 2022.  URL http://arxiv.org/pdf/2211.01910v2.
  * Ziegler et al. (2019) Daniel M Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving.  Fine-tuning language models from human preferences.  _arXiv preprint arXiv:1909.08593_ , 2019.

##  Appendix A Prompt formation

Language models in AutoGRAMS take in a list of input turns (corresponding to
user turns or instructions), a list of output turns (corresponding to
assistant turns/model outputs), as well as an optional prefix for how the
reply should start (for internal models only).

When preparing the prompt to get a new language model output at either a
thought node or a chat node, the autogram re-iterates over past turns to get
these input and output turns.

For every previously visited node that is within the current scope (depending
functions and function types if used), if that node is a thought node or a
chat node, it adds a turn to the conversation history that the language model
sees. the inputs and outputs are determined as follows:

for each past thought or chat node visited:

  * •

if it is a thought node

    * –

if the thought node came immediately after a user reply, then both the
instruction and user reply will appear in the input for that turn

    * –

otherwise, only the instruction will appear in the input for that turn

  * •

if it is a chat node

    * –

if the chat node came immediately after a user reply, then only user reply
will appear in the input for that turn. The old instruction is discarded for
chat nodes to avoid long prompts. We may later add an option to retain chat
instructions in the autogram config

    * –

otherwise,the chat node’s instruction will appear in the input for that turn

The initial prompt is pre-appended to the first input.

For the last input, corresponding to the end of the prompt, the string is
determined by the instruction template. The default instruction template is:

<last_response> Instruction for <agent_name>: <instruction>, where
<last_response>, <agent_name>, and <instruction> are special place holder
tokens for the instruction, agent name, and last response. Sometimes the last
response will be the empty string if the last user response is already in one
of the past inputs (which will happen if there is at least 1 thought node
before replying with a chat node).

Lastly, replies have a reply start, which is “Agent’s reply:” if the default
start template and agent name are used. Depending on whether the start type is
suffix or prefix, the reply start either appears at the end of the last input,
or at the start of the model’s reply.

##  Appendix B Language models in AutoGRAMS

Every autogram has 3 language models which can share weights or use the same
API. The are referred to as the chatbot, classifier, and userbot. The chatbot
executes the instruction of thought and chat nodes, the classifier predicts
transitions, and the userbot can be used simulate the user in a conversational
autogram.

The Chatbot is the language model that generates the output text of chat and
thought nodes, corresponding to chain of thought and replies to the user. The
chatbot is assumed to be a chat completion style model. As arguments, it takes
a list in inputs (treated as user side inputs) and outputs (treated as
previous model outputs). Depending on the nodes used, some of the inputs may
be instructions or combinations of instructions and user replies, instead of
just user replies.

The classifier model is a language model that predicts the answers to
transition questions, and is used to predict transitions. The classifier model
only needs to generate a single token, and uses a large positive logit bias on
the allowable answer choices (which are always multiple choice A-Z, or yes/no)
to ensure the model generates a valid answer. Without a logit bias, the model
would likely sometimes generate things like “The answer is C.” instead of just
“C”, which would make determining the predicted answer more difficult. Unlike
the chatbot, which receives previous inputs and output turns, the conversation
history to the classifier is all concatenated to a single turn.

The userbot of an autogram is by default set to be the same object as the
chatbot, meaning that it will share the same API or the same weights if
running on the local machine. The prompts to the userbot are what distinguish
its behavior, since the instructions will typically be along the lines of
“reply as the user and say x”

##  Appendix C Interjection Nodes

“Interjection nodes” are nodes that can be reached after any chat node. They
are mostly meant for scenarios where the user does or says something
unexpected and it may be necessary to leave the main conversational
trajectory. Interjection nodes can be any action type–so a thought node could
also be an interjection node. If interjection nodes are used, an additional
multiple choice question will be applied after every chat node before applying
the transition question. The autogram may override the main transition
behavior depending on the answer predicted to that question.

Any node can be made an interjection node by defining a
$condition\\_interjection$ field when making a node. The default interjection
question is “Which of the following is True?”. The last answer, by default, is
“None of the above.”. The other answer choices correspond to the
$condition\\_interjection$ set by each of the interjection nodes. These
defaults can be changed in the Autogram Config. If any interjection nodes are
defined, the classifier will predict the answer to the interjection question
after every conversation turn. If the classifier predicts that answer is the
default (“none of the above”), the autogram will proceed as normal to ask the
language model the transition question. However, if the classifier predicts an
answer that corresponds to a $condition\\_interjection$, the normal transition
behavior will be overridden and the autogram will jump to the interjection
node and execute that next. It is important to define the answers for
interjection transitions carefully, since if the model incorrectly predicts
and interjection transition it can lead to large breaks in the conversational
flow.

It would be possible to obtain a similar result to interjection nodes by using
additional transition nodes after every chat node. The chat node could ask the
interjection question as its transition question and map the answers to the
interjection nodes. An additional transition node could be added for when the
answer is the default or “none of the above”, and this transition node could
ask the original transition question of the chat node and map to the original
transitions. Interjection nodes are essentially a shortcut to achieve this
result more conveniently without needing to define an additional transition
node after every chat node.

##  Appendix D Simulating a user in conversational autograms

We built the ability to simulate the user’s end into conversational autograms.
For nodes with multiple transitions, a list of user prompts corresponding to
each transition can be provided. The autogram can then be used to simulate
entire conversations, using the reply method to simulate the agents end, and
the simulate_user method to reply automatically on the users behalf. Simulate
user samples a transition at random and then generates text using the user
prompt corresponding to the sampled transition. This simulation ability has
several potential usecases. Firstly, it allows a conversational autogram to
quickly be tested by generating full conversations automatically that can be
inspected to see if the agent is replying in the desired way. Secondly, it can
act as a check to be sure the classifier language model used for transitions
is predicting the correct output. If the user simulation and classifier both
work correctly, then the state sampled by the user simulation should match the
transition predicted by the classifier. Lastly, we envision user simulation to
be useful for finetuning the underlying language models from human or AI
feedback–since it allows for full trajectories to be generated without the
need for human input at every step.

##  Appendix E Wild card transition visualization

We give the flow chart of nodes to illustrate can wildcard transitions can use
if/else-if/else logic to facilitate transitions in figure 11.

Figure 11: Depiction of how wildcard transitions can be used to implement
if/else-if/else logic in AutoGRAMS. Any transition ending in “.*”, such as
$node\\_cond.*$ in the example above, is interpreted as a transition to one of
several possible nodes that have the same prefix but a alphabet character as
the ending. In the example above, $node\\_cond.*$ can transition to
$node\\_cond.a$,$node\\_cond.b$,and $node\\_cond.c$, and the node selected
will depend on the $boolean\\_condition$ attribute set for these nodes. The
boolean condition is evaluated in Python and can include variables visible at
the current scope in the autogram. The nodes will be considered in
alphabetical order; if the boolean condition for $node\\_cond.a$ is true,
$node\\_cond.a$ will be selected. Otherwise the boolean condition for
$node\\_cond.b$ will be evaluated, and if its true, $node\\_cond.b$ will be
selected. If the boolean condition for both $a$ and $b$ are false, then $c$
will be selected. The last node in alphabetical order does not need a boolean
condition. This process allows.

##  Appendix F Additional details of the AutoGRAMS graph compiler

The AutoGRAMS compiler translates code with python-like syntax into a
representation that the AutoGRAMS interpreter can understand. It allows for
Python statements, conditionals, loops, variable assignments, and function
calls, to be combined with special $exec\\_node()$ statements that create
AutoGRAMS nodes at specific points in the program. T AutoGRAMS compiler
processes code down to the level of individual nodes and statements, and to
form an AutoGRAMS graph around these statements. It uses the Python abstract
syntax tree (AST) to parse the code, and recursively traverses the AST until
it reaches code that can be represented as a single statement. It then uses
this statement to define a new AutoGRAMS node. If the statement is a Python
statement, the compiler will define a python-type node, whereas if it is a
special $exec\\_node()$ statement it will be a node type defined by the
arguments given to $exec\\_node()$

The high level steps implemented by the AutoGRAMS compiler are:

  1. 1.

Convert code to Python AST representation

  2. 2.

Extract the name of every function definition in the code and the subtree of
the AST associated with that function

  3. 3.

Recursively traverse the subtree of the AST for each function to get the
AutoGRAMS graph for that function

  4. 4.

Recursively traverse the AST for code that is outside of a function to get the
AutoGRAMS graph of the outer scope/main program

Any nodes that aren’t explicitly named by exec_node statements need to be
named automatically by the AutoGRAMS compiler. Node names need to be unique,
so the compiler assigns names with prefixes based on functions and statement
bodies that they reside in, and using a numbering system that increments for
each statement type and node within each prefix. All automatically named names
start with the’_’ character to reduce the risk of conflicts between nodes
named by the designer and nodes named automatically by the compiler.

As we define the operations performed during AST traversal, we will use “AST
node” to refer to a node in the AST tree and “AutoGRAMS node” to refer to e
anode in the AutoGRAMS graph. The AutoGRAMS compile recursively traverses the
AST tree, starting at the root AST node, and recursively traversing the
subtree of the children AST nodes.

When encountering an AST node representing a conditional (if/else-if/else),
the compiler creates a new independent chain object for each branch to
recursively processes each branch of the conditional. Each chain returns with
the AutoGRAMS nodes needed to implement that branch. The subgraphs in each
branch are then combined by defining new AutoGRAMS transition nodes at the
root of the conditional, and at the start of each branch of the conditional.
The AutoGRAMS node at the root of the conditional is given an automatically
defined wildcard style transition, and the AutoGRAMS nodes at the start of
each branch are given names that match the name defined in the wildcard style
transition, and a alphabet character suffix corresponding to their order in
the branch logic. These AutoGRAMS nodes are also defined with boolean
conditions that match the boolean condition defined in the Python conditional
statement. This combination of a wild card transition, node names, and boolean
conditions allows them to implement the exact transition logic of the if/else
statement defined in Python.

When encountering an AST node corresponding to a while loop, the compiler
first collects the condition or the while loop. It then creates a new
independent chain object that recursively processes the body of the while loop
to obtain the AutoGRAMS nodes in the body.The compiler can then follow the
process to design conditionals described above, and can connect the final
AutoGRAMS node in the body of the while loop with root AutoGRAMS node of the
conditional, allowing the loop of the graph to be repeatedly traversed until
the exit condition is met.

When encountering an AST node corresponding to a while loop, the compiler can
execute a similar process to a while loop, while also creating an AutoGrams
node to make node before the loop to define an iterator variable, as well as
AutoGrams nodes in the forloop that extracts elements from the iterable object
defined in the forloop, and increment the iterator variable. The forloop is
given an exit condition that causes it to exit the graph when the iterator
variable is equal to the length of the iterable object.

When encountering AST nodes corresponding to single Python statements, the
autograms compiler adds a python-type $python\\_function$ node to cause that
statement to be executed at that point in the graph. If the statement contains
a varaible assignment, this is also added to the instruction off the
corresponding AutoGRAMS node.

When encountering $exec\\_node$ statements, the autograms compiler defines a
new node with the arguments given in the statement. If the exec node statement
contains a pythnon style variable assignment, this assignment is added to the
node’s instruction string. If the $exec\\_node$ statement contains no defined
transitions, it is assumed that it will transition to the next node
encountered in the code. It is also possible to connect an $exec\\_node$
statement with nodes defined by other $exec\\_node$ statements, however
jumping into a loop or function from an external point will lead to errors.

##  Appendix G The reply algorithm

We present high-level pseudo-code to show the operations performed by the
AutoGRAMS reply method to get a conversational response and updated memory
from an autogram.

Algorithm 2 autogram.reply() method. Some arguments to functions are omitted
for simplicity.

1:function $reply$($user\\_reply,memory\\_object$,$new\\_node\\_id$)

2: $response\\_to\\_user\leftarrow{False}$

3: while $!({response\\_to\\_user})$ do

4: if $new\\_node\\_id\,\,is\,\,None$ then

5:$\triangleright$ get output of previously executed node

6: $variable\\_output\leftarrow node.get\\_variable\\_output(memory\\_object)$

7:$\triangleright$ assign variables in memory

8: $memory\\_object.assign\\_variables(variable\\_output)$

9:$\triangleright$ apply transition function

10: $new\\_node\\_id\leftarrow
node.apply\\_transition(user\\_reply,memory\\_object)$

11:$\triangleright$ process transition string returned by transition function

12: $new\\_node\\_id\leftarrow
process\\_node\\_id(new\\_node\\_id,memory\\_object)$ $\triangleright$ select
new node

13: $node\leftarrow self.nodes[new\\_node\\_id]$

14:$\triangleright$ executes node’s function, which will call a language model
for chat or thought style nodes

15: $response,user\\_reply,response\\_to\\_user\leftarrow$

16: $node.apply\\_instruction(user\\_reply,memory\\_object,self.chatbot)$

17: $\textbf{return}\,\,response,memory\\_object$

When only using chat style nodes with only standard transitions, the algorithm
for replying to a user becomes simplified, allowing it to be more fully
explained in a single block of pseudo-code. This pseudo-code assumes that it
is not the first step in the conversation (if the model goes first, it will
reply from a predetermined node).

Algorithm 3 A simplified pseudo-code for reply algorithm with only chat-style
nodes

1:function $reply$($user\\_reply$,$conv\\_history$,$old\\_node\\_id$)

2: $node\leftarrow nodes[old\\_node\\_id]$

3: $conv\\_hisory.add\\_user\\_reply(user\\_reply)$

4: if $len(node.transition\\_choices)>1$ then

5: $transition\\_question\leftarrow node.transition\\_question$

6: $transition\\_choices\leftarrow node.transition\\_choices$ $\triangleright$
Format as a multiple choice question of the form “Question A. choice 1 B.
choice 2 C. choice 3” etc.

7: $mc\\_question\leftarrow
form\\_mc\\_prompt(transition\\_question,transition\\_choices)$

8: $classifier\\_prompt\leftarrow
form\\_classifier\\_prompt(conv\\_history,mc\\_question)$ $\triangleright$
classifier is a language model that predicts probability distribution over a
single next token. It will predict A, B, C etc. depending on number of
transition choices.

9: $next\\_token\\_probs\leftarrow
classifier\\_lm.predict\\_next\\_token(classifier\\_prompt)$ $\triangleright$
get token ids corresponding to capital letters that model language will
predict for answer choice

10: $mc\\_indices\leftarrow ABCD\\_TOKENS[:len(transition\\_choices)]$
$\triangleright$ only select probabilities corresponding to allowable
predictions

11: $abcd\\_probs\leftarrow next\\_token\\_probs[mc\\_indices]$
$\triangleright$ get the answer choice the language model thinks is most
likely

12: $answer\\_pred\leftarrow argmax(abcd\\_probs)$ $\triangleright$ select the
next node to visit

13: $new\\_node\\_id\leftarrow node.transitions[answer\\_pred]$

14: else

15: $new\\_node\\_id\leftarrow node.transitions[0]$

16: $node\leftarrow nodes[new\\_node\\_id]$

17: $chatbot\\_prompt\leftarrow
form\\_chatbot\\_prompt(conv\\_history,node.instruction)$

18:$\triangleright$ apply chatbot to generate response

19: $agent\\_reply\leftarrow chatbot\\_lm(chatbot\\_prompt)$

20: $conv\\_history.add\\_agent\\_reply(agent\\_reply)$

21: $response\leftarrow node.apply\\_instruction(user\\_reply,self.chatbot)$

22: $\textbf{return}\,\,response,conv\\_history,new\\_node\\_id$

##  Appendix H A simple self-referential autogram

We demonstrate a very simple proof-of-concept of a self-referential autogram
below. This autogram creates a new node at every conversation turn to handle
the user’s reply. the autogram was implemented in AutoGRAMS compiled from
Python and automatically compiled into AutoGRAMS nodes. The fields for the
nodes that make up the autogram are given below.

The first node sets a variable to hold an instruction we will use later.

    name: "_node1"
    action: "python_function"
    transitions: [’intro’]
    instruction: "meta_inst = \"We need to decide how to
       deal with the last user reply.
       Rather than replying directly,
       write an instruction for another language model
       to reply.The instruction could be
       along the lines of
       ’respond to the user and tell them xyz’\""

the next node initiates a conversation turn with the user with the exact text
”Hi there, What can i help you with?”.

    name: "intro"
    action: "chat_exact"
    instruction: "Hi there, What can i help you with?"
    transitions: [’dynamic_node’]

The next node calls a function that will do the majority of the heavy lifting.
The function add_new_node() will add a new node to the autogram.
add_new_node() also returns the string of the next node, which will be saved
in the variable next_node. Since transitions are executed after instructions,
the transition $next_node will cause the autogram to transition to the node
defined by the $add\\_new\\_node$ function.

    name: "dynamic_node"
    action: "function"
    transitions: [’$next_node’]
    instruction: "next_node=add_new_node(meta_inst)"

The next node defines a callable node that starts the add_new_node function,
with the name field defining the function name and arguments.

    name: add_new_node(meta_inst)
    action: transition
    transitions: [’_add_new_node_node2’]

The next node is a thought node that uses the language model to generate an
instruction for the new node are are designing. It conditions on the variable
meta_inst as a prompt. The resulting generated text is saved in a new variable
called new_inst.

    name: _add_new_node_node2
    action: thought
    instruction: new_inst=$meta_inst
    transitions: [’_add_new_node_node3’]

The next node is a thought node that uses the language model to name the new
node being created. It conditions on the new instruction generated at the
previous node, which is embeded in the prompt. The result is saved in a
variable called new_node

    name: _add_new_node_node3
    action: thought
    instruction: new_name=We need to come up with a short
         name for a node that executes this instruction: $new_inst
         The name must be all lower case letters and underscore (’_’).
         Reply with the name only with no spaces
    transitions: [’_add_new_node_whileloop1_start’]

the next node is an automatically generated node to facilitate the start of a
while loop by applying a wildcard transition to check the exit conditions,
which will be set in the boolean_condition attributes of 2 later nodes

    name: _add_new_node_whileloop1_start
    action: transition
    transitions: [’_add_new_node_whileloop1.*’]

The first condition referenced by the wildcard transition. This will enter the
loop if the node name we have defined not valid.

    name: _add_new_node_whileloop1.a
    action: transition
    transitions: [’_add_new_node_whileloop1_node1’]
    boolean_condition: not meta_utils.check_node_name(new_name)

This node regenerates node name if node name from before was invalid.

    name: _add_new_node_whileloop1_node1
    action: thought
    instruction: new_name=We need to come up with a short
         name for a node that executes this instruction: $new_inst
         The name must be all lower case letters and underscore (’_’).
         The last name was invalid, please be sure to reply with
         the name only and obey the rules about characters.
    transitions: [’_add_new_node_whileloop1.*’]

this node is to exit the while loop when the boolean condition set in
_add_new_node_whileloop1.a is False.

    name: _add_new_node_whileloop1.b
    action: transition
    transitions: [’_add_new_node_node8’]

This node adds the node that we just defined the arguments for to the autogram
interpreter object that is executing the graph, which is referenced using
self. We set the action of this node to chat, since we will use it to give
conversational replies. Notice that it transitions back to dynamic_node which
we defined earlier. This ensures that the autogram will continue–after the
generated node is executed, the autogram will transition back to dynamic_node
and design another node to handle the new conversation turn.

    name: _add_new_node_node8
    action: python_function
    instruction: self.add_node(action=’chat’,
         instruction=new_inst,
         name=new_name,
         transitions=[’dynamic_node’])
    transitions: [’_add_new_node_node9’]

Lastly, a node with a return transition returns the new_name variable. This
returns the graph to dynamic_node, which called the function.

    name: _add_new_node_node9
    action: transition
    transitions: [’return new_name’]

After the function returns, dynamic_node will then transition to the new node
we just defined (since it’s name is saved in the new_name variable and
dynamic_node uses $new_name as its only transition. The new node will execute
to give a conversational reply, and then transition back to dynamic_node,
which will call the function again to design another node.

Figure 12: AutoGRAMS compiled from Python code for self-referential autogram
described in Appendix H. The autogram implements a function that designs a new
node, and modifies its own representation with the self.add_node() method. The
name of the new node is returned and set as the transition. The newly
designing node gives a conversational reply, and transitions back to the node-
designing node, allowing the autogram to design a new node at each
conversation turn.