transforming networks into some or all of the elements of a series of sequential decision problems allows a user to increase the overall value of the network system . the value of the network system is based on the current and anticipated benefits of the network , given the operator &# 39 ; s ability to control it now and in the future . the benefits to the operator , who may be a manger , a network operator , a business owner or anyone with control over a network or input into the management of a network , can be defined in terms of revenue or profit from visitor activity on the site ( such as for a website ); throughput or accident - free trips through a traffic or logistics network ; or successful routes or other visitor activity through another type of network . the disclosed system and method may be implemented by a variety of computing systems ( fig1 ). the computing system has a computer 102 with a processor 103 , an input device 100 , an output device 101 , a hard drive 104 , a controlled network 105 and is connected to another network ( here shown as the internet 106 . parts of the computer may be located or stored remotely as shown in remote parts 107 . the computer 102 is capable of executing computer software ( not shown ) that programs the computer to receive information on a network , compose a sequential decision problem or set of sequential decision problems ( optionally transforming the network information into directed graphs ), validate the problem ( s ) by performing convergence and error checks , solve the problem ( s ), and output the resulting solution ( s ) to the user . the computer 102 may be a personal computer , laptop , desktop , smart phone , tablet , personal digital assistant , a networked server or any other similar device . the computer software may be stored on a hard drive , or other local storage devices ( not shown ) such as a solid state drive , magnetic tape or optical storage . additionally , parts of the computer system may be located remotely . for example , the computer software for programming the computing system may be remotely stored in remote storage device and accessed over a network . the network may be a local network , a wide area network , a virtual private network , the internet or any combination of linked devices . for example , the computing system may be on a local network connected to the internet connected to another local network connected to the remote storage device . the network has elements that are controlled by the operator ( or the processor ), but may connect to other elements that are not controllable by the operator ( or the processor ). for example , the network displays several webpages that the operator may change the content of , the links between the pages or the number of pages , and the network may also connect to the internet which the operator does not control . the processor 103 is any computer processor capable of executing computer software . the processor does not refer specifically to a central processing unit , but may also refer to combinations of central processing units and graphics processing units or other combinations of electronic devices capable of processing information according to instructions provided by computer software . additionally , the computing system need not be connected to the internet , but may be entirely self contained in a local system or network . for example , the computer system could be distributed across a network of traffic lights each light containing a small processor optionally attached to one or more central processors with processor 103 standing in for the distributed computing power of all the processors . the input device 100 inputs information into the computing system from the operator ( not shown ) or from other sources ( such as inputs provided over the network ). the input device 100 could be a mouse and keyboard , data port or observation and processing hardware or various combinations of such devices . furthermore , the input device 100 may be a mouse , a keyboard , voice control , gesture or image recognition or any other device capable of inputting information to the computer system . in a network optimization example concerning traffic throughout a city the input device 100 could include a mouse and keyboard allowing operator input and weight sensors and traffic cameras showing the location of cars . in a website optimization setting the input device 100 could include data collection about visitor mouse location , visitor duration on various webpages included in the website or phrases searched by the visitor that lead the visitor to the website . the output device 101 outputs information from the computing system to the operator . the input device 100 and the output device 101 may be the same device . for example , a touchscreen monitor may be the input device 100 and the output device 101 , as the touchscreen monitor is capable of displaying and receiving information . the output device 101 may be a computer screen , phone screen , monitor , television , printer , traffic signal or any other device capable of outputting information from the computer system . for example , the output device 101 could include traffic lights signaling cars , walk signs signaling safe pedestrian crossing times and hard - drive storage of all output signals and past decision advice . in another example , the output device 101 could include a server displaying a website on the internet to visitors . the disclosed system and method can be stored or provided as computer software to the operator on a variety of computer - readable media , such as hard drives , solid state drives , optical discs , or any other computer - readable medium capable of storing instructions for the processor . the disclosed system may also be provided as a built in apparatus requiring no day to day operator interaction . additionally , the disclosed system and method may be transmitted electronically to the operator over the network . fig2 . shows a flowchart of the computer ( or computer system as described above ) receiving network data in step 200 , composing a sequential decision problem in step 201 , error checking , validating and convergence checking the sequential decision problem in step 202 , solving the sequential decision problem in step 203 , outputting the optimal policy in step 204 and providing an opportunity for operator review in step 205 which may lead to re - optimization . in step 200 the computer receives the network data , which may include a graph of the network and other information relevant to the network and analyzes the provided information to define the elements of a sequential decision problem or series of sequential decision problems ), including at least the states , actions , transitions , rewards , and discount factor . the network data includes a set of nodes and a set of edges , plus a set of probabilities of transitioning from one node to another ( intensity information ), which may be affected by management decisions of the network operator . the network information may include additional information such as user activities on the network and network operations including purchasing , disengagement , costs and profit margins . the network information may further include funnel information , describing the fraction of users that voluntarily execute commands that transition them to other pages , select products to purchase , request information , purchase , and make payments . the other information may be sales records , customer information , pricing data or any other information the user knows , or believes to be , relevant to the situation the network the user is attempting to optimize . additionally , the other information ( either with or without the network information ) may be statistically analyzed to sort the relevant information from the irrelevant information , and calibrated to minimize computational requirements and provide decision making advice that will most accurately guide the user . the network data is analyzed to compose a sequential decision problem . a sequential decision problem comprises states , transitions , rewards , actions , a discount factor and time index . the states consist of possible situations in which the subject must make a decision . each state describes the value of one or more state dimensions , which represent the conditions the subject would like to consider when solving the decision problem to maximize the expected future value . for example , in a network optimization problem states may consist of web pages a customer can be on , as well as conditions of a web page ( such as scroll location on a multi - screen page ) or information about the customer . in a traffic management problem states may consist of car locations on the overall network of streets or the number of cars at each intersection in a road network . the transitions represent the probabilities of moving between states when an action is taken . for example , transitions may represent the probabilities of certain outcomes when a driver moves on a street with certain light timings , or a new visitor comes to a web - page with a sale advertisement displayed . the actions is the set of actions from which the operator selects , at every point in the time index . for example , a user may be able to change light timings in a traffic setting , add new links between pages on a website , or choose content to be displayed on the website . action costs are the cost to the operator of performing a specific action from the set of actions . at each point in the time index , the operator and network are in a state and the operator selects a specific action , then the operator receives a reward determined by the rewards and the action costs . additionally , certain actions may cause the computer to generate multiple sequential decision problems , for example when the actions change the state space by adding or eliminating a state . in other cases ( based upon the information received in step 200 or operator review in step 205 there may be actions that change other parts of the sequential decision problem , including state , rewards or transition matrices . for example , a certain action may trigger in a specified state changing the decision index from daily to hourly . the reward represents the benefit received by the user when transitioning between states at a point in the time index after having performed an action . for example , the user adds a new web - page presenting new product information between a home page and a catalog page on a website . the discount factor represents the time value of money , as well as potentially other subjective criteria . the time index is a set of decision points — discrete moments in time — when the operator takes a specific action from the set of actions . the discount factor takes into account both the risk of considerations related to the timing of rewards and actions . for example the discount factor may represent the operator &# 39 ; s preference for immediate over delayed rewards or the operator &# 39 ; s assessment of the underlying risk to the subject embedded in the decision problem . the use of such a factor is sometimes considered as converting future potential rewards to their “ present value ” rewards . in steps 201 the computer composes the sequential decision problem . the computer system processes the elements from the network data to compose a sequential decision problem by defining a functional equation ( or series of functional equations ). the functional equation is an equation that may be solved to maximize the value to the operator for a decision problem in all states . the operator may be prompted to select a solution technique , or a default solution technique can be set and used . in step 202 the computer error - checks , validates and checks for convergence the functional equation to ensure that the functional equation is solvable . these checks confirm that all necessary portions of the problem have been defined through processing the network data . in step 203 the computer solves the functional equation . different solution techniques may be used depending on the particular form of the functional equation . some solution techniques include value function iteration ( also known as successive approximations , over - relaxation , or pre - jacobi iteration ), policy iteration ( also known as policy improvement ), a root - finding algorithm , or other numeric solution techniques . the output is the optimal policy , a mapping of states to a recommended set of actions and the value to the operator in each state . in step 204 the optimal policy is output . this may mean display for an operator to review or directly implemented by the computer . for example , if the network is a traffic network and the optimal policy contains light timings the computer system may implement the light timings . in another example the network is a series of web sites and the optimal policy may contain decision advice about offering sales or changing links between web sites and the computer may automatically implement the advice . the operator may only implement some of the decision advice or merely review in step 205 . in step 205 the operator may review the solution and implement some or all of the solution , change the inputs or allow the network to operate for a period of time . fig3 . shows a flowchart of an embodiment where the computer implements the decision advice , observes how the decision advice performs and re - optimizes after a trigger has been reached or upon operator decision . the computer populates initial information in step 300 , handles a sequential decision problem in step 301 , produces and implements decision making advice in step 302 , observes results ( potentially including waiting for a trigger to occur ) in step 303 and changes the information in step 304 . in step 300 the computer receives or identifies from available inputs the elements of a sequential decision problem as described for step 200 . in step 301 the computer performs the functions as described in steps 201 , 202 and 203 . in step 302 the computer creates decision making advice largely as described in step 204 , however in step 302 the computer automatically implements the decision making advice on the network . this may include changing elements of a web - page , offering sales to visitors coming from specific out - of - network locations , changing the links ( edges in graph parlance ) between pages ( nodes ), displaying targeted advertising , prompting the operator to completely or nearly completely restructure the network ( an expensive and time consuming option ) or changes to nodes , edges , node characteristics , actions , or other elements of the sequential decision problem . in step 303 the computer monitors input , from the network or any other available sources , observing for triggering events and storing data to use in improving the information used in re - optimization . re - optimization may happen after a pre - set time or event or otherwise automatically re - optimize the network in an ongoing fashion . for example , a traffic network could be set to re - optimize whenever a traffic jam of a certain length is detected at specified locations or social network could be set to prune links between people once a week based upon detection of specified language . this may mean implementation ( for example the policy is used to change traffic lights or displayed content of webpages in a website or the links between webpages on a website ). additionally , re - optimization may happen on a fixed schedule . one important thing to consider is that for many problems many decision points must pass before the monitored information becomes significant enough to change the decision making advice . it may be sub - optimal to re - optimize after every visitor to a website or car passes through a traffic light , instead re - optimization may be set to happen after a relatively large amount of data is collected compared to the initial information . furthermore the meaning of a relatively large amount of data may change as optimization cycles pass . for example , when optimizing a website re - optimization may be set to occur every time total traffic through the site slows for ten consecutive days or after the site &# 39 ; s number of hits has increased by a fixed percentage . fig4 . shows a further embodiment of a highly customized ( which can also be thought of as a choose - your - own - adventure ) network . in fig4 . there are nodes ( 400 , 401 , 402 , 403 , 404 , 405 , 406 and 407 ), connected by edges ( 400 a , 400 b , 400 c , 400 d ) and the nodes have characteristics displayed in the node and the table 400 ch . in this embodiment , the operator changes the content and links visible to the visitor of a website based on information about the visitor . the processor decides to display content from a group of available content ( front page , written info , video info , pricing , testimonials and checkout ). additionally , the visitors may be classified according to type , shown in fig4 as types a , b , c and d ; and certain links displayed to the visitor depending on the node and the visitor classification . these network nodes may be pages on a website . in this embodiment , a visitor enters the website at node 400 ( with the node characteristic “ front page ). depending on the classification of the visitor , based upon information known or available about the visitor , the visitor will observe a link to node 401 , or node 407 . in fig4 if the visitor to node 400 is classified as type a the visitor will be shown links ( shown as 400 a ) to node 401 and node 402 . if the visitor to node 400 is classified as type b the visitor will be shown a link to node 407 . if a visitor of type b chooses to clock on the link to node 407 , the visitor will observe written information , video information and pricing content . if the visitor on node 407 is classified as type b , the visitor will observe a link to node 406 . if the visitor on node 407 is classified as type d , the visitor will observe links to node 403 and node 404 . in this embodiment all visitors to node 400 are initially characterized as types a or b , and based upon information learned from the visitor while they are on the site they may be further categorized as type c or d . however , it may also be beneficial to have a ( not shown ) default network for first time or anonymous visitors , or to have many more links or node characteristics than shown here . fig5 . shows an exemplary embodiment of a graph having nodes 501 , 502 , 503 , 504 , 505 , 506 , 507 , 508 and 509 , edges 510 , 511 , 512 ( only edges specifically discussed labeled ), and a chart 500 c showing an alternative embodiment of the information with additional other information ( also known as node characteristics ) relevant to the network and operator . fig5 . generally , and the nodes specifically , may be viewed as an embodiment showing traffic through a webpage where individual nodes are websites , or it may be viewed as a breakdown of a single or several websites with individual nodes representing elements on the webpages and the edges being derived from heat - map data showing how visitors move between areas on the webpages and websites . nodes may have characteristics , included as other information . in fig5 . this is shown in the chart 500 c ( column other ) with node 501 being defined as the front page , nodes 502 and 503 being internal pages and node 509 being the checkout page . in this sales website embodiment the reward received by the operator may be entirely vested in the checkout node , node 509 . in a traffic embodiment the reward may be vested in the exit nodes or distributed throughout the nodes depending on whether the primary concern is throughput through the network or movement within the network . edges 510 , 511 and 512 have different widths representing throughput , thicker lines showing higher throughput . for example , edge 510 as the thickest line shows the highest level of throughput , edge 510 is connected to node 501 and 502 , in fig5 . node 501 may represent the landing page of a website , the thick lines emanating from node 501 showing that the majority of visitors first go to the landing page of a website and then move through the website . some visitors may drop off ( for example , by closing the browser , or in a car - traffic problem by entering off - road parking ). other visitors enter directly to a node other than 501 . edges may be used as directed or non - directed , i . e . they may represent only traffic from one to another node , all the traffic between two nodes or some combination or average of the net - traffic between two nodes . the flow column shows one way of tabulating the throughput or flow data also shown as line width . fig6 . shows an exemplary embodiment of a traffic network that can be used as network data . fig6 shows three decision points , decision point 600 ( the upper - most ), decision point 601 ( the middle ) and decision point 602 ( the bottom - most ). arrow 600 f shows the first shown action being selected at decision point 600 and the network transition to its state at decision point 601 . arrow 600 s shows the second shown action being selected at decision point 601 and the network transition to its state at decision 602 . charts 600 a , 601 a and 602 a show the entrances to the intersections , their loads and the color of the signal lights . formula 603 shows the relationship between possible lights , possible loads and the number of nodes . formula 604 shows the relationship between the size of the state vector and number of nodes . formula 605 shows the relationship between the size of the action vector and the number of nodes . enclosed numerals 1 through 6 are nodes , representing entrances to an intersection . each node defines a dimension of the state vector and a dimension of the action vector . each state vector element defines the numbers of cars at that node . each action vector element defines the color of the light at the intersection . the color of the lights is shown by the numeral being enclosed by a circle ( red light , for stop ) or rectangle ( green light , for go ). the edges define how one can move from one intersection to the next . for example , from node 4 a transition to node 5 ( by going straight ) or a transition out of the network ( by making a right turn ) is possible . a transition directly from node 4 to node 6 is not possible . the agent ( who could be the user or someone the user has control over or someone employing the user or otherwise connected to the user ) has control over the light colors and allowable transitions . the actions the operator takes change how the car load is distributed after transitions between states . this embodiment shows merely one way of encoding network information into a sequential decision problem , others are possible . for example , it is possible to analyze a network by considering each state as a total description of the network , but it is also possibly to have each state represent a node with different state dimension representing different node characteristics .