According to some embodiments of the present invention there is provided a method for determining a control action in a control system using a Markov decision process. The method comprises an action of receiving two or more predefined transition probability values of a Markov decision process (MDP) of a control system, where each of the predefined transition probability values is associated with a transition between a first state and a second state, both from two or more system states, resulting from execution of one or more control actions of the control system. The method comprises an action of computing one or more new transition probability values by an analysis of the predefined transition probability values, the system states and the control actions. The method comprises an action of determining one or more recommended control actions for the respective system state based on the new transition probability value.
Please help me write a proper abstract based on the patent claims. CLAIM: 1. A method for determining a control action in a control system using a Markov decision process, comprising: receiving a plurality of predefined transition probability values of a Markov decision process (MDP) of a control system, wherein each of said plurality of predefined transition probability values is associated with a transition between a first state of a plurality of system states and a second state of said plurality of system states resulting from execution of one of a plurality of control actions of said control system; computing, by at least one hardware processor, at least one new transition probability value from an analysis of said plurality of predefined transition probability values, said plurality of system states and said plurality of control actions; and determining, by said at least one hardware processor, at least one recommended control action from said plurality of control actions for respective at least one of said plurality of system states based on said at least one new transition probability value. 2. The method of claim 1, wherein said analysis is an interpolation of at least some of said plurality of predefined transition probability values to compute said at least one new transition probability value. 3. The method of claim 1, wherein each of said plurality of predefined transition probability values is computed by said at least one hardware processor from some of a plurality of sensor measurement values and some of a plurality of control measurement values acquired by an application system. 4. The method of claim 1, wherein said plurality of predefined transition probability values are computed by said at least one hardware processor of a simulation system from simulations of at least one system model of an application system, wherein said at least one system model computes said second state as output when said first state and one of said control actions is received as input. 5. The method of claim 1, wherein each of said plurality of system states comprises a plurality of system variable values, wherein each of plurality of system variable values corresponds to a value of one of a plurality of sensors of an application system. 6. The method of claim 5, wherein said analysis is an average of some of said plurality of predefined transition probability values, wherein corresponding said plurality of system states are neighboring states having equal values for all but one of said plurality of system variable values. 7. The method of claim 6, wherein each of said plurality of system variable values are classified into a series of value steps and wherein corresponding said one unequal value of said plurality of system variable values is unequal by one of said values steps. 8. The method of claim 6, wherein said average of some of said plurality of predefined transition probability values are predefined transition probability values computed from a corresponding number of transitions greater than a threshold value. 9. The method of claim 1, wherein some of said plurality of system states are classified as controllable system states and the remaining of said plurality of system states are classified as action-independent system states, wherein said classification is made according to each of said plurality of system states being affected by said control system, and wherein said analyzing is performed by multiplying a sum of corresponding transition probabilities between said action-independent system states with a sum of corresponding transition probabilities between said controllable system states and said action-independent system states. 10. The method of claim 1, wherein said analyzing is assigning a minimum probability value to one of said at least one new transition probability values when corresponding one of said plurality of predefined transition probability values are below said minimum probability value. 11. The method of claim 1, wherein each of said plurality of system states is associated with at least one of a reward value and a cost value, and wherein determining said at least one new control action is further based on respective said at least one of said reward value and said cost value. 12. The method of claim 11, wherein said at least one new control action is a plurality of control actions and corresponding said at least one of said reward value and said cost value are used to compute at least one of a reward function and a cost function for said plurality of control actions. 13. The method of claim 5, wherein said application system is an industrial system and each of said plurality of sensors is a sensor from the group of a temperature sensor, a position sensor, a pressure sensor, a flow sensor, a light sensor, a chemical species sensor, a pH sensor, a gas sensor, a fluid level sensor, a status sensor, and a purity sensor. 14. The method of claim 1, wherein said receiving and computing are performed by at least one first hardware processor and said determining is performed by at least one second hardware processor. 15. The method of claim 4, wherein said control action is missing a predefined transition probability value, and wherein said simulations are control action driven simulations for computing said missing predefined transition probability value. 16. The method of claim 4, wherein said second state is missing a predefined transition probability value, and wherein said simulations are system state driven simulations for computing said missing predefined transition probability value. 17. A computer program product for determining a control action in a control system using a Markov decision process, said computer program product comprising a computer readable non-volatile storage medium having stored thereon: first program instructions executable by a hardware processor to cause said device to receive a plurality of predefined transition probability values of a Markov decision process (MDP) of a control system, wherein each of said plurality of predefined transition probability values is associated with a transition between a first state of a plurality of system states and a second state of said plurality of system states resulting from execution of one of a plurality of control actions of said control system; second program instructions executable by said hardware processor to cause said device to compute at least one new transition probability value by an analysis of said plurality of predefined transition probability values, said plurality of system states and said plurality of control actions; and third program instructions executable by said hardware processor to cause said device to determine at least one recommended control action from said plurality of control actions for respective at least one of said plurality of system states based on said at least one new transition probability value. 18. A computerized system for determining a control action in a control system using a Markov decision process, comprising: a network interface for receiving system state transition data from at least one of a simulation system and an application system; at least one hardware processor; and at least one memory device having processor instructions stored thereon to cause said at least one hardware processor to perform the actions of: receiving a plurality of predefined transition probability values of a Markov decision process (MDP) of a control system, wherein each of said plurality of predefined transition probability values is associated with a transition between a first state of a plurality of system states and a second state of said plurality of system states resulting from execution of one of a plurality of control actions of said control system; computing at least one new transition probability value by an analysis of said plurality of predefined transition probability values, said plurality of system states and said plurality of control actions; and determining at least one recommended control action from said plurality of control actions for respective at least one of said plurality of system states based on said at least one new transition probability value. 19. The computerized system of claim 18, further comprising said at least one hardware processor performing the action of sending said at least one new transition probability value to at least one second hardware processor, wherein said determining is performed on said at least one second hardware processor.