title: Tensorflow - build cyclic graph
body: <p>I'm building TF graph,this is part of my code:</p>
  
  <pre><code>class Model:
    def __init__(self, neurons, links, input_neurons_num=4):
      """
      Constructor.
      :param neurons: array list of neurons
      :param input_neurons_num: number of input neurons
      """
      # neuron_id as key and weights entering in it as value
      self.weights = {}
      # neuron_id as key and neurons entering in it as value
      self.connections = {}
      self.graph = None
  
   def build_graph(self):
       with self.graph.as_default():
          operations = {}
  
          # create Variables for input vertices
          for neuron_id in self.input_neurons:
              self.inputs[neuron_id] = tf.get_variable(name=str(neuron_id), shape=(),
                                                       initializer=tf.zeros_initializer)
  
          # create input &amp; output vertices
          for neuron_id in self.connections:
              input_neuron_ids = self.connections[neuron_id]
  
              # weights
              v_weights = tf.constant(self.weights[neuron_id])
              # input vertices
              v_inputs = []
  
              for input_neuron_id in input_neuron_ids:
                  if self.is_input_neuron(input_neuron_id):
                      vertex = self.inputs[input_neuron_id]
                  else:
                      # KeyError if input_neuron_id isn't alreay created
                      vertex = operations[input_neuron_id]
  
                  v_inputs.append(vertex)
  
  
              # multiply weights and inputs
              mul = tf.multiply(v_inputs, v_weights, str(neuron_id))
  </code></pre>
  
  <p>So I have list of links, where each link has <em>from_neuron</em>, <em>to_neuron</em> and <em>weight</em>. For example: (1,2,3) => <em>edge</em>(connection) from 1 to 2 with weight 2.
  I want to iterate through all links and based on connections build graph.</p>
  
  <p>At start I know input and output nodes. Idea is to iterate through links and gradually build graph. If there is <em>node 4</em>: (1,4,2), (2,4,3.5) I would like to create a <em>tf.operation</em> which will multiply output from 1 and its weight(2), output from 2 and its weight(3.5), sum obtain values and pass it forward through network.
  But problem is if I have input nodes: 1,2,3, and <em>node 4</em> that has connection with etc. <em>node 7</em> which is not created yet. It will try to reference node that isn't created yet, and I will get <em>KeyError</em>.</p>
  
  <p>Then I tried to skip nodes that are connected with ones that aren't exists yet:</p>
  
  <pre><code>deletion = []
          while len(self.connections) &gt; 0:
              for neuron_id in deletion:
                  self.connections.pop(neuron_id, None)
              deletion = []
              # create input &amp; output vertices
              for neuron_id in self.connections:
                  # same logic with addition:
                  deletion.append(neuron_id)
  </code></pre>
  
  <p>And this works, but problem is when I have cycle in graph. This will fall into infinite loop.</p>
  
  <p>Only idea I have to solve this problem is to do this in two passes. In first pass to create all nodes in graph, and in second to replace them with actually values. I thought to use placeholders, but I'm not sure how to implement that. </p>
  
  <p>So any help is welcome. </p>