title: Inserting large graph data into Neo4j using py2neo WriteBatch
body: <p>I have a graph represented by the following files:  </p>
  
  <ul>
  <li>VertexLabel.txt -> each line contains properties for each vertex.  </li>
  <li>EdgeLabel.txt -> each line contains properties for each edge.  </li>
  <li>EdgeID.txt -> each line contains 3 separated integers which correspond to indexes in the label files:
  source_index target_index edge_index.</li>
  </ul>
  
  <p>There are roughly 44K vertices with 240K edges. I'm trying to use <a href="http://nigelsmall.com/py2neo/1.6/batches/" rel="nofollow">neo4j.Writebatch</a> to batch insert the graph data.  </p>
  
  <pre><code>from py2neo import Graph, neo4j, node, rel
  
  graph_db = Graph()
  nodes = {}
  batchNodes = {}
  edges = {}
  edgeList = []
  
  # Read vertex label file into nodes, where node[i] is indexed according to the order the nodes appear in the file.
  # Each entry is of type node, e.g. node("FILM", title = "Star Trek"), node("CAST", name = "William Shatner")
  ...  
  
  # Read edge label file into edges, where edges[i] is indexed according to the order the edges appear in the file.
  # Each entry is a tuple (edge_type, edge_task), e.g. ("STAFF", "Director")
  ...  
  
  # Read edge id file into edgeList
  # Each entry is the tuple (source_index, target_index, edge_index), e.g. (1, 4, 8)
  ...  
  
  # Iterate nodes, store in graph
  # Note, store result of batch.create into batchNodes
  batch = neo4j.WriteBatch(graph_db)
  count = 0
  for n in nodes:
      batchNodes[n] = batch.create(nodes[n])
      count += 1
  
      # Submit every 500 steps
      if count % 500 == 0:
          count = 0
          batch.submit()
          batch = neo4j.WriteBatch(graph_db)
  
  # Submit remaining batch
  batch.submit()
  
  # Iterate edgeList, store in graph
  batch = neo4j.WriteBatch(graph_db)
  count = 0
  for i, j, k in edgeList:
      # Lookup reference in batchNodes
      source = batchNodes[i]
      target = batchNodes[j]
      edge = edges[k]
      batch.create(rel(source, edge[0], target, {"task": edge[1]}))
      count += 1
  
      # Submit every 500 steps
      if count % 500 == 0:
          count = 0
          batch.submit()
          batch = neo4j.WriteBatch(graph_db)
  
  # Submit remaining batch
  batch.submit()
  </code></pre>
  
  <p>I get the following error:</p>
  
  <pre><code>Traceback (most recent call last):   File "test4.py", line 87, in &lt;module&gt;
      batch.create(rel(source, edge[0], target, {"task": edge[1]}))   File "C:\Python34\lib\site-packages\py2neo\batch\write.py", line 181, in create
      start_node = self.resolve(entity.start_node)   File "C:\Python34\lib\site-packages\py2neo\batch\core.py", line 374, in resolve
      return NodePointer(self.find(node))   File "C:\Python34\lib\site-packages\py2neo\batch\core.py", line 394, in find
      raise ValueError("Job not found in batch") ValueError: Job not found in batch
  </code></pre>
  
  <p>I presume that batchNodes is not actually containing the proper reference to the nodes which I want to lookup for adding relationships (perhaps reinitializing the batch object invalidates the references). In this case, how should I perform this task?  </p>
  
  <p>I am using Neo4j 2.1.7 (Community Edition) and py2neo 2.0.4.</p>