<h1>TLDR</h1>
<p>To compute embeddings on large graphs use <a href="https://github.com/AnacletoLAB/grape" rel="nofollow noreferrer">GRAPE</a>.</p>
<pre><code>pip install grape
</code></pre>
<p>Example:</p>
<pre class="lang-py prettyprint-override"><code>from grape import Graph
from grape.embedders import Node2VecGloVeEnsmallen

graph = Graph.from_csv(
    ## The path to the edges list tsv
    edge_path=&quot;edges.csv&quot;,
    sources_column=&quot;source&quot;,
    destinations_column=&quot;destination&quot;,
    directed=False,
)
embedding = Node2VecGloVeEnsmallen().fit_transform(graph)
</code></pre>
<h1>Longer answer</h1>
<p>To solve this issue, we developed <a href="https://github.com/AnacletoLAB/grape" rel="nofollow noreferrer">GRAPE</a>, our Rust library with Python bindings. We needed to run node2vec on big graphs and the libraries we found weren't fast enough. We re-implemented and optimized many models, including Node2vec's, from the ground up without using Tensorflow or Pytorch.</p>
<p>Here's some benchmarks on our server with 12 cores (24 threads) and 128GB of ram.</p>
<p><a href="https://i.stack.imgur.com/d2i91.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/d2i91.png" alt="enter image description here" /></a>
Info about the tested graphs:</p>
<ul>
<li>WikiEN has 130M edges and 17M nodes.</li>
<li>CTD has 45M edges and 100K nodes.</li>
<li>PheKnowLator has 7M edges and 800K nodes.</li>
</ul>
<p>Here's the tutorial on how to load your own custom graph, we support CSV-like formats:
<a href="https://github.com/AnacletoLAB/grape/blob/main/tutorials/Loading_a_Graph_in_Ensmallen.ipynb" rel="nofollow noreferrer">https://github.com/AnacletoLAB/grape/blob/main/tutorials/Loading_a_Graph_in_Ensmallen.ipynb</a></p>
<p>and here's the complete tutorial on how to run and visualize node2vec with Glove on a given graph:
<a href="https://github.com/AnacletoLAB/grape/blob/main/tutorials/Using_Node2Vec_GloVe_to_embed_Cora.ipynb" rel="nofollow noreferrer">https://github.com/AnacletoLAB/grape/blob/main/tutorials/Using_Node2Vec_GloVe_to_embed_Cora.ipynb</a>
<a href="https://i.stack.imgur.com/v3uU2.jpg" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/v3uU2.jpg" alt="enter image description here" /></a></p>
<p>Our paper is still under review ( <a href="https://arxiv.org/abs/2110.06196" rel="nofollow noreferrer">https://arxiv.org/abs/2110.06196</a> ) and we are just two developers, so, if you need any help contact us on Discord, Github, or Twitter <a href="https://twitter.com/GRAPElib" rel="nofollow noreferrer">@GRAPElib</a>.</p>
Is there any way to make Node2Vec faster?
<p>I have a graph with 480k nodes and 34M edges. I want to create node embeddings using Node2Vec on this graph. But, It is not even able to calculate transition probabilities. I am using a Google Cloud Machine with 32 cores and 120 GB RAM. Infrastructure is not the problem, the problem is that the function _precompute_probabilities in the node2vec pip library is not paraller. It is using only a single thread to calculate the transition probabilities. Is there a way to make this parallel or is they any other parallel version of Node2Vec ?</p>