title: How to preserve extra column information when using sparse matrices?
body: <p>I am trying find out how I can preserve extra information when using sparse matrices. I am writing some code that turns pandas dataframes into networks. The dataframe has a column with nodes, a column with edges, and some attribute columns.</p>
  
  <pre><code>&gt;&gt;&gt; df.head()
  
    authors  papers     attr1             attr2
  0       g  [4, 8]  yikpmrbf  FCCLYDWSFJVORGXL
  1       h     [7]  zxdkzyam  DHJZADCAPRQYAXGR
  2       c     [4]  lnynqqhf  PACVSZULQSKXBURW
  3       k  [5, 3]  vjzgyebn  QMLHUIDWXGNPVQTH
  4       r     [7]  etjghgcp  CSYIMFXKOYFWWUTH
  </code></pre>
  
  <p>So far, I have been using the following code to make a networkx graph, where <code>authors</code> are nodes which have edges if they share a value in <code>papers</code>. </p>
  
  <pre><code>from itertools import chain
  import numpy as np
  from scipy import sparse
  import networkx as nx
  
  node='authors'
  link='papers'
  
  indptr = np.fromiter(chain((0,), map(len, df[link])),
                       int, len(df[link])+1).cumsum()
  unq, idx = np.unique(np.concatenate(df[link]), return_inverse=True)
  node_link_matrix = sparse.csr_matrix(
      (np.ones(idx.size, int), idx, indptr),
      (len(df[link]),
       len(unq))
  )
  node_node_matrix = (node_link_matrix@node_link_matrix.T).tocoo()
  G = nx.convert_matrix.from_scipy_sparse_matrix(node_node_matrix)
  </code></pre>
  
  <p>This calculates edge weights for each node-node pair in the data. <code>node_link_matrix</code> has as many rows as there are rows in the dataframe, and as many columns as there are unique edge-values (unique papers in this case). The values thus indicate "how many of x paper are present in x row". <code>node_node_matrix</code> is square and has as many rows/columns as there are rows in the dataframe. It is the dot product of <code>node_link_matrix</code> and <code>node_link_matrix</code> transposed. The values are edge weights, and this matrix can easily be turned into a graph with networkx.</p>
  
  <pre><code>&gt;&gt;&gt; node_link_matrix
  &lt;74x10 sparse matrix of type '&lt;class 'numpy.int64'&gt;'
      with 140 stored elements in Compressed Sparse Row format&gt;
  
  &gt;&gt;&gt; node_node_matrix
  &lt;74x74 sparse matrix of type '&lt;class 'numpy.int64'&gt;'
      with 1786 stored elements in COOrdinate format&gt;
  </code></pre>
  
  <p>What I am trying is get information about the <code>attr1</code> column into this. I first tried replacing the "ones" with the actual attribute strings, so the previous code becomes:</p>
  
  <pre><code>from itertools import chain
  import numpy as np
  from scipy import sparse
  import networkx as nx
  
  node='authors'
  link='papers'
  
  indptr = np.fromiter(chain((0,), map(len, df[link])),
                       int, len(df[link])+1).cumsum()
  unq, idx = np.unique(np.concatenate(df[link]), return_inverse=True)
  node_link_matrix = sparse.csr_matrix(
      (np.repeat(df.attr1.values, np.diff(indptr)), idx, indptr),  # &lt;-- Changed this part
      (len(df[link]),
       len(unq))
  )
  </code></pre>
  
  <p>This doesn't work, as sparse matrices don't handle strings well. I tried to overcome this by replacing the actual values with a dictionary mapping each occurrence to the correct attribute</p>
  
  <pre><code>from itertools import chain
  import numpy as np
  from scipy import sparse
  import networkx as nx
  
  node='authors'
  link='papers'
  
  indptr = np.fromiter(chain((0,), map(len, df[link])),
                       int, len(df[link])+1).cumsum()
  unq, idx = np.unique(np.concatenate(df[link]), return_inverse=True)
  node_link_matrix = sparse.csr_matrix(
      (range(idx.size), idx, indptr),  # &lt;-- Values are now IDs
      (len(df[link]),
       len(unq))
  )
  
  edge_attr_dict = dict(zip(range(idx.size), np.repeat(df.attr1.values, np.diff(indptr))))
  </code></pre>
  
  <p>This, then, gives me a <code>node_link_matrix</code> mapping nodes values to edge values and their corresponding attr1 value. The problem is, since the values now are symbolic numbers and not just ones and zeros, the dot product function to get the node-node relationship don't work anymore. So what I need is a way to retain the node attribute information in the sparse matrix, but still be able to use the dot product function as it is meant to. I have been considering whether this could be done by "pushing" the attribues into a third dimension, using a 3d tensor instead of a sparse matrix, but I don't have much experience with this. </p>