title: Cython interfaced with C++: segmentation fault for large arrays
body: <p>I am transferring my code from Python/C interfaced using ctypes to Python/C++ interfaced using Cython. The new interface will give me an easier to maintain code, because I can exploit all the C++ features and need relatively few lines of interface-code. </p>
  
  <p>The interfaced code works perfectly with small arrays. However it encounters a <em>segmentation fault</em> when using large arrays. I have been wrapping my head around this problem, but have not gotten any closer to a solution. I have included a minimal example in which the segmentation fault occurs. Please note that it consistently occurs on Linux and Mac, and also valgrind did not give insights. Also note that the exact same example in pure C++ does work without problems.</p>
  
  <p>The example contains (part of) a Sparse matrix class in C++. An interface is created in Cython. As a result the class can be used from Python. </p>
  
  <h1>C++ side</h1>
  
  <p><code>sparse.h</code></p>
  
  <pre><code>#ifndef SPARSE_H
  #define SPARSE_H
  
  #include &lt;iostream&gt;
  #include &lt;cstdio&gt;
  
  using namespace std;
  
  class Sparse {
  
    public:
      int* data;
      int  nnz;
  
      Sparse();
      ~Sparse();
      Sparse(int* data, int nnz);
      void view(void);
  
  };
  
  #endif
  </code></pre>
  
  <p><code>sparse.cpp</code></p>
  
  <pre><code>#include "sparse.h"
  
  Sparse::Sparse()
  {
    data = NULL;
    nnz  = 0   ;
  }
  
  Sparse::~Sparse() {}
  
  Sparse::Sparse(int* Data, int NNZ)
  {
    nnz  = NNZ ;
    data = Data;
  }
  
  void Sparse::view(void)
  {
  
    int i;
  
    for ( i=0 ; i&lt;nnz ; i++ )
      printf("(%3d) %d\n",i,data[i]);
  
  }
  </code></pre>
  
  <h1>Cython interface</h1>
  
  <p><code>csparse.pyx</code></p>
  
  <pre><code>import  numpy as np
  cimport numpy as np
  
  # UNCOMMENT TO FIX
  #from cpython cimport Py_INCREF
  
  cdef extern from "sparse.h":
    cdef cppclass Sparse:
      Sparse(int*, int) except +
      int* data
      int  nnz
      void view()
  
  
  cdef class PySparse:
  
    cdef Sparse *ptr
  
    def __cinit__(self,**kwargs):
  
      cdef np.ndarray[np.int32_t, ndim=1, mode="c"] data
  
      data = kwargs['data'].astype(np.int32)
  
      # UNCOMMENT TO FIX
      #Py_INCREF(data)
  
      self.ptr = new Sparse(
        &lt;int*&gt; data.data if data is not None else NULL,
        data.shape[0],
      )
  
    def __dealloc__(self):
      del self.ptr
  
    def view(self):
      self.ptr.view()
  </code></pre>
  
  <p><code>setup.py</code></p>
  
  <pre><code>from distutils.core import setup, Extension
  from Cython.Build   import cythonize
  
  setup(ext_modules = cythonize(Extension(
    "csparse",
    sources=["csparse.pyx", "sparse.cpp"],
    language="c++",
  )))
  </code></pre>
  
  <h1>Python side</h1>
  
  <pre><code>import numpy as np
  import csparse
  
  data = np.arange(100000,dtype='int32')
  
  matrix = csparse.PySparse(
    data = data
  )
  
  matrix.view() # --&gt; segmentation fault
  </code></pre>
  
  <p>To run:</p>
  
  <pre><code>$ python setup.py build_ext --inplace
  $ python example.py
  </code></pre>
  
  <p><em>Note that <code>data = np.arange(100,dtype='int32')</code> does work</em>.</p>