```python
import sympy
from phasor.utilities.ipynb.displays import *
from phasor.utilities.ipynb.ipy_sympy import *
import scipy.linalg

import numpy.testing as np_test
import declarative as decl
from declarative.bunch import (
    DeepBunch,
)

from test_SVD import SVD_gen_check
from phasor.system import DAG_algorithm
from phasor.system import scisparse_algorithm
```

```python
%%prun -s cumtime
r = SVD_gen_check(
    N = 100,
    length = 1000,
    solver = DAG_algorithm,
)
```

    SPARSITY FRAC:  0.0258
    TRIVIAL STAGE, REMAINING 100
    TRIVIAL STAGE, REMAINING 46
    BADGUY STAGE, REMAINING 40

    /home/mcculler/local/home_sync/projects/phasor/phasor/system/DAG_algorithm.py:393: RuntimeWarning: divide by zero encountered in true_divide
      vprint("NORM: ", np.max(normr / norma), np.max(normc / norma))
    /home/mcculler/local/home_sync/projects/phasor/phasor/system/DAG_algorithm.py:732: RuntimeWarning: invalid value encountered in true_divide
      u_mod_edge = edge + norm_orig * edge / abs(edge)
    /home/mcculler/local/home_sync/projects/phasor/phasor/system/DAG_algorithm.py:355: RuntimeWarning: invalid value encountered in greater
      rel_r_to_c = np.asarray(np.count_nonzero(normr > normc))
    /home/mcculler/local/home_sync/projects/phasor/phasor/system/DAG_algorithm.py:373: RuntimeWarning: invalid value encountered in greater_equal
      bignodes_r = np.array(rvec) >= 1./(len(req[node]))**.5
    /home/mcculler/local/home_sync/projects/phasor/phasor/system/DAG_algorithm.py:386: RuntimeWarning: invalid value encountered in greater_equal
      bignodes_c = np.array(cvec) >= 1./(len(seq[node]))**.5

```python
%%prun -s cumtime
r = SVD_gen_check(
    N = 100,
    length = 1000,
    solver = scisparse_algorithm,
)
```

    SPARSITY FRAC:  0.0196

    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 2.19e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.00e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.02e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.03e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.05e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.06e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.08e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.10e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.11e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.13e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.15e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.17e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.18e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.20e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.22e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.24e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.26e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.28e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 3.78e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.32e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.34e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.36e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.38e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.40e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.42e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.44e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.47e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.49e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.51e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.54e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.56e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.58e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.61e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.63e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.66e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.68e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.71e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.74e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.76e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.79e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.82e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.85e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.88e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.90e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.93e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.96e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.99e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 2.02e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 2.06e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 2.09e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 2.12e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 2.15e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 2.22e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 2.26e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 2.29e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 2.33e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 2.36e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 2.40e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 2.44e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 2.47e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 2.51e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 2.55e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 2.59e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 2.63e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 2.67e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 2.71e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 2.76e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 2.80e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 2.84e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 2.89e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 2.93e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 2.98e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 3.02e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 3.07e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 3.12e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 3.17e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 3.22e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 3.27e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 3.32e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 3.37e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 3.42e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 3.48e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 3.53e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 3.59e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 3.64e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 3.70e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 3.76e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 3.81e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 3.87e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 3.93e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 4.00e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 4.06e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 4.12e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 4.19e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 4.25e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 4.32e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 4.39e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 4.46e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 4.53e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 4.60e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 4.67e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 4.74e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 4.82e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 4.89e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.04e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.07e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.14e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.23e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.30e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.37e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.39e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.31e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.29e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.21e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.19e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.16e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.01e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.09e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.12e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.25e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.27e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.46e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.48e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.53e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.55e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.57e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.60e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.62e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.64e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.67e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.69e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.72e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.77e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.80e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.83e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.91e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.94e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.97e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 2.00e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 2.03e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 2.07e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 2.10e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 2.13e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 2.16e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 2.20e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 2.23e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 2.27e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 2.31e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 2.34e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 2.38e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 2.42e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 2.46e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 2.50e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 2.54e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 2.58e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 2.72e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 2.81e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 2.86e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 2.91e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 2.95e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 3.01e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 3.06e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 3.11e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 3.16e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 3.28e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 3.33e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 3.39e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 3.45e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 3.51e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 3.58e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 3.71e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 3.77e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 3.84e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 3.91e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 3.98e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 4.13e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 4.21e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 4.28e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 4.36e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 4.45e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 4.62e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 4.70e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 4.79e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 4.98e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 5.08e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 5.18e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 5.28e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 5.38e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 5.49e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 5.53e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 5.56e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 5.60e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 5.63e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 5.67e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 5.70e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 5.74e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 5.78e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 5.82e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 5.85e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 5.89e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 5.94e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 5.98e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 6.02e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 6.06e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 6.11e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 6.15e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 6.20e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 6.24e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 6.29e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 6.34e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 6.39e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 6.44e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 6.49e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 6.54e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 6.59e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 6.64e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 6.70e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 6.75e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 6.81e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 6.86e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 6.92e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 6.98e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 7.04e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 7.10e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 7.16e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 7.22e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 7.29e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 7.35e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 7.42e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 7.48e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 7.55e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 7.62e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 7.69e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 7.76e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 7.83e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 7.91e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 7.98e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 8.06e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 8.13e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 8.21e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 8.29e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 8.37e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 8.45e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 8.54e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 8.62e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 8.71e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 8.79e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 8.88e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 8.97e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 9.06e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 9.15e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 9.25e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 9.34e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 9.44e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 9.54e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 9.64e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 9.74e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 9.84e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 9.95e+12)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.01e+13)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.02e+13)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.03e+13)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.04e+13)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.05e+13)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.06e+13)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.07e+13)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.09e+13)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.10e+13)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.11e+13)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.12e+13)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.13e+13)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.15e+13)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.16e+13)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.17e+13)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.19e+13)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.20e+13)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.21e+13)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.23e+13)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.24e+13)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.26e+13)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.27e+13)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.29e+13)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.30e+13)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.32e+13)
      warnings.warn(msg, UmfpackWarning)
    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.33e+13)
      warnings.warn(msg, UmfpackWarning)

    /home/mcculler/.local/lib/python3.5/site-packages/scikits/umfpack/umfpack.py:718: UmfpackWarning: (almost) singular matrix! (estimated cond. number: 1.35e+13)
      warnings.warn(msg, UmfpackWarning)

```python
SVD_gen_check
```

    <function test_SVD.SVD_gen_check>

```python

```


```python
%matplotlib inline
```

```python
!ls \content
```

    ls: cannot access 'content': No such file or directory

Adversarial Example Generation
==============================

**Author:** `Nathan Inkawhich <https://github.com/inkawhich>`__

If you are reading this, hopefully you can appreciate how effective some
machine learning models are. Research is constantly pushing ML models to
be faster, more accurate, and more efficient. However, an often
overlooked aspect of designing and training models is security and
robustness, especially in the face of an adversary who wishes to fool
the model.

This tutorial will raise your awareness to the security vulnerabilities
of ML models, and will give insight into the hot topic of adversarial
machine learning. You may be surprised to find that adding imperceptible
perturbations to an image *can* cause drastically different model
performance. Given that this is a tutorial, we will explore the topic
via example on an image classifier. Specifically we will use one of the
first and most popular attack methods, the Fast Gradient Sign Attack
(FGSM), to fool an MNIST classifier.

Threat Model
------------

For context, there are many categories of adversarial attacks, each with
a different goal and assumption of the attacker’s knowledge. However, in
general the overarching goal is to add the least amount of perturbation
to the input data to cause the desired misclassification. There are
several kinds of assumptions of the attacker’s knowledge, two of which
are: **white-box** and **black-box**. A *white-box* attack assumes the
attacker has full knowledge and access to the model, including
architecture, inputs, outputs, and weights. A *black-box* attack assumes
the attacker only has access to the inputs and outputs of the model, and
knows nothing about the underlying architecture or weights. There are
also several types of goals, including **misclassification** and
**source/target misclassification**. A goal of *misclassification* means
the adversary only wants the output classification to be wrong but does
not care what the new classification is. A *source/target
misclassification* means the adversary wants to alter an image that is
originally of a specific source class so that it is classified as a
specific target class.

In this case, the FGSM attack is a *white-box* attack with the goal of
*misclassification*. With this background information, we can now
discuss the attack in detail.

Fast Gradient Sign Attack
-------------------------

One of the first and most popular adversarial attacks to date is
referred to as the *Fast Gradient Sign Attack (FGSM)* and is described
by Goodfellow et. al. in `Explaining and Harnessing Adversarial
Examples <https://arxiv.org/abs/1412.6572>`__. The attack is remarkably
powerful, and yet intuitive. It is designed to attack neural networks by
leveraging the way they learn, *gradients*. The idea is simple, rather
than working to minimize the loss by adjusting the weights based on the
backpropagated gradients, the attack *adjusts the input data to maximize
the loss* based on the same backpropagated gradients. In other words,
the attack uses the gradient of the loss w.r.t the input data, then
adjusts the input data to maximize the loss.

Before we jump into the code, let’s look at the famous
`FGSM <https://arxiv.org/abs/1412.6572>`__ panda example and extract
some notation.

.. figure:: /_static/img/fgsm_panda_image.png
   :alt: fgsm_panda_image

From the figure, $\mathbf{x}$ is the original input image
correctly classified as a “panda”, $y$ is the ground truth label
for $\mathbf{x}$, $\mathbf{\theta}$ represents the model
parameters, and $J(\mathbf{\theta}, \mathbf{x}, y)$ is the loss
that is used to train the network. The attack backpropagates the
gradient back to the input data to calculate
$\nabla_{x} J(\mathbf{\theta}, \mathbf{x}, y)$. Then, it adjusts
the input data by a small step ($\epsilon$ or $0.007$ in the
picture) in the direction (i.e.
$sign(\nabla_{x} J(\mathbf{\theta}, \mathbf{x}, y))$) that will
maximize the loss. The resulting perturbed image, $x'$, is then
*misclassified* by the target network as a “gibbon” when it is still
clearly a “panda”.

Hopefully now the motivation for this tutorial is clear, so lets jump
into the implementation.

```python
from __future__ import print_function
import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim
from torchvision import datasets, transforms
import numpy as np
import matplotlib.pyplot as plt
```

Implementation
--------------

In this section, we will discuss the input parameters for the tutorial,
define the model under attack, then code the attack and run some tests.

Inputs
~~~~~~

There are only three inputs for this tutorial, and are defined as
follows:

-  **epsilons** - List of epsilon values to use for the run. It is
   important to keep 0 in the list because it represents the model
   performance on the original test set. Also, intuitively we would
   expect the larger the epsilon, the more noticeable the perturbations
   but the more effective the attack in terms of degrading model
   accuracy. Since the data range here is $[0,1]$, no epsilon
   value should exceed 1.

-  **pretrained_model** - path to the pretrained MNIST model which was
   trained with
   `pytorch/examples/mnist <https://github.com/pytorch/examples/tree/master/mnist>`__.
   For simplicity, download the pretrained model `here <https://drive.google.com/drive/folders/1fn83DF14tWmit0RTKWRhPq5uVXt73e0h?usp=sharing>`__.

-  **use_cuda** - boolean flag to use CUDA if desired and available.
   Note, a GPU with CUDA is not critical for this tutorial as a CPU will
   not take much time.

```python
from google.colab import drive
drive.mount('/gdrive')

```

    Go to this URL in a browser: https://accounts.google.com/o/oauth2/auth?client_id=947318989803-6bn6qk8qdgf4n4g3pfee6491hc0brc4i.apps.googleusercontent.com&redirect_uri=urn%3aietf%3awg%3aoauth%3a2.0%3aoob&response_type=code&scope=email%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdocs.test%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdrive%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdrive.photos.readonly%20https%3a%2f%2fwww.googleapis.com%2fauth%2fpeopleapi.readonly

    Enter your authorization code:
    ··········
    Mounted at /gdrive

```python
epsilons = [0, .05, .1, .15, .2, .25, .3]
pretrained_model = '/gdrive/My Drive/Tmp/lenet_mnist_model.pth' #pretrained_model = "lenet_mnist_model.pth"
use_cuda=True
```

Model Under Attack
~~~~~~~~~~~~~~~~~~

As mentioned, the model under attack is the same MNIST model from
`pytorch/examples/mnist <https://github.com/pytorch/examples/tree/master/mnist>`__.
You may train and save your own MNIST model or you can download and use
the provided model. The *Net* definition and test dataloader here have
been copied from the MNIST example. The purpose of this section is to
define the model and dataloader, then initialize the model and load the
pretrained weights.

```python
# LeNet Model definition
class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.conv1 = nn.Conv2d(1, 10, kernel_size=5)
        self.conv2 = nn.Conv2d(10, 20, kernel_size=5)
        self.conv2_drop = nn.Dropout2d()
        self.fc1 = nn.Linear(320, 50)
        self.fc2 = nn.Linear(50, 10)

    def forward(self, x):
        x = F.relu(F.max_pool2d(self.conv1(x), 2))
        x = F.relu(F.max_pool2d(self.conv2_drop(self.conv2(x)), 2))
        x = x.view(-1, 320)
        x = F.relu(self.fc1(x))
        x = F.dropout(x, training=self.training)
        x = self.fc2(x)
        return F.log_softmax(x, dim=1)

# MNIST Test dataset and dataloader declaration
test_loader = torch.utils.data.DataLoader(
    datasets.MNIST('../data', train=False, download=True, transform=transforms.Compose([
            transforms.ToTensor(),
            ])),
        batch_size=1, shuffle=True)

# Define what device we are using
print("CUDA Available: ",torch.cuda.is_available())
device = torch.device("cuda" if (use_cuda and torch.cuda.is_available()) else "cpu")

# Initialize the network
model = Net().to(device)

# Load the pretrained model
model.load_state_dict(torch.load(pretrained_model, map_location='cpu'))

# Set the model in evaluation mode. In this case this is for the Dropout layers
model.eval()
```

    Downloading http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz to ../data/MNIST/raw/train-images-idx3-ubyte.gz

    HBox(children=(IntProgress(value=1, bar_style='info', max=1), HTML(value='')))

    Extracting ../data/MNIST/raw/train-images-idx3-ubyte.gz to ../data/MNIST/raw
    Downloading http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz to ../data/MNIST/raw/train-labels-idx1-ubyte.gz

    HBox(children=(IntProgress(value=1, bar_style='info', max=1), HTML(value='')))

    Extracting ../data/MNIST/raw/train-labels-idx1-ubyte.gz to ../data/MNIST/raw
    Downloading http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz to ../data/MNIST/raw/t10k-images-idx3-ubyte.gz

    HBox(children=(IntProgress(value=1, bar_style='info', max=1), HTML(value='')))

    Extracting ../data/MNIST/raw/t10k-images-idx3-ubyte.gz to ../data/MNIST/raw
    Downloading http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz to ../data/MNIST/raw/t10k-labels-idx1-ubyte.gz

    HBox(children=(IntProgress(value=1, bar_style='info', max=1), HTML(value='')))

    Extracting ../data/MNIST/raw/t10k-labels-idx1-ubyte.gz to ../data/MNIST/raw
    Processing...
    Done!
    CUDA Available:  True

    Net(
      (conv1): Conv2d(1, 10, kernel_size=(5, 5), stride=(1, 1))
      (conv2): Conv2d(10, 20, kernel_size=(5, 5), stride=(1, 1))
      (conv2_drop): Dropout2d(p=0.5, inplace=False)
      (fc1): Linear(in_features=320, out_features=50, bias=True)
      (fc2): Linear(in_features=50, out_features=10, bias=True)
    )

FGSM Attack
~~~~~~~~~~~

Now, we can define the function that creates the adversarial examples by
perturbing the original inputs. The ``fgsm_attack`` function takes three
inputs, *image* is the original clean image ($x$), *epsilon* is
the pixel-wise perturbation amount ($\epsilon$), and *data_grad*
is gradient of the loss w.r.t the input image
($\nabla_{x} J(\mathbf{\theta}, \mathbf{x}, y)$). The function
then creates perturbed image as

\begin{align}perturbed\_image = image + epsilon*sign(data\_grad) = x + \epsilon * sign(\nabla_{x} J(\mathbf{\theta}, \mathbf{x}, y))\end{align}

Finally, in order to maintain the original range of the data, the
perturbed image is clipped to range $[0,1]$.

```python
# FGSM attack code
def fgsm_attack(image, epsilon, data_grad):
    # Collect the element-wise sign of the data gradient
    sign_data_grad = data_grad.sign()
    # Create the perturbed image by adjusting each pixel of the input image
    perturbed_image = image + epsilon*sign_data_grad
    # Adding clipping to maintain [0,1] range
    perturbed_image = torch.clamp(perturbed_image, 0, 1)
    # Return the perturbed image
    return perturbed_image
```

Testing Function
~~~~~~~~~~~~~~~~

Finally, the central result of this tutorial comes from the ``test``
function. Each call to this test function performs a full test step on
the MNIST test set and reports a final accuracy. However, notice that
this function also takes an *epsilon* input. This is because the
``test`` function reports the accuracy of a model that is under attack
from an adversary with strength $\epsilon$. More specifically, for
each sample in the test set, the function computes the gradient of the
loss w.r.t the input data ($data\_grad$), creates a perturbed
image with ``fgsm_attack`` ($perturbed\_data$), then checks to see
if the perturbed example is adversarial. In addition to testing the
accuracy of the model, the function also saves and returns some
successful adversarial examples to be visualized later.

```python
def test( model, device, test_loader, epsilon, scale=1 ):

    # Accuracy counter
    correct = 0
    adv_examples = []

    # Loop over all examples in test set
    for data, target in test_loader:

        # scaling the inout
        data = scale * data
        # data = torch.clamp(data, 0, 1)

        # Send the data and label to the device
        data, target = data.to(device), target.to(device)

        # Set requires_grad attribute of tensor. Important for Attack
        data.requires_grad = True

        # Forward pass the data through the model
        output = model(data)
        init_pred = output.max(1, keepdim=True)[1] # get the index of the max log-probability

        # If the initial prediction is wrong, dont bother attacking, just move on
        if init_pred.item() != target.item():
            continue

        # Calculate the loss
        loss = F.nll_loss(output, target)

        # Zero all existing gradients
        model.zero_grad()

        # Calculate gradients of model in backward pass
        loss.backward()

        # Collect datagrad
        data_grad = data.grad.data

        # Call FGSM Attack
        perturbed_data = fgsm_attack(data, epsilon, data_grad)

        # Re-classify the perturbed image
        output = model(perturbed_data)

        # Check for success
        final_pred = output.max(1, keepdim=True)[1] # get the index of the max log-probability
        if final_pred.item() == target.item():
            correct += 1
            # Special case for saving 0 epsilon examples
            if (epsilon == 0) and (len(adv_examples) < 5):
                adv_ex = perturbed_data.squeeze().detach().cpu().numpy()
                adv_examples.append( (init_pred.item(), final_pred.item(), adv_ex) )
        else:
            # Save some adv examples for visualization later
            if len(adv_examples) < 5:
                adv_ex = perturbed_data.squeeze().detach().cpu().numpy()
                adv_examples.append( (init_pred.item(), final_pred.item(), adv_ex) )

    # Calculate final accuracy for this epsilon
    final_acc = correct/float(len(test_loader))
    print("Epsilon: {}\tTest Accuracy = {} / {} = {}".format(epsilon, correct, len(test_loader), final_acc))

    # Return the accuracy and an adversarial example
    return final_acc, adv_examples
```

Run Attack
~~~~~~~~~~

The last part of the implementation is to actually run the attack. Here,
we run a full test step for each epsilon value in the *epsilons* input.
For each epsilon we also save the final accuracy and some successful
adversarial examples to be plotted in the coming sections. Notice how
the printed accuracies decrease as the epsilon value increases. Also,
note the $\epsilon=0$ case represents the original test accuracy,
with no attack.

```python
scales = [.5, 1, 2, 5, 10, 100]
examples = []

all_res = []
# Run test for each epsilon
for s in scales:
    print(f'sclae={s}')
    accuracies = []
    for eps in epsilons:
        acc, ex = test(model, device, test_loader, eps, s)
        accuracies.append(acc)
#         examples.append(ex)
    all_res.append(accuracies)
```

    sclae=0.5
    Epsilon: 0	Test Accuracy = 9744 / 10000 = 0.9744
    Epsilon: 0.05	Test Accuracy = 7925 / 10000 = 0.7925
    Epsilon: 0.1	Test Accuracy = 3031 / 10000 = 0.3031
    Epsilon: 0.15	Test Accuracy = 674 / 10000 = 0.0674
    Epsilon: 0.2	Test Accuracy = 192 / 10000 = 0.0192
    Epsilon: 0.25	Test Accuracy = 92 / 10000 = 0.0092
    Epsilon: 0.3	Test Accuracy = 63 / 10000 = 0.0063
    sclae=1
    Epsilon: 0	Test Accuracy = 9810 / 10000 = 0.981
    Epsilon: 0.05	Test Accuracy = 9426 / 10000 = 0.9426
    Epsilon: 0.1	Test Accuracy = 8510 / 10000 = 0.851
    Epsilon: 0.15	Test Accuracy = 6826 / 10000 = 0.6826
    Epsilon: 0.2	Test Accuracy = 4301 / 10000 = 0.4301
    Epsilon: 0.25	Test Accuracy = 2082 / 10000 = 0.2082
    Epsilon: 0.3	Test Accuracy = 869 / 10000 = 0.0869
    sclae=2
    Epsilon: 0	Test Accuracy = 9777 / 10000 = 0.9777
    Epsilon: 0.05	Test Accuracy = 9448 / 10000 = 0.9448
    Epsilon: 0.1	Test Accuracy = 8742 / 10000 = 0.8742
    Epsilon: 0.15	Test Accuracy = 7423 / 10000 = 0.7423
    Epsilon: 0.2	Test Accuracy = 5305 / 10000 = 0.5305
    Epsilon: 0.25	Test Accuracy = 2938 / 10000 = 0.2938
    Epsilon: 0.3	Test Accuracy = 1355 / 10000 = 0.1355
    sclae=5
    Epsilon: 0	Test Accuracy = 9725 / 10000 = 0.9725
    Epsilon: 0.05	Test Accuracy = 9335 / 10000 = 0.9335
    Epsilon: 0.1	Test Accuracy = 8596 / 10000 = 0.8596
    Epsilon: 0.15	Test Accuracy = 7289 / 10000 = 0.7289
    Epsilon: 0.2	Test Accuracy = 5206 / 10000 = 0.5206
    Epsilon: 0.25	Test Accuracy = 2961 / 10000 = 0.2961
    Epsilon: 0.3	Test Accuracy = 1388 / 10000 = 0.1388
    sclae=10
    Epsilon: 0	Test Accuracy = 9689 / 10000 = 0.9689
    Epsilon: 0.05	Test Accuracy = 9267 / 10000 = 0.9267
    Epsilon: 0.1	Test Accuracy = 8491 / 10000 = 0.8491
    Epsilon: 0.15	Test Accuracy = 7121 / 10000 = 0.7121
    Epsilon: 0.2	Test Accuracy = 5114 / 10000 = 0.5114
    Epsilon: 0.25	Test Accuracy = 2994 / 10000 = 0.2994
    Epsilon: 0.3	Test Accuracy = 1404 / 10000 = 0.1404
    sclae=100
    Epsilon: 0	Test Accuracy = 9641 / 10000 = 0.9641
    Epsilon: 0.05	Test Accuracy = 9174 / 10000 = 0.9174
    Epsilon: 0.1	Test Accuracy = 8308 / 10000 = 0.8308
    Epsilon: 0.15	Test Accuracy = 6929 / 10000 = 0.6929
    Epsilon: 0.2	Test Accuracy = 4941 / 10000 = 0.4941
    Epsilon: 0.25	Test Accuracy = 2867 / 10000 = 0.2867
    Epsilon: 0.3	Test Accuracy = 1379 / 10000 = 0.1379

```python
scales = [.5, 1, 2, 5, 10, 100]
examples = []

all_res = []
# Run test for each epsilon
for s in scales:
    print(f'sclae={s}')
    accuracies = []
    for eps in epsilons:
        acc, ex = test(model, device, test_loader, eps, s)
        accuracies.append(acc)
#         examples.append(ex)
    all_res.append(accuracies)
```

Results
-------

Accuracy vs Epsilon
~~~~~~~~~~~~~~~~~~~

The first result is the accuracy versus epsilon plot. As alluded to
earlier, as epsilon increases we expect the test accuracy to decrease.
This is because larger epsilons mean we take a larger step in the
direction that will maximize the loss. Notice the trend in the curve is
not linear even though the epsilon values are linearly spaced. For
example, the accuracy at $\epsilon=0.05$ is only about 4% lower
than $\epsilon=0$, but the accuracy at $\epsilon=0.2$ is 25%
lower than $\epsilon=0.15$. Also, notice the accuracy of the model
hits random accuracy for a 10-class classifier between
$\epsilon=0.25$ and $\epsilon=0.3$.

```python
symbs = ['*-', 'o-', 's-', 'd-', '+-', 'x-', '^-', '<-']
plt.figure(figsize=(5,5))

for idx, accuracies in enumerate(all_res):
  plt.plot(epsilons, accuracies, symbs[idx])
plt.yticks(np.arange(0, 1.1, step=0.1))
plt.xticks(np.arange(0, .35, step=0.05))
plt.title("Accuracy vs Epsilon")
plt.xlabel("Epsilon")
plt.ylabel("Accuracy")
plt.legend(scales)
plt.show()
```

Sample Adversarial Examples
~~~~~~~~~~~~~~~~~~~~~~~~~~~

Remember the idea of no free lunch? In this case, as epsilon increases
the test accuracy decreases **BUT** the perturbations become more easily
perceptible. In reality, there is a tradeoff between accuracy
degredation and perceptibility that an attacker must consider. Here, we
show some examples of successful adversarial examples at each epsilon
value. Each row of the plot shows a different epsilon value. The first
row is the $\epsilon=0$ examples which represent the original
“clean” images with no perturbation. The title of each image shows the
“original classification -> adversarial classification.” Notice, the
perturbations start to become evident at $\epsilon=0.15$ and are
quite evident at $\epsilon=0.3$. However, in all cases humans are
still capable of identifying the correct class despite the added noise.

```python
# Plot several examples of adversarial samples at each epsilon
cnt = 0
plt.figure(figsize=(8,10))
for i in range(len(epsilons)):
    for j in range(len(examples[i])):
        cnt += 1
        plt.subplot(len(epsilons),len(examples[0]),cnt)
        plt.xticks([], [])
        plt.yticks([], [])
        if j == 0:
            plt.ylabel("Eps: {}".format(epsilons[i]), fontsize=14)
        orig,adv,ex = examples[i][j]
        plt.title("{} -> {}".format(orig, adv))
        plt.imshow(ex, cmap="gray")
plt.tight_layout()
plt.show()
```

Where to go next?
-----------------

Hopefully this tutorial gives some insight into the topic of adversarial
machine learning. There are many potential directions to go from here.
This attack represents the very beginning of adversarial attack research
and since there have been many subsequent ideas for how to attack and
defend ML models from an adversary. In fact, at NIPS 2017 there was an
adversarial attack and defense competition and many of the methods used
in the competition are described in this paper: `Adversarial Attacks and
Defences Competition <https://arxiv.org/pdf/1804.00097.pdf>`__. The work
on defense also leads into the idea of making machine learning models
more *robust* in general, to both naturally perturbed and adversarially
crafted inputs.

Another direction to go is adversarial attacks and defense in different
domains. Adversarial research is not limited to the image domain, check
out `this <https://arxiv.org/pdf/1801.01944.pdf>`__ attack on
speech-to-text models. But perhaps the best way to learn more about
adversarial machine learning is to get your hands dirty. Try to
implement a different attack from the NIPS 2017 competition, and see how
it differs from FGSM. Then, try to defend the model from your own
attacks.