#3.2 Continunity and Differentiation

##Limits and Continuity


```python
from sympy import *
x = symbols('x')
f = log(x)/x
y = limit(f,x,0)
print('f(x)=', f)
print(y)
```

    f(x)= log(x)/x
    -oo



```python
x = symbols('x')
f = (1/x)**2
y = limit(f,x,2)
print('f(x)=', f)
print(y)
```

    f(x)= x**(-2)
    1/4



```python
x = symbols('x')
f = (5*x**3 - 2*x**2 + 1)/(-3*x**3 + 4*x +8)
y = limit(f,x,0)
print('f(x)=', f)
print(y)
```

    f(x)= (5*x**3 - 2*x**2 + 1)/(-3*x**3 + 4*x + 8)
    1/8


Derivatives

### Single-Variable Case


```python
import sympy as sp
x=sp.Symbol('x')
f=sp.sin(x)+x**2+sp.exp(4*x)
print('f(x) =',f)

df=sp.diff(f,x)
print('First derivative:', df)

ddf=sp.diff(f,x,2)
print('Second derivative:',ddf)
```

    f(x) = x**2 + exp(4*x) + sin(x)
    First derivative: 2*x + 4*exp(4*x) + cos(x)
    Second derivative: 16*exp(4*x) - sin(x) + 2



```python
x=sp.Symbol('x')
f=(5*x**3 + 6*x - 9)/(log(3*x/2))
print('f(x) =',f)

df=sp.diff(f,x)
print('First derivative:', df)

ddf=sp.diff(f,x,2)
print('Second derivative:', ddf)
```

    f(x) = (5*x**3 + 6*x - 9)/log(3*x/2)
    First derivative: (15*x**2 + 6)/log(3*x/2) - (5*x**3 + 6*x - 9)/(x*log(3*x/2)**2)
    Second derivative: (30*x - 6*(5*x**2 + 2)/(x*log(3*x/2)) + (1 + 2/log(3*x/2))*(5*x**3 + 6*x - 9)/(x**2*log(3*x/2)))/log(3*x/2)


### General Case

Partial Derivative


```python
import sympy as sym
x, y = sym.symbols('x y')
f = x**4 + x*y**4
partial_df = f.diff(x, y)
print('Partial derivative:', partial_df)
```

    Partial derivative: 4*y**3



```python
from sympy import symbols, cos, diff
x, y, z = symbols('x y z', real=True)
f = 7*x*y - y*log(z) + z**3 + z**8*y

#differntiating function f  respect to a
print('Partial derivative wrt x', diff(f, x))
print('Partial derivative wrt y', diff(f, y))
print('Partial derivative wrt z', diff(f, z))
```

    Partial derivative wrt x 7*y
    Partial derivative wrt y 7*x + z**8 - log(z)
    Partial derivative wrt z 8*y*z**7 - y/z + 3*z**2


Jacobian


```python
!pip install symengine
```

    Collecting symengine
      Downloading symengine-0.9.0-cp37-cp37m-manylinux2010_x86_64.whl (37.5 MB)
    [K     |████████████████████████████████| 37.5 MB 3.0 MB/s 
    [?25hInstalling collected packages: symengine
    Successfully installed symengine-0.9.0



```python
import symengine
vars = symengine.symbols('x y') 
f = symengine.sympify(['x**4 + 3*(y**2)*x', '5*y**2 + 2*x*y + 1']) 
J = symengine.zeros(len(f),len(vars)) 
for i, fi in enumerate(f):
  for j, s in enumerate(vars):
    J[i,j] = symengine.diff(fi, s)
print(J)
```

    [4*x**3 + 3*y**2, 6*x*y]
    [2*y, 2*x + 10*y]
    



```python
print(symengine.Matrix.det(J))
```

    -12*x*y**2 + (4*x**3 + 3*y**2)*(2*x + 10*y)



```python
import symengine
vars = symengine.symbols('x y z') 
f = symengine.sympify(['z*tan(x**2 - y**2)', 'x*y*ln(z/2)']) 
J = symengine.zeros(len(f),len(vars)) 
for i, fi in enumerate(f):
  for j, s in enumerate(vars):
    J[i,j] = symengine.diff(fi, s)
print(J)
```

    [2*x*z*(1 + tan(-x**2 + y**2)**2), -2*y*z*(1 + tan(-x**2 + y**2)**2), -tan(-x**2 + y**2)]
    [y*log((1/2)*z), x*log((1/2)*z), x*y/z]
    


Chain Rule


```python
import sympy as sym
x = sym.symbols('x')
sym.diff((x**2 - 3*x + 5)**3)
```




$\displaystyle \left(6 x - 9\right) \left(x^{2} - 3 x + 5\right)^{2}$



Directional Derivative


```python
!pip install numdifftools
```

    Collecting numdifftools
      Downloading numdifftools-0.9.40-py2.py3-none-any.whl (99 kB)
    [K     |████████████████████████████████| 99 kB 3.5 MB/s 
    [?25hRequirement already satisfied: scipy>=0.8 in /usr/local/lib/python3.7/dist-packages (from numdifftools) (1.4.1)
    Requirement already satisfied: numpy>=1.9 in /usr/local/lib/python3.7/dist-packages (from numdifftools) (1.21.5)
    Collecting algopy>=0.4
      Downloading algopy-0.5.7.zip (189 kB)
    [K     |████████████████████████████████| 189 kB 34.2 MB/s 
    [?25hRequirement already satisfied: statsmodels>=0.6 in /usr/local/lib/python3.7/dist-packages (from numdifftools) (0.10.2)
    Requirement already satisfied: pandas>=0.19 in /usr/local/lib/python3.7/dist-packages (from statsmodels>=0.6->numdifftools) (1.3.5)
    Requirement already satisfied: patsy>=0.4.0 in /usr/local/lib/python3.7/dist-packages (from statsmodels>=0.6->numdifftools) (0.5.2)
    Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.7/dist-packages (from pandas>=0.19->statsmodels>=0.6->numdifftools) (2018.9)
    Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.7/dist-packages (from pandas>=0.19->statsmodels>=0.6->numdifftools) (2.8.2)
    Requirement already satisfied: six in /usr/local/lib/python3.7/dist-packages (from patsy>=0.4.0->statsmodels>=0.6->numdifftools) (1.15.0)
    Building wheels for collected packages: algopy
      Building wheel for algopy (setup.py) ... [?25l[?25hdone
      Created wheel for algopy: filename=algopy-0.5.7-py3-none-any.whl size=107608 sha256=2fd27c631bb35cb9dc8a0a0cc8393797fd89190b72ffab52e9e6a709d0bb900a
      Stored in directory: /root/.cache/pip/wheels/a8/fd/06/21dc6aba9f680fdbde386c06b1a8a1cf3d9a5f6bd96e31dcd4
    Successfully built algopy
    Installing collected packages: algopy, numdifftools
    Successfully installed algopy-0.5.7 numdifftools-0.9.40



```python
import numpy as np
import numdifftools as nd
vec = np.r_[1, 2]
f = lambda x: (5*x**3 + 6*x - 9)/(3*x**3/2)
re, info = nd.directionaldiff(f, [2, -3], vec, full_output=True)
print('The directional derivative of f at point (2,-3) in the direction vector (1,2) is', re)
```

    The directional derivative of f at point (2,-3) in the direction vector (1,2) is [0.0559017  0.46377706]



```python
print(info)
```

    info(f_value=array([3.58333333, 4.        ]), error_estimate=array([2.07216189e-13, 4.87776486e-13]), final_step=array([0.0078125, 0.0078125]), index=array([12, 13]))


Hessian


```python
import numdifftools.nd_algopy as nda
f = lambda x: (5*x[0]**3 + 6*x[1] - 9)/(3*x[0]**3/2)
hess_f = nda.Hessian(f)
hess = hess_f([1,1]) #minimize at (1,1)
hess
```




    array([[-24., -12.],
           [-12.,   0.]])



##Taylor’s Theorem


```python
import math
def tay_series_in_x(n):
    x = math.pi/4
    tay_se = 0
    for i in range(n):
        tay_se += ((-1)**n * (x)**(2*n+1)) / np.math.factorial(2*n+1)
    return tay_se
```


```python
tay_series_in_x(4)
```




    1.253446756151248e-06



#3.3 Unconstrained Optimization

##Necessary and Sufficent Conditions of Local Minimizers

Local Minimizer


```python
from scipy.optimize import minimize
f = lambda x: (5*x[0]**3 + 6*x[0]**2 + x[0] - 9)
x0 = [1, 2.1, 4]
loc_min = minimize(f, x0, method='Nelder-Mead')
loc_min
```




     final_simplex: (array([[-0.0945155 ,  2.75647652,  4.90846093],
           [-0.0944733 ,  2.75646613,  4.90839586],
           [-0.09452481,  2.75649113,  4.90847296],
           [-0.0944599 ,  2.75644522,  4.908496  ]]), array([-9.04513804, -9.04513804, -9.04513804, -9.04513804]))
               fun: -9.045138041306283
           message: 'Optimization terminated successfully.'
              nfev: 96
               nit: 49
            status: 0
           success: True
                 x: array([-0.0945155 ,  2.75647652,  4.90846093])



Global Minimizer


```python
from scipy.optimize import basinhopping
f = lambda x: (5*x[0]**3 + 6*x[0]**2 + x[0] - 9)
x0 = [1]
glo_min = basinhopping(f, x0)
glo_min
```




                            fun: -1.6849058174187942e+26
     lowest_optimization_result:       fun: -1.6849058174187942e+26
     hess_inv: array([[1]])
          jac: array([0.])
      message: 'Optimization terminated successfully.'
         nfev: 3
          nit: 0
         njev: 1
       status: 0
      success: True
            x: array([-3.22999519e+08])
                        message: ['requested number of basinhopping iterations completed successfully']
          minimization_failures: 0
                           nfev: 375
                            nit: 100
                           njev: 125
                              x: array([-3.22999519e+08])



##Convexity and Global Minimizers

###Global Minimizers of Convex Functions


```python
from scipy.optimize import minimize
from numpy.random import rand
import sympy as sp
def func(x):
	return sp.sin(x[0])+x[1]**2
def derivative(x):
	return np.array([np.cos(x[0]), 2*x[1]])
a, b = -3,3
pt = a + rand(2) * a - (b + 1/2)
result = minimize(func, pt, method='BFGS', jac=derivative)
result
```




          fun: -0.9999999999999819
     hess_inv: array([[1.00139636e+00, 5.28874735e-06],
           [5.28874735e-06, 4.99776051e-01]])
          jac: array([-1.18120478e-07, -2.11052302e-07])
      message: 'Optimization terminated successfully.'
         nfev: 6
          nit: 5
         njev: 6
       status: 0
      success: True
            x: array([-7.85398175e+00, -1.05526151e-07])




```python
print('Status : %s' % result['message'])
print('Total Evaluations: %d' % result['nfev'])
solution = result['x']
evaluation = func(solution)
print('Solution: f(%s) = %.5f' % (solution, evaluation))
```

    Status : Optimization terminated successfully.
    Total Evaluations: 6
    Solution: f([-7.85398175e+00 -1.05526151e-07]) = -1.00000



```python
def func(x):
	return sp.sin(x[0])+x[0]**2
def derivative(x):
	return np.array([np.cos(x[0]), 2*x[0]])
x0=rand(2)
result = minimize(func, x0, method='Newton-CG', jac=derivative)
result
```




         fun: 1.80937198308175
         jac: None
     message: "Warning: CG iterations didn't converge.  The Hessian is not positive definite."
        nfev: 1
        nhev: 0
         nit: 0
        njev: 81
      status: 3
     success: False
           x: array([0.98732741, 0.73207205])




```python
print('Status : %s' % result['message'])
print('Total Evaluations: %d' % result['nfev'])
solution = result['x']
evaluation = func(solution)
print('Solution: f(%s) = %.5f' % (solution, evaluation))
```

    Status : Warning: CG iterations didn't converge.  The Hessian is not positive definite.
    Total Evaluations: 1
    Solution: f([0.98732741 0.73207205]) = 1.80937


8821540a9fe3a7166c20697eb523951e26f46244
ipynb
Jupyter Notebook
Module_E.ipynb
Caressiw86/pergro
9d6befa4b05c631688174331108f9693a95252e5
MIT
Module_E.ipynb
Caressiw86/pergro
9d6befa4b05c631688174331108f9693a95252e5
MIT
Module_E.ipynb
Caressiw86/pergro
9d6befa4b05c631688174331108f9693a95252e5
MIT
Qwen/Qwen-72B
1. YES
2. YES