**Implementation**

Our software provides the user with a package he or she can use to evaluate functions of multiple variables at a specified point while at the same time evaluating the partial derivatives of the function at that same point. A class is provided to perform simple calculations, and two functions are provided to handle vector inputs and outputs.

Our strategy has been to create input-variable objects in python that once defined may be used in python calculations as if they were standard python variables of type float. However, when calculations are done with these AutDiff objects, a partial derivative with respect to one of the input variables is calculated and carried along. The object representing the result then contains an attribute that contains the value of the function at the specified point (val) and an attribute that contains a partial derivative with respect to one of the input variables (derv). Which partial derivative is calculated is specified by the user by intiating the derv attribute of one input variable to 1 and the derv attributes of the other input variables to 0.  The calculation can then be repeated to get partial derivatives with respect to other input variables by changing which input variable’s derv attribute is intiated to 1. A function *all_derivatives( )* is supplied to automate this process.

*The AutDiff( ) class*

Most of the functionality of our package is implemented in a single class called AutoDiff( ). Fully functional code for this class can be found in the module 'AutoDiff.py' in the *code* directory. It has two attributes, self.val and self.derv, that respectively hold the value of the function being evaluated and the partial derivative of the function being evaluated up to some point in the calculation.

Autodiff() has many methods. Each is short and designed to replace a standard mathematical operation. Thus, for example there is a method \_\_add\_\_ that will overload the standard addition operator (+) and thus specify how objects of type AutoDiff will be added to scalar values or to each other. And similar methods are included to overload the other standard mathematical operators (- * / ^ +). Also, methods are included to replace the 
numpy functions (*exp, ln, log, sin, cos, tan, sqrt, log10, $a^{x}$*) so that these functions may be used as well in any calculations involving AutoDiff objects.

Central to all Autodiff( ) methods is the principle that each method returns a new AutDiff object that will represent the next stage in the function’s evaluation. For example, if x is an AutDiff object, and it is multiplied by the scalar 2 (x\*2), then x’s \_\_mul\_\_ method is called, and it returns a new object that will represent x\*2. The AutoDiff.\_\_mul\_\_ method performs this calculation as follows. It assumes that 2 is another object of type AutoDiff, and trys to multiply the two AutoDiff objects together. It finds that 2 is not an AutoDiff object: it falls to its except block, and then it makes two calculations: 2 * x.val and 2 * x.derv and stores these values in the returned object’s val and derv attributes. If the 2 had been to the right of x (2*x), then the \_\_rmul\_\_ method would have been called, and it would have executed the same operation.  

More subtle is the case where two AutoDiff objects, say x and y, are multiplied together x\*y. Here x’s \_\_mul\_\_ method is called. It finds that the two objects are instances of AutoDiff, and it 1) multiplies x.val and y.val together and stores the result as the returned object’s val attribute, and 2) calculates (x.derv\*y.val + y.derv\*x.val) and store this value as the returned object’s derv attribute. Notice in the second calculation the derivatives from both objects x and y are carried along as a sum. Recall, however, that the derv value of either x or y will have been initiated to 0, so one term in the sum (x.derv\*y.val + y.derv\*x.val) will always evaluate to 0. In this way, the algorithm can automatically choose the proper derivative to carry along for subsequent calculations (here either that of x or that of y) when two Autodiff objects collide. This same approach can be used more generally and will be used when overloading the division and power operators.

An example of a elementary mathematical function implemented in AutoDiff( ) is the exponential function. This works as follows. For a given AutoDiff object, say x, when exp(x) is executed, x’s exp() function is called on itself. It uses numpy’s exp() function to evaluate np.exp(x.val) and store this value in the return object’s val attribute. It then evaluates the derivative of exp(x.value) —which in this case is just exp(x.value)— and stores this value in the return object’s derv attribute. This same approach is used for all the elementary functions.

Autdiff objects are fully functional replacements for python float variables such that once the AutoDiff class is defined, these objects can be used to perform forward autodifferentiation on any multi-input, single output function at any specified point. 



*The all_derivatives() function*

This function has yet to be implemented. It will allow the user to calculate all partial derivatives at once without having to manually initiate the derv attributes of the input variables several times and each time repeat the calculation.

Psuedo code for this function is listed below. It takes advantage of the AutoDiff( ) class.


    def all_derivatives(func, in_vars, in_vals):
        ***
        Inputs:

            func():   —the function to be evaluated
            in_vars:  —a vector containing the input variables.
            in_vals:  —a vector of values, one for each variable in in_vars that specify  
                         a point where the function and its partial derivatives are to be evaluated.
        Returns:

            func():    —the function evaluated
            val:       —the value of func( ) at the input point given by in_vals
            derv_vals: —a vector of partial derivative values, one for each input variable in in_vars.         	        
        ***

        Steps:
        * make a vector diff_vars which  coorepsonds to in_vars but all
          values are AutoDiff objects.
        * Set the derv value of the first element of diff_vars to 1 and the derv
          values of the rest of the diff_vars elements to 0.

        * Create the derv_vals vector that will hold the calculated partial derivatives

        * For i in range( the number of elements in in_vars):
            * Set the derv value of each element of in_vars to 0
            * Set  the derv value of the ith element to 1.
            * Evaluate func()  

            * If it is the first time thought the loop:
                  set val to the val of the AutoDiff output object
            * Else:
                * Store derivative of the output AutoDiff object as the ith element of derv_vals

        * Return  func, val, derv_vals


*The multi_func_all_derivatives() function*

This has yet to be implemented. I will handle vector inputs and outputs. It will allow the user to specify: a vector of functions, a vector of input variables, and a vectors of values defining a specific point. It will return: the input vector containing the functions, a vector of output values (one for each function evaluated at the specified point), and a Hessian matrix of partial derivatives containing partial derivatives of all functions evaluated at the specified point. Consider the equation below as an example:

\begin{equation}
\begin{pmatrix} f(x,y) \\ g(x,y)\end{pmatrix} =
\begin{pmatrix} 2xy \\ x+y^2 \end{pmatrix} at \begin{pmatrix} x=a \\ y=b \end{pmatrix}
\end{equation}


Here the multi_func_all_derivatives() function would return:  
1) funcs, a vector of the functions that were evaluated ($2xy$,  $x+y^2$)  
2) out_vals, a vector of the values of each function after evaluation $(f(a,b), g(a,b))$  
3) out_derv_matrix, a matrix containing the partial derivatives evaluated at the point (a, b)

Pseudo code for this function is below. It takes advantge of the AutoDiff( ) class and the all_derivatives( ) function:

    def multi_func_all_derivatives(funcs, in_vars, in_vals):
        ***
        Inputs: 
            funcs:    —a vector of input functions
            in_vars: —a vector of input variables
            in_vals:  —a vector of input values specifying a point.

        Returns:
            funcs: 	   —the vector of input functions
            out_vals —a vector of output values, one for each function
            out_derv_matrix —the Hessian matrix of partial derivatives evaluated at the input point					 
        ***
        Steps:
        •	Create an empty vector out_vals
        •	Create an empty matrix out_derv_matrix
        •	For each function in funcs:
                - Run all_derivatives(func, in_vars, in_vals) 
                - Store the returned val in the vector out_vals
                - Store the returned vector derv_vals in the matrix out_derv_matrix

        •	Return funcs, out_vals, out_derv_matrix

*Testing*

We have implemented a series of tests to test the AutoDiff( ) class. They are designed to be run by *pytest* and to provide full code coverage. They can be found in the module 'test_AutoDiff.py'. The tests are divided into the following sections: *Testing Urnary, Testing Addition, Testing Subtraction, Testing Multiplication, Testing Division, Testing Powers, Testing trig functions, Testing exp, log, sqrt, log10, Testing exp, log, sqrt, log10, Testing Complicated functions*. Travis CI and codecov inidicate that all tests are currently passed by our code and our code coverage is 100%.


```python

```

3ef266db417ba59b5dfcfa98205fbe02380387b8
ipynb
Jupyter Notebook
docs/Implementation_for_milestone2.ipynb
IACS-CS-207-FantasticFour/cs207-FinalProject
3e916af3f1a38fe919a27eff014bb5ab6ee1bb74
MIT
docs/Implementation_for_milestone2.ipynb
IACS-CS-207-FantasticFour/cs207-FinalProject
3e916af3f1a38fe919a27eff014bb5ab6ee1bb74
MIT
2019-10-30T04:17:59.000Z
2021-12-13T20:28:45.000Z
docs/Implementation_for_milestone2.ipynb
IACS-CS-207-FantasticFour/cs207-FinalProject
3e916af3f1a38fe919a27eff014bb5ab6ee1bb74
MIT
Qwen/Qwen-72B
1. YES
2. YES