src/core/optimization/Adagrad.java
public void optimize(Matrix matrix, Matrix matrixGradient) throws MatrixException, DynamicParamException {
        if (m2Sum == null) m2Sum = new HashMap<>();

        Matrix dM2Sum;
        if (m2Sum.containsKey(matrix)) dM2Sum = m2Sum.get(matrix);
        else m2Sum.put(matrix, dM2Sum = new DMatrix(matrix.getRows(), matrix.getColumns()));

        dM2Sum.add(matrixGradient.multiply(matrixGradient), dM2Sum);

        double epsilon = 10E-8;
        matrix.subtract(matrixGradient.divide(dM2Sum.add(epsilon).apply(UnaryFunctionType.SQRT)).multiply(learningRate), matrix);
    }
/**
     * Optimizes single matrix (M) using calculated matrix gradient (dM).<br>
     * Matrix can be for example weight or bias matrix with gradient.<br>
     *
     * @param matrix matrix to be optimized.
     * @param matrixGradient matrix gradients for optimization step.
     * @throws MatrixException throws exception if matrix operation fails.
     * @throws DynamicParamException throws exception if parameter (params) setting fails.
     */
Optimizes single matrix (M) using calculated matrix gradient (dM).
Matrix can be for example weight or bias matrix with gradient.

@param matrix matrix to be optimized.
@param matrixGradient matrix gradients for optimization step.
@throws MatrixException throws exception if matrix operation fails.
@throws DynamicParamException throws exception if parameter (params) setting fails.
Optimizes single matrix ( M ) using calculated matrix gradient ( dM ) . Matrix can be for example weight or bias matrix with gradient . @param matrix matrix to be optimized . @param matrixGradient matrix gradients for optimization step . @throws MatrixException throws exception if matrix operation fails . @throws DynamicParamException throws exception if parameter ( params ) setting fails .
public void optimize(Matrix matrix, Matrix matrixGradient) throws MatrixException, DynamicParamException {
        if (m2Sum == null) m2Sum = new HashMap<>();
        Matrix dM2Sum;
        if (m2Sum.containsKey(matrix)) dM2Sum = m2Sum.get(matrix);
        else m2Sum.put(matrix, dM2Sum = new DMatrix(matrix.getRows(), matrix.getColumns()));
        dM2Sum.add(matrixGradient.multiply(matrixGradient), dM2Sum);
        double epsilon = 10E-8;
        matrix.subtract(matrixGradient.divide(dM2Sum.add(epsilon).apply(UnaryFunctionType.SQRT)).multiply(learningRate), matrix);
    }
public void optimize ( Matrix matrix , Matrix matrixGradient ) throws MatrixException , DynamicParamException { if ( m2Sum == null ) m2Sum = new HashMap < > ( ) ; Matrix dM2Sum ; if ( m2Sum . containsKey ( matrix ) ) dM2Sum = m2Sum . get ( matrix ) ; else m2Sum . put ( matrix , dM2Sum = new DMatrix ( matrix . getRows ( ) , matrix . getColumns ( ) ) ) ; dM2Sum . add ( matrixGradient . multiply ( matrixGradient ) , dM2Sum ) ; double epsilon = 10E-8 ; matrix . subtract ( matrixGradient . divide ( dM2Sum . add ( epsilon ) . apply ( UnaryFunctionType . SQRT ) ) . multiply ( learningRate ) , matrix ) ; }
Optimizes single matrix (M) using calculated matrix gradient (dM).<br>
 Matrix can be for example weight or bias matrix with gradient.<br>
Optimizes single matrix ( M ) using calculated matrix gradient ( dM ) . <br > Matrix can be for example weight or bias matrix with gradient . <br >