<p>Gradient ascent/descent can only find <em>local</em> optima, in order to find "global" optima you just run that procedure many times with random initialization, and take the best value you find.</p>

<p>You can do the same in your situation as well: take random initial points and follow the gradient, stopping at convergence or when you step outside the domain.</p>

<p>You can make this a bit faster by dynamically restricting the domain when you step out of it. For example, suppose you are maximizing between -10 and 10, and choose 6 as an initial point; you run gradient ascent and reach 10. You can now exclude the interval [6,10] from the random initialization, since you know you will end up reaching 10 and stopping there.</p>

<p>But I would actually advise you to use <a href="https://arxiv.org/abs/1807.02811" rel="nofollow noreferrer">Bayesian optimization</a>. Its advantages over gradient ascent/descent are:</p>

<ol>
<li>does not require gradient</li>
<li>made for global optimization</li>
<li>allows to set bounds on parameters</li>
<li>requires much fewer function evaluations</li>
</ol>

<p>Finally, obligatory word of caution: this problem cannot be solved <em>in the general case</em>, consider e.g. a function that equals <code>1</code> at <code>x=3.4131242351</code>, and <code>0</code> everywhere else. However, in practice, you should be fine.</p>
Algorithms to find min/max of a single variable function in fixed domain
<p>I was looking for a <strong>numerical</strong> algorithm to find <strong>global</strong> minimum or maximum of a function in "given interval [a, b]", for example finding minimum and maximum of function </p>

<blockquote>
  <p>f(x) = sin(x)</p>
</blockquote>

<p>in domain [3*pi/4, 5*pi/4].</p>

<p>I know how to find global min/max of a multi-variable function using Gradient Descent or Gradient Ascend, but I'm only able to use these algorithms on entire function domain, for example when I use GD on function sin(x), it gives me -1 which is correct for domain [0, 2*pi] not [3*pi/4, 5*pi/4], any help?</p>

<p>I have reached to this solution so far (code in python 2.7, language isn't important, my questions is about Algorithms):</p>

<pre><code>import math
import random

# function
def f(x):
    return math.sin(x)

# xmin-xmax interval
xmin = 3.0 * math.pi / 4.0
xmax = 5.0 * math.pi / 4.0

# find ymin-ymax
steps = 10000
ymin = f(xmin)
ymax = ymin

for i in range(steps):
    x = xmin + (xmax - xmin) * float(i) / steps
    y = f(x)
    if y &lt; ymin: ymin = y
    if y &gt; ymax: ymax = y

print ymin
print ymax
</code></pre>

<p><strong>answer</strong></p>

<p>thanks to @BlackBear, I wrote a program that does what i actually need, this function searches through interval [a, b] using Gradient Descent algorithm, on each loop it start with a new random starting point between a and b, then compares the values, at the end it returns the x where the minimum occurs</p>

<pre><code>double gradientDescentInterval(const char *expression, double a, double b, double ete, double ere, double gamma,
                               unsigned int maxiter, int mode) {
    /*
     * Gradient descent is a first-order iterative optimization algorithm for finding the minimum of a function.
     * To find a local minimum of a function using gradient descent, one takes steps proportional to the negative of
     * the gradient (or approximate gradient) of the function at the current point.
     *
     * This function searches minimum on an interval [a, b]
     *
     * ARGUMENTS:
     * expressions  the function expression, it must be a string array like "x^2+1"
     * a            starting point of interval [a, b]
     * b            ending point of interval [a, b]
     * ete          estimated true error
     * ere          estimated relative error
     * gamma        step size (also known as learning rate)
     * maxiter      maximum iteration threshold
     * mode         show process {0: no, 1: yes}
     *
     */

    // fix interval reverse
    if (a &gt; b) {
        double temp = a;
        a = b;
        b = temp;
    } // end of if

    // check error thresholds
    if (ere &lt; 0 || ete &lt; 0) {
        printf("\nError: ete or ere argument is not valid\n");
        Exit();
        exit(EXIT_FAILURE);
    } // end of if

    // check mode
    if (mode != 0 &amp;&amp; mode != 1) {
        printf("\nError: mode argument is not valid\n");
        Exit();
        exit(EXIT_FAILURE);
    } // end of if

    // check maxiter to be more than zero
    if (maxiter &lt;= 0) {
        printf("Error: argument maxiter must be more than zero!\n");
        Exit();
        exit(EXIT_FAILURE);
    } // end of maxiter check

    // initializing variables
    unsigned int iter = 0, innerIter = 0;
    // choose an arbitrary result at midpoint between a and b to be updated later
    double coefficient = (b - a), result = a + coefficient / 2;
    double x, past_x, fx, fresult;
    double ete_err, ere_err;
    double fa = function_1_arg(expression, a);
    double fb = function_1_arg(expression, b);

    // set the seed for random number generator
    seed();

    while (iter &lt; maxiter) {
        // try maxiter times to find minimum in given interval [a, b] and return lowest result
        // update fresult with new result
        fresult = function_1_arg(expression, result);
        // choose a random starting point
        x = a + coefficient * zeroToOneUniformRandom();

        // set inner iter to zero before new loop
        innerIter = 0;
        // go in a loop to find a minimum with random starting point
        while (innerIter &lt; maxiter) {
            // calculate new x by subtracting the derivative of function at x multiplied by gamma from x
            past_x = x;
            x -= firstDerivative_1_arg(expression, x, DX) * gamma;
            fx = function_1_arg(expression, x);

            // calculate errors
            ete_err = fabs(past_x - x);
            ere_err = fabs(ete_err / x);

            if (mode) {
                printf("\nIn this iteration [#%d][#%d], x = %.5e f(x) = %.5e\n"
                       "and estimated true error = %.5e and estimated relative error = %.5e,\n",
                       iter, innerIter, x, fx, ete_err, ere_err);
            } // end if(mode)

            // Termination Criterion
            // if new x goes beyond interval lower than a
            if (x &lt; a) {
                if (mode) {
                    printf("\nIn this iteration the calculated x is less than a : %.5e &lt; %f"
                           "so minimum of the function occurs at a\n",
                           x, a);
                } // end if(mode)

                // if fa is lower than f(result), then a is where the minimum occurs
                if (fa &lt; fresult) {
                    result = a;
                } // end of if
                break;
            } // end of if

            // if new x goes beyond interval bigger than b
            if (x &gt; b) {
                if (mode) {
                    printf("\nIn this iteration the calculated x is bigger than b : %.5e &gt; %f"
                           "so minimum of the function occurs at b\n",
                           x, b);
                } // end if(mode)

                // if fb is lower than f(result), then b is where the minimum occurs
                if (fb &lt; fresult) {
                    result = b;
                } // end of if
                break;
            } // end of if

            // if calculated error is less than estimated true error threshold
            if (ete != 0 &amp;&amp; ete_err &lt; ete) {
                if (mode) {
                    printf("\nIn this iteration the calculated estimated true error is less than the threshold\n"
                           "(estimated true error) %.5e &lt; %.5e (threshold)\n"
                           "so the calculated x is the point on domain that minimum of the function happens\n",
                           ete_err, ete);
                } // end if(mode)

                // if fx is lower than f(result), then x is where the minimum occurs
                if (fx &lt; fresult) {
                    result = x;
                } // end of if
                break;
            } // end of estimated true error check

            // if calculated error is less than estimated relative error threshold
            if (ere != 0 &amp;&amp; ere_err &lt; ere) {
                if (mode) {
                    printf("\nIn this iteration the calculated estimated real error is less than the threshold\n"
                           "(estimated real error) %.5e &lt; %.5e (threshold)\n"
                           "so the calculated x is the point on domain that minimum of the function happens\n",
                           ere_err, ere);
                } // end if(mode)

                // if fx is lower than f(result), then x is where the minimum occurs
                if (fx &lt; fresult) {
                    result = x;
                } // end of if
                break;
            } // end of estimated relative error check
            innerIter++;
        } // end of inner while loop
        iter++;
    } // end of while loop

    // return result
    return result;
}
</code></pre>

<p>many functions here are may seem unknown to you, they are coded in separate files. you can see them at <a href="https://github.com/MahdiBaghbani/C-Math/tree/development" rel="nofollow noreferrer">my Github repository</a>.</p>