<p>There can be confusion around this issue because heuristics mean different things in different contexts. So, let me talk about the different meaning of heuristics. Then we can return to evaluation functions.</p>

<p><strong>Single Agent Heuristic Search</strong></p>

<p>In single-agent heuristic search (eg A*, IDA*) heuristics are usually qualified with the words <em>admissible</em> or <em>consistent</em>. In this context heuristics are lower bounds on the cost to reach the goal. That is, they are the result of a function which returns a numerical value. If the heuristic is <em>admissible</em>, the value returned does not overestimate the true distance to the goal. If the heuristic is <em>consistent</em>, the heuristic between adjacent states never changes more than the edge cost. Consistent heuristics are admissible if the goal has a heuristic of 0, but not all admissible heuristics are consistent.</p>

<p>There are many properties proven on the combinations of heuristics and algorithms. A* and IDA* will find optimal paths with a consistent heuristic. A* is <em>optimal</em> in necessary node expansions with a consistent heuristic, but with a inconsistent heuristic A* can perform 2^N expansions of N states. (See <a href="https://www.movingai.com/SAS/INC/" rel="nofollow noreferrer">this demo</a> for an example of where this occurs.)</p>

<p><strong>Game Playing</strong></p>

<p>In game playing programs using algorithms like alpha-beta or Monte Carlo tree search (MCTS), heuristics represent approximations of the win/loss value of a game. For instance, the value might be scaled between -1 (loss) and +1 (win), with values in between representing uncertainty about the true value. Here there are no guarantees on underestimate or overestimation, but the better ordering of values (wins > draws > losses) the better the performance of the algorithms. Alpha-beta pruning will return the same result even if an affine transform is applied to the heuristic, because alpha-beta uses the relative ordering of values to search. See, <a href="https://arxiv.org/pdf/1406.0486.pdf" rel="nofollow noreferrer">this paper</a> for an example of heuristics in MCTS. Note that in this context a heuristic still has a numerical value.</p>

<p><strong>Optimization</strong></p>

<p>In search for optimization problems like SAT (satisfiability problems) or CSP (constraint satisfaction problems), algorithms are much more efficient if they can find good solutions quickly. Thus, instead of searching in a naive manner, they order their search in a way that is expected to be more efficient. If the ordering is good the search might be able to terminate earlier, but this isn't guaranteed. In this context a heuristic is a way of ordering choices that is likely to end up with finding a solution more quickly. (A satisfying assignment of variables in SAT or in a CSP.) <a href="https://arxiv.org/pdf/1401.4601.pdf" rel="nofollow noreferrer">Here is an example</a> of work that explores different ordering heuristics for these problems.</p>

<p>In this context a heuristic is used for ordering, so it doesn't have to be numerically based. If it is numerically based, the numbers would not necessarily have global meaning as heuristics do in other types of search. There are many, many variants of optimizations problems besides SAT and CSP where heuristics are used in this manner.</p>

<p><strong>Evaluation Function</strong></p>

<p>So, then, what is an evaluation function? It is probably most commonly used in the second context of games, where heuristic and evaluation function can be interchanged, but it is more generally referring to the numerical evaluation of a state. The primary point would be that an evaluation function is more specific than a heuristic function, as a heuristic has broad use in wider contexts.</p>
Difference between heuristic function and evaluation function
<p>I'm reading about searching algorithms and heuristic search and I am slightly confused about heuristic and evaluation functions. People seem to use them quite freely to describe seemingly same things. What am I missing?</p>