Compare predictions to a reference answer using string edit distances.
EXACT_MATCH = 'exact_match'¶
Compare predictions to a reference answer using exact matching.
REGEX_MATCH = 'regex_match'¶
Compare predictions to a reference answer using regular expressions.
PAIRWISE_STRING_DISTANCE = 'pairwise_string_distance'¶
Compare predictions based on string edit distances.
EMBEDDING_DISTANCE = 'embedding_distance'¶
Compare a prediction to a reference label using embedding distance.
PAIRWISE_EMBEDDING_DISTANCE = 'pairwise_embedding_distance'¶
Compare two predictions using embedding distance.
JSON_VALIDITY = 'json_validity'¶
Check if a prediction is valid JSON.
JSON_EQUALITY = 'json_equality'¶
Check if a prediction is equal to a reference JSON.
JSON_EDIT_DISTANCE = 'json_edit_distance'¶
Compute the edit distance between two JSON strings after canonicalization.
JSON_SCHEMA_VALIDATION = 'json_schema_validation'¶
Check if a prediction is valid JSON according to a JSON schema.
Examples using EvaluatorType¶
LangSmith Walkthrough
Criteria Evaluation