langchain.evaluation.parsing.json_schema.JsonSchemaEvaluator¶
class langchain.evaluation.parsing.json_schema.JsonSchemaEvaluator(**kwargs: Any)[source]¶
An evaluator that validates a JSON prediction against a JSON schema reference.
This evaluator checks if a given JSON prediction conforms to the provided JSON schema.
If the prediction is valid, the score is True (no errors). Otherwise, the score is False (error occurred).
requires_input¶
Whether the evaluator requires input.
Type
bool
requires_reference¶
Whether the evaluator requires reference.
Type
bool
evaluation_name¶
The name of the evaluation.
Type
str
Examples
evaluator = JsonSchemaEvaluator()
result = evaluator.evaluate_strings(
prediction=’{“name”: “John”, “age”: 30}’,
reference={
“type”: “object”,
“properties”: {
“name”: {“type”: “string”},
“age”: {“type”: “integer”}
}
}
)
assert result[“score”] is not None
Initializes the JsonSchemaEvaluator.
Parameters
**kwargs – Additional keyword arguments.
Raises
ImportError – If the jsonschema package is not installed.
Attributes
evaluation_name
Returns the name of the evaluation.
requires_input
Returns whether the evaluator requires input.
requires_reference
Returns whether the evaluator requires reference.
Methods
__init__(**kwargs)
Initializes the JsonSchemaEvaluator.
aevaluate_strings(*, prediction[, ...])
Asynchronously evaluate Chain or LLM output, based on optional input and label.
evaluate_strings(*, prediction[, reference, ...])
Evaluate Chain or LLM output, based on optional input and label.
__init__(**kwargs: Any) → None[source]¶
Initializes the JsonSchemaEvaluator.
Parameters
**kwargs – Additional keyword arguments.