classmethod update_forward_refs(**localns: Any) → None¶
Try to update ForwardRefs on fields based on this Model, globalns and localns.
classmethod validate(value: Any) → Model¶
class Criteria[source]¶
Bases: SingleKeyEvalConfig
Configuration for a reference-free criteria evaluator.
Parameters
criteria (Optional[CRITERIA_TYPE]) – The criteria to evaluate.
llm (Optional[BaseLanguageModel]) – The language model to use for the evaluation chain.
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param criteria: Optional[Union[Mapping[str, str], langchain.evaluation.criteria.eval_chain.Criteria, langchain.chains.constitutional_ai.models.ConstitutionalPrinciple]] = None¶
param evaluator_type: langchain.evaluation.schema.EvaluatorType = EvaluatorType.CRITERIA¶
param input_key: Optional[str] = None¶
The key from the traced run’s inputs dictionary to use to represent the
input. If not provided, it will be inferred automatically.
param llm: Optional[langchain_core.language_models.base.BaseLanguageModel] = None¶
param prediction_key: Optional[str] = None¶
The key from the traced run’s outputs dictionary to use to
represent the prediction. If not provided, it will be inferred
automatically.
param reference_key: Optional[str] = None¶
The key in the dataset run to use as the reference string.
If not provided, we will attempt to infer automatically.
classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) → Model¶
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.