# -> {“_type”: “foo”, “verbose”: False, …}
evaluate(examples: List[dict], predictions: List[dict], question_key: str = 'query', context_key: str = 'context', prediction_key: str = 'result', *, callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None) → List[dict]¶
Evaluate question answering examples and predictions.
evaluate_strings(*, prediction: str, reference: Optional[str] = None, input: Optional[str] = None, **kwargs: Any) → dict¶
Evaluate Chain or LLM output, based on optional input and label.
Parameters
prediction (str) – The LLM or chain prediction to evaluate.
reference (Optional[str], optional) – The reference label to evaluate against.
input (Optional[str], optional) – The input to consider during evaluation.
**kwargs – Additional keyword arguments, including callbacks, tags, etc.
Returns
The evaluation results containing the score or value.
Return type
dict
classmethod from_llm(llm: BaseLanguageModel, prompt: Optional[PromptTemplate] = None, **kwargs: Any) → CotQAEvalChain[source]¶
Load QA Eval Chain from LLM.
classmethod from_orm(obj: Any) → Model¶
classmethod from_string(llm: BaseLanguageModel, template: str) → LLMChain¶
Create LLMChain from LLM and template.
generate(input_list: List[Dict[str, Any]], run_manager: Optional[CallbackManagerForChainRun] = None) → LLMResult¶
Generate LLM result from inputs.
invoke(input: Dict[str, Any], config: Optional[RunnableConfig] = None) → Dict[str, Any]¶