# -> {"_type": "foo", "verbose": False, ...}
evaluate_strings(*, prediction: str, reference: Optional[str] = None, input: Optional[str] = None, **kwargs: Any) → dict¶
Evaluate Chain or LLM output, based on optional input and label.
Parameters
prediction (str) – The LLM or chain prediction to evaluate.
reference (Optional[str], optional) – The reference label to evaluate against.
input (Optional[str], optional) – The input to consider during evaluation.
**kwargs – Additional keyword arguments, including callbacks, tags, etc.
Returns
The evaluation results containing the score or value.
Return type
dict
classmethod from_llm(llm: BaseLanguageModel, *, prompt: Optional[PromptTemplate] = None, criteria: Optional[Union[Mapping[str, str], Criteria, ConstitutionalPrinciple, str]] = None, normalize_by: Optional[float] = None, **kwargs: Any) → ScoreStringEvalChain[source]¶
Initialize the ScoreStringEvalChain from an LLM.
Parameters
llm (BaseChatModel) – The LLM to use (GPT-4 recommended).
prompt (PromptTemplate, optional) – The prompt to use.
**kwargs (Any) – Additional keyword arguments.
Returns
The initialized ScoreStringEvalChain.
Return type
ScoreStringEvalChain
Raises
ValueError – If the input variables are not as expected.
classmethod from_orm(obj: Any) → Model¶
classmethod from_string(llm: BaseLanguageModel, template: str) → LLMChain¶
Create LLMChain from LLM and template.
generate(input_list: List[Dict[str, Any]], run_manager: Optional[CallbackManagerForChainRun] = None) → LLMResult¶
Generate LLM result from inputs.