langchain.evaluation.scoring.eval_chain.ScoreStringEvalChain¶
class langchain.evaluation.scoring.eval_chain.ScoreStringEvalChain[source]¶
Bases: StringEvaluator, LLMEvalChain, LLMChain
A chain for scoring on a scale of 1-10 the output of a model.
output_parser¶
The output parser for the chain.
Type
BaseOutputParser
Example
>>> from langchain.chat_models import ChatOpenAI
>>> from langchain.evaluation.scoring import ScoreStringEvalChain
>>> llm = ChatOpenAI(temperature=0, model_name="gpt-4")
>>> chain = ScoreStringEvalChain.from_llm(llm=llm)
>>> result = chain.evaluate_strings(
...     input = "What is the chemical formula for water?",
...     prediction = "H2O",
...     reference = "The chemical formula for water is H2O.",
... )
>>> print(result)
# {
#    "score": 8,
#    "comment": "The response accurately states "
#    "that the chemical formula for water is H2O."
#    "However, it does not provide an explanation of what the formula means."
# }
param callback_manager: Optional[BaseCallbackManager] = None¶
Deprecated, use callbacks instead.
param callbacks: Callbacks = None¶
Optional list of callback handlers (or callback manager). Defaults to None.
Callback handlers are called throughout the lifecycle of a call to a chain,
starting with on_chain_start, ending with on_chain_end or on_chain_error.
Each custom chain can optionally call additional callback methods, see Callback docs
for full details.
param criterion_name: str [Required]¶
The name of the criterion being evaluated.