ight into the kinds of mistakes the LLM is making.Most notably, the scores we're currently getting are pretty high, so it would make sense to pass in harder questions and be more specific in the judging criteria. This will give us more room to improve as it is sure that the system is not perfect.

While this evaluation approach serves as a solid foundation, it's worth noting that there are other frameworks available that can further enhance the evaluation process. Frameworks such as ragas, trulens, DeepEval, and UpTrain can be integrated with ZenML depending on your specific use-case and understanding of the underlying concepts. These frameworks, although potentially complex to set up and use, can provide more sophisticated evaluation capabilities as your project evolves and grows in complexity.

We now have a working evaluation of both the retrieval and generation evaluation components of our RAG pipeline. We can use this to track how our pipeline improves as we make changes to the retrieval and generation components.

Code Example

To explore the full code, visit the Complete Guide repository and for this section, particularly the eval_e2e.py file.

PreviousRetrieval evaluation

NextEvaluation in practice

Last updated 1 month ago