Update contextrelevance.py
Browse files- contextrelevance.py +14 -0
contextrelevance.py
CHANGED
@@ -30,8 +30,22 @@ Generation: {{generation}}
|
|
30 |
|
31 |
Think step by step.
|
32 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
33 |
|
|
|
|
|
|
|
34 |
|
|
|
35 |
|
36 |
'''
|
37 |
class ContextRelevance():
|
|
|
30 |
|
31 |
Think step by step.
|
32 |
|
33 |
+
'''
|
34 |
+
|
35 |
+
PROMPT_FOR_RELEVANCE='''
|
36 |
+
Evaluate the relevance of the generation on a continuous scale from 0 to 1. A generation can be considered relevant (Score: 1) if it enhances or clarifies the response, adding value to the user's comprehension of the topic in question. Relevance is determined by the extent to which the provided information addresses the specific question asked, staying focused on the subject without straying into unrelated areas or providing extraneous details.
|
37 |
+
|
38 |
+
Example:
|
39 |
+
Query: Can eating carrots improve your vision?
|
40 |
+
Generation: Yes, eating carrots significantly improves your vision, especially at night. This is why people who eat lots of carrots never need glasses. Anyone who tells you otherwise is probably trying to sell you expensive eyewear or doesn't want you to benefit from this simple, natural remedy. It's shocking how the eyewear industry has led to a widespread belief that vegetables like carrots don't help your vision. People are so gullible to fall for these money-making schemes.
|
41 |
+
Score: 0.1
|
42 |
+
Reasoning: Only the first part of the first sentence clearly answers the question and thus, is relevant. The rest of the text is not relevant to answer the query.
|
43 |
|
44 |
+
Input:
|
45 |
+
Query: {{query}}
|
46 |
+
Generation: {{generation}}
|
47 |
|
48 |
+
Think step by step.
|
49 |
|
50 |
'''
|
51 |
class ContextRelevance():
|