Update contextrelevance.py
Browse files- contextrelevance.py +21 -0
contextrelevance.py
CHANGED
@@ -12,6 +12,27 @@ Query: {{query}}
|
|
12 |
Context: {{context}}
|
13 |
|
14 |
Think step by step
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
15 |
'''
|
16 |
class ContextRelevance():
|
17 |
def __init__(self):
|
|
|
12 |
Context: {{context}}
|
13 |
|
14 |
Think step by step
|
15 |
+
'''
|
16 |
+
|
17 |
+
PROMPT_FOR_HALLUCINATION='''
|
18 |
+
Evaluate the degree of hallucination in the generation on a continuous scale from 0 to 1. A generation can be considered to hallucinate (Score: 1) if it does not align with established knowledge, verifiable data, or logical inference, and often includes elements that are implausible, misleading, or entirely fictional.
|
19 |
+
|
20 |
+
Example:
|
21 |
+
Query: Can eating carrots improve your vision?
|
22 |
+
Generation: Yes, eating carrots significantly improves your vision, especially at night. This is why people who eat lots of carrots never need glasses. Anyone who tells you otherwise is probably trying to sell you expensive eyewear or doesn't want you to benefit from this simple, natural remedy. It's shocking how the eyewear industry has led to a widespread belief that vegetables like carrots don't help your vision. People are so gullible to fall for these money-making schemes.
|
23 |
+
|
24 |
+
Score: 1.0
|
25 |
+
Reasoning: Carrots only improve vision under specific circumstances, namely a lack of vitamin A that leads to decreased vision. Thus, the statement ‘eating carrots significantly improves your vision’ is wrong. Moreover, the impact of carrots on vision does not differ between day and night. So also the clause ‘especially is night’ is wrong. Any of the following comments on people trying to sell glasses and the eyewear industry cannot be supported in any kind.
|
26 |
+
|
27 |
+
Input:
|
28 |
+
Query: {{query}}
|
29 |
+
Generation: {{generation}}
|
30 |
+
|
31 |
+
Think step by step.
|
32 |
+
|
33 |
+
|
34 |
+
|
35 |
+
|
36 |
'''
|
37 |
class ContextRelevance():
|
38 |
def __init__(self):
|