Create gen_api_answer.py

gen_api_answer.py

@@ -0,0 +1,51 @@
+ATLA_PROMPT = """You are tasked with evaluating a response based on a given instruction (which may contain an Input) and a scoring rubric that serve as the evaluation standard. Provide a comprehensive feedback on the response quality strictly adhering to the scoring rubric, without any general evaluation. Follow this with a score between 1 and 5, referring to the scoring rubric. Avoid generating any additional opening, closing, or explanations.
+  Here are some rules of the evaluation:
+  (1) You should prioritize evaluating whether the response satisfies the provided rubric. The basis of your score should depend exactly on the rubric. However, the response does not need to explicitly address points raised in the rubric. Rather, evaluate the response based on the criteria outlined in the rubric.
+
+  Your reply should strictly follow this format:
+  **Reasoning:** <Your feedback>
+
+  **Result:** <an integer between 1 and 5>
+
+  Here is the data:
+
+  Instruction:
+  ```
+  {human_input}
+  ```
+
+  Response:
+  ```
+  {ai_response}
+  ```
+
+  Score Rubrics:
+  {eval_criteria}"""
+
+ATLA_PROMPT_WITH_REFERENCE = """You are tasked with evaluating a response based on a given instruction (which may contain an Input) and a scoring rubric and reference answer that serve as the evaluation standard. Provide a comprehensive feedback on the response quality strictly adhering to the scoring rubric, without any general evaluation. Follow this with a score between 1 and 5, referring to the scoring rubric. Avoid generating any additional opening, closing, or explanations.
+
+  Here are some rules of the evaluation:
+  (1) You should prioritize evaluating whether the response satisfies the provided rubric. The basis of your score should depend exactly on the rubric. However, the response does not need to explicitly address points raised in the rubric. Rather, evaluate the response based on the criteria outlined in the rubric.
+
+  Your reply should strictly follow this format:
+  **Reasoning:** <Your feedback>
+
+  **Result:** <an integer between 1 and 5>
+
+  Here is the data:
+
+  Instruction:
+  ```
+  {human_input}
+  ```
+
+  Response:
+  ```
+  {ai_response}
+  ```
+
+  Score Rubrics:
+  {eval_criteria}
+
+  Reference answer:
+  {ground_truth_input}"""